hiekay_个人页

把CSS、JS，等一些静态文件都合并加载

是的，性能的损失，hive支持通过类似sql语句的语法来操作hbase中的数据, 但是速度慢；

百亿

每条rowkey都要发起一次请求，这种方法效率十分低

    1) Servlet接口：提供最基本的管理Servlet生命周期的功能，主要有init（构造）、service（执行服务）、destroy（析构）三大功能，因此Servlet接口是任何Servlet类的生命和希望；    2) ServletConfig接口：提供最基本的查询Servlet配置信息的功能，主要有getServletName（获取当前Servlet注册名）、getInitParameter（获取Servlet初始参数）、getServletContext（获取Servlet环境句柄）等，但是它只是一个接口，因此里面没有ServletConfig对象，只是规定了该接口必须实现的功能；    3) GnericServlet类：它是个真正的类，它实现了Servlet和ServletConfig接口，并且它是所有具体Servlet的框架类，所有Servlet类都是该类的子类（都继承自它），包括HttpServlet类，并且它包含真正的数据对象ServletConfig，可以通过ServletConfig接口中的各种方法获取该对象中的信息（即Servlet配置信息）；启动一个Servlet程序的步骤：整个过程都是由Web容器控制的，实际中不实现Servlet类的构造器（构造器是空的），生命周期完全用init、service和destroy控制    1) 首先Web容器会读取web.xml配置信息并创建一个ServletConfig对象，将配置信息存在该对象中；    2) 接着Web容器调用Servlet类的空的构造函数构造一个空的Servlet类对象（里面什么都没有初始化）将其实例化；    3) 然后才是初始化，调用Servlet接口的init(config: ServletConfig)方法，将前面保存Servlet配置的ServletConfig对象作为参数传入init进行初始化，init中只有两句话：public void init(ServletConfig config) throws ServletException { this.config = config; this.init();}         i. 第一句当然是初始化config对象；         ii. 第二句不是调用基类的init函数，因为当前方法本身就是从Servlet接口继承来的方法，第二个无参的init方法是GenericServlet自己定义的方法，专门用来初始化自定义的数据成员（比如自己继承HttpServlet类实现了一个MyServlet类，里面自定义了一个数据成员String myName，而这样的数据成员的初始化就可以放在GenericServlet定义的无参init方法中；    4) 接受请求创建request和response对象并传入service进行服务；    5) 最后在一定条件下（关闭服务器、或主动关闭Servlet）调用destroy方法关闭并回收Servlet的空间；

socket 报错

一．移植Curl工具到Android环境步骤1.修改cURL源码下的mk文件。源码下面的Android.mk文件最后生成的是静态库libcurl.a，做如下修改(编译成动态库)。LOCAL_PRELINK_MODULE := false LOCAL_MODULE:= libcurl LOCAL_MODULE_TAGS := optional Copy the licence to a place where Android will find it. Actually, this doesn't quite work because the build system searches for NOTICE files before it gets to this point, so it will only be seen on subsequent builds. ALL_PREBUILT += $(LOCAL_PATH)/NOTICE $(LOCAL_PATH)/NOTICE: $(LOCAL_PATH)/COPYING | $(ACP) $(copy-file-to-target) include $(BUILD_STATIC_LIBRARY) include $(BUILD_SHARED_LIBRARY) 2.配置编译环境（cd 到Android.mk同一目录，直接在控制台输入下列代码或者把下面代码弄成sh脚本执行）红色部分根据自己源码情况ANDROID_HOME=/home/zhoulc/android/ && \ NDK_HOME=/home/zhoulc/android/ndk && \ PATH='$ANDROID_HOME/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin:$PATH' \ ./configure --host=arm-linux CC=arm-eabi-gcc --with-random=/dev/urandom \ CPPFLAGS='-I$NDK_HOME/platforms/android-8/arch-arm/usr/include \ -I $ANDROID_HOME/external/curl/include/ \ -I $ANDROID_HOME/external/curl/3rd/include \ -I $ANDROID_HOME/external/curl \ -I $ANDROID_HOME/out/target/product/generic/obj/STATIC_LIBRARIES/libcurl_intermediates \ -I $ANDROID_HOME/dalvik/libnativehelper/include/nativehelper \ -I $ANDROID_HOME/system/core/include \ -I $ANDROID_HOME/hardware/libhardware/include \ -I $ANDROID_HOME/hardware/libhardware_legacy/include \ -I $ANDROID_HOME/hardware/ril/include \ -I $ANDROID_HOME/dalvik/libnativehelper/include \ -I $ANDROID_HOME/frameworks/base/include \ -I $ANDROID_HOME/frameworks/base/opengl/include \ -I $ANDROID_HOME/frameworks/base/native/include \ -I $ANDROID_HOME/external/skia/include \ -I $ANDROID_HOME/out/target/product/generic/obj/include \ -I $ANDROID_HOME/bionic/libc/arch-arm/include \ -I $ANDROID_HOME/bionic/libc/include \ -I $ANDROID_HOME/bionic/libstdc++/include \ -I $ANDROID_HOME/bionic/libc/kernel/common \ -I $ANDROID_HOME/bionic/libc/kernel/arch-arm \ -I $ANDROID_HOME/bionic/libm/include \ -I $ANDROID_HOME/bionic/libm/include/arch/arm \ -I $ANDROID_HOME/bionic/libthread_db/include \ -include $ANDROID_HOME/system/core/include/arch/linux-arm/AndroidConfig.h \ -I $ANDROID_HOME/system/core/include/arch/linux-arm/ \ -D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__ -DANDROID -DNDEBUG -DNDEBUG -DHAVE_CONFIG_H' \ CFLAGS='-fno-exceptions -Wno-multichar -msoft-float -fpic -ffunction-sections \ -funwind-tables -fstack-protector -Wa,--noexecstack -Werror=format-security \ -fno-short-enums -march=armv5te -mtune=xscale -Wno-psabi -mthumb-interwork \ -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith \ -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point \ -g -Wstrict-aliasing=2 -finline-functions -fno-inline-functions-called-once \ -fgcse-after-reload -frerun-cse-after-loop -frename-registers -UDEBUG \ -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 \ -Wpointer-arith -Wwrite-strings -Wunused -Winline -Wnested-externs \ -Wmissing-declarations -Wmissing-prototypes -Wno-long-long -Wfloat-equal \ -Wno-multichar -Wsign-compare -Wno-format-nonliteral -Wendif-labels \ -Wstrict-prototypes -Wdeclaration-after-statement -Wno-system-headers' \ LIBS='-nostdlib -Bdynamic -Wl,-T,$ANDROID_HOME/build/core/armelf.x \ -Wl,-dynamic-linker,/system/bin/linker -Wl,--gc-sections -Wl,-z,nocopyreloc \ -L$ANDROID_HOME/out/target/product/generic/obj/lib -Wl,-z,noexecstack \ -Wl,-rpath-link=$ANDROID_HOME/out/target/product/generic/obj/lib \ -lc -llog -lcutils -lstdc++ \ -Wl,--no-undefined $ANDROID_HOME/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/lib/gcc/arm-eabi/4.4.0/libgcc.a \ $ANDROID_HOME/out/target/product/generic/obj/lib/crtend_android.o \ -lm $ANDROID_HOME/out/target/product/generic/obj/lib/crtbegin_dynamic.o \ -L$ANDROID_HOME/external/curl/3rd/libs'3.编译libcurl.so库cd进入android/external/curl源码目录 mm-》编译生成libcurl.so库 4.编写测试case 以及Android.mk文件并生成可执行文件新建一个测试案例curl_test.cpp include 'curl/curl.h' include ; int main() { CURL *curl; CURLcode res; curl_global_init(CURL_GLOBAL_ALL); curl = curl_easy_init(); if (curl) { curl_easy_setopt(curl, CURLOPT_URL, 'http://www.baidu.com/'); res = curl_easy_perform(curl); if (0!=res) { printf('curl error: %d\n', res); } curl_easy_cleanup(curl); } curl_global_cleanup(); return 0; }在同一目录下写一个Android.mk文件生成curl_test可执行文件 LOCAL_PATH := $(call my-dir) include $(CLEAR_VARS) LOCAL_C_INCLUDES += \ $(TOP)/external/curl/include/ \ LOCAL_SRC_FILES:= curl_test.cpp No shared libraries. No static libraries. LOCAL_SHARED_LIBRARIES := libcurl LOCAL_MODULE_TAGS := optional LOCAL_MODULE := curl_test include $(BUILD_EXECUTABLE) 生成可执行文件：curl_test 4.运行查看测试结果运行测试case：curl_test 5.(补充)移植libcurl到android4.0，修改两个地方1）把生成的路径改一下，一般默认为out/target/product/generic下面，我们根据系统不同(根据lunch选择不同，最终生成的路径不一样)改为系统的全局变量， 把$ANDROID_HOME/out/target/product/generic替换成$ANDROID_PRODUCT_OUT。 ANDROID_HOME_CURL=../../ && \ NDK_HOME_CURL=../../prebuilt/ndk && \ PATH='$ANDROID_HOME_CURL/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin:$PATH' \ ./configure --host=arm-linux CC=gcc --with-random=/dev/urandom \ CPPFLAGS='-I$NDK_HOME_CURL/platforms/android-8/arch-arm/usr/include \ -I $ANDROID_HOME_CURL/external/curl/include/ \ -I $ANDROID_HOME_CURL/external/curl/3rd/include \ -I $ANDROID_HOME_CURL/external/curl \ -I $ANDROID_HOME_CURL/out/target/product/generic/obj/STATIC_LIBRARIES/libcurl_intermediates \ -I $ANDROID_HOME_CURL/dalvik/libnativehelper/include/nativehelper \ -I $ANDROID_HOME_CURL/system/core/include \ -I $ANDROID_HOME_CURL/hardware/libhardware/include \ -I $ANDROID_HOME_CURL/hardware/libhardware_legacy/include \ -I $ANDROID_HOME_CURL/hardware/ril/include \ -I $ANDROID_HOME_CURL/dalvik/libnativehelper/include \ -I $ANDROID_HOME_CURL/frameworks/base/include \ -I $ANDROID_HOME_CURL/frameworks/base/opengl/include \ -I $ANDROID_HOME_CURL/frameworks/base/native/include \ -I $ANDROID_HOME_CURL/external/skia/include \ -I $ANDROID_HOME_CURL/out/target/product/generic/obj/include \ -I $ANDROID_HOME_CURL/bionic/libc/arch-arm/include \ -I $ANDROID_HOME_CURL/bionic/libc/include \ -I $ANDROID_HOME_CURL/bionic/libstdc++/include \ -I $ANDROID_HOME_CURL/bionic/libc/kernel/common \ -I $ANDROID_HOME_CURL/bionic/libc/kernel/arch-arm \ -I $ANDROID_HOME_CURL/bionic/libm/include \ -I $ANDROID_HOME_CURL/bionic/libm/include/arch/arm \ -I $ANDROID_HOME_CURL/bionic/libthread_db/include \ -include $ANDROID_HOME_CURL/system/core/include/arch/linux-arm/AndroidConfig.h \ -I $ANDROID_HOME_CURL/system/core/include/arch/linux-arm/ \ -D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__ -DANDROID -DNDEBUG -DNDEBUG -DHAVE_CONFIG_H' \ CFLAGS='-fno-exceptions -Wno-multichar -msoft-float -fpic -ffunction-sections \ -funwind-tables -fstack-protector -Wa,--noexecstack -Werror=format-security \ -fno-short-enums -march=armv5te -mtune=xscale -Wno-psabi -mthumb-interwork \ -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith \ -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point \ -g -Wstrict-aliasing=2 -finline-functions -fno-inline-functions-called-once \ -fgcse-after-reload -frerun-cse-after-loop -frename-registers -UDEBUG \ -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 \ -Wpointer-arith -Wwrite-strings -Wunused -Winline -Wnested-externs \ -Wmissing-declarations -Wmissing-prototypes -Wno-long-long -Wfloat-equal \ -Wno-multichar -Wsign-compare -Wno-format-nonliteral -Wendif-labels \ -Wstrict-prototypes -Wdeclaration-after-statement -Wno-system-headers' \ LIBS='-nostdlib -Bdynamic -Wl,-T,$ANDROID_HOME_CURL/build/core/armelf.x \ -Wl,-dynamic-linker,/system/bin/linker -Wl,--gc-sections -Wl,-z,nocopyreloc \ -L$ANDROID_PRODUCT_OUT/obj/lib -Wl,-z,noexecstack \ -Wl,-rpath-link=$ANDROID_PRODUCT_OUT/obj/lib \ -lc -llog -lcutils -lstdc++ \ -Wl,--no-undefined $ANDROID_HOME_CURL/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/lib/gcc/arm-eabi/4.4.0/libgcc.a \ $ANDROID_PRODUCT_OUT/obj/lib/crtend_android.o \ -lm $ANDROID_PRODUCT_OUT/obj/lib/crtbegin_dynamic.o \ -L$ANDROID_HOME_CURL/external/curl/3rd/libs'2）修改Android.mk ALL_PREBUILT += $(LOCAL_PATH)/NOTICE $(LOCAL_PATH)/NOTICE: $(LOCAL_PATH)/COPYING | $(ACP) $(copy-file-to-target) 把关于ALL_PREBUILT模块全部注释调

没有设置好吧 ，看一下说明

 云HBase团队为大家提供了一个github项目供大家参考使用上面的三种方式来开发Spark分析HBase的程序，项目地址; https://github.com/lw309637554/alicloud-hbase-spark-examples?spm=a2c4e.11153940.blogcont573569.14.320377b4U14MDa依赖项：需要下载云HBase及云Phoenix的client包分析HFILE：需要先开通云HBase的HDFS访问权限，参考文档在hbase shell中对表生成snapshot表“snapshot 'sourceTable', ‘snapshotName'”在项目中配置自己的hdfs-sit.xml文件，然后通过直读HDFS的方式分析snapshot表具体的exampleRDD API对应：org.apache.spark.hbase.NativeRDDAnalyzeSQL API对应：org.apache.spark.sql.execution.datasources.hbase.SqlAnalyze分析HFILE对应：org.apache.spark.hfile.SparkAnalyzeHFILE

等待事件占总的call time的比率

springX

你应该说清楚 环境， 版本 ，问题 详细说清楚

在哪里？配置

权限呢

第一步：MAVEN配置    org.apache.sparkspark-core_2.111.6.0org.apache.sparkspark-mllib_2.111.6.0  org.apache.spark  spark-sql_2.11   1.6.0org.scala-langscala-library2.11.0org.scala-langscala-compiler2.11.0org.scala-langscala-reflect2.11.0 第二步：Spring配置                                                          class='org.springframework.beans.factory.config.PropertyPlaceholderConfigurer'> -->                  classpath:jdbc.properties           classpath:spark.properties                     第三步：新增属性文件  spark.properties spark.master=localspark.url=jdbc:mysql://192.168.0.202:3306/spark?useUnicode=true&characterEncoding=UTF-8spark.table=testtablespark.username=rootspark.password=mysql  第四步：写代码 /** *  */package com.harleycorp.service.impl; import java.util.ArrayList;import java.util.List;import java.util.Properties; import javax.annotation.Resource; import org.apache.log4j.Logger;import org.apache.spark.api.java.JavaRDD;import org.apache.spark.api.java.JavaSparkContext;import org.apache.spark.sql.DataFrame;import org.apache.spark.sql.SQLContext;import org.apache.spark.sql.SaveMode;import org.springframework.beans.factory.annotation.Value;import org.springframework.stereotype.Service; import com.harleycorp.pojo.SparkUser;import com.harleycorp.service.ISparkUpperService; /** * @author kevin * */@Servicepublic class SparkUpperServiceImpl implements ISparkUpperService { private Logger logger =Logger.getLogger(SparkUpperServiceImpl.class); @Value('${spark.master}')public String master ; // = 'local' @Value('${spark.url}')public String url ;//= 'jdbc:mysql://192.168.0.202:3306/spark?useUnicode=true&characterEncoding=UTF-8'; @Value('${spark.table}')public String table ; //= 'testtable' @Value('${spark.username}')public String username ;// = 'root'; //@Value('${spark.password}')public String password = 'mysql'; @Resourcepublic SQLContext sqlContext; @Resourcepublic JavaSparkContext sc; public Properties getConnectionProperties(){Properties connectionProperties = new Properties();connectionProperties.setProperty('dbtable',table);connectionProperties.setProperty('user',username);//数据库用户connectionProperties.setProperty('password',password); //数据库用户密码return connectionProperties;} public String query() {logger.info('=======================this url:'+this.url);logger.info('=======================this table:'+this.table);logger.info('=======================this master:'+this.master);logger.info('=======================this username:'+this.username);logger.info('=======================this password:'+this.password); DataFrame df = null;//以下数据库连接内容请使用实际配置地址代替df = sqlContext.read().jdbc(url,table, getConnectionProperties());df.registerTempTable(table);String result = sqlContext.sql('select * from testtable').javaRDD().collect().toString();logger.info('=====================spark mysql:'+result);return result;} public String queryByCon(){logger.info('=======================this url:'+this.url);logger.info('=======================this table:'+this.table);logger.info('=======================this master:'+this.master);logger.info('=======================this username:'+this.username);logger.info('=======================this password:'+this.password); DataFrame df = sqlContext.read().jdbc(url, table, new String[]{'password=000000'}, getConnectionProperties());String result = df.collectAsList().toString();logger.info('=====================spark mysql:'+result);return null;} public void add(){List list = new ArrayList();SparkUser us = new SparkUser();us.setUsername('kevin');us.setPassword('000000');list.add(us);SparkUser us2 = new SparkUser();us2.setUsername('Lisa');us2.setPassword('666666');list.add(us2); JavaRDD personsRDD = sc.parallelize(list);DataFrame userDf = sqlContext.createDataFrame(personsRDD, SparkUser.class);userDf.write().mode(SaveMode.Append).jdbc(url, table, getConnectionProperties());} } 第五步：junit调用 package com.harleycorp.testmybatis; import javax.annotation.Resource; import org.apache.log4j.Logger;import org.junit.Test;import org.junit.runner.RunWith;import org.springframework.test.context.ContextConfiguration;import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import com.harleycorp.service.ISparkUpperService; @RunWith(SpringJUnit4ClassRunner.class) //表示继承了SpringJUnit4ClassRunner类@ContextConfiguration(locations = {'classpath:spring-mybatis.xml'}) public class TestSpark { private static Logger logger=Logger.getLogger(TestSpark.class);@Resource private  ISparkUpperService sparkUpperService = null; @Testpublic void test1(){sparkUpperService.query();} @Testpublic void test2(){  sparkUpperService.add();   } @Testpublic void test3(){sparkUpperService.queryByCon();}} 第六步：运行

Phoenix

一.准备阶段 1.准备2套能正常运行的hbase集群(new cluster:222|oldcluster:226) 2.2套集群的hosts文件内容都需要包含对方的主机地址 3.zookeeper可以单独部署2个集群，也可用一个zookeeper集群管理2套hbase集群，就是不能用hbase自带的zookeeper集群做管理 4.hadoop、hbase等组件版本号保持一致 二.配置阶段 1.修改222集群的hbase-site.xml文件 添加如下内容： hbase.replication true 2.在222集群上添加peer ./hbase shell add_peer'1','172.16.205.226:2181:/hbase' 执行该命令会报错，但是不影响执行结果，如果不想让其有报错提示，可进入zookeeper将peerid删除，再执行此命令就行了 3.启动复制 ./hbase shellstart_replication 执行该命令也会报错，不予理会。若想看看状态是否被开启，同样进入zookeeper查看state 4.创建表 在两套集群创建同样的表(结构需要完全一样) 5.在226上添加replication属性，并刷新其结构 disable 'your_table' alter 'your_table', {NAME =>'family_name', REPLICATION_SCOPE => '1'} enable 'your_table' 6.测试数据同步 在226上put一条数据进hbase 222上将能在随后被scan到

Spark读取HBase数据分析 并将结果存入HBase分析结果表 hbase只是数据