报错:
2018-09-04 00:05:05,718 (lifecycleSupervisor-1-1) [ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)] Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@253fe6bd counterGroup:{ name:null counters:{} } } - Exception follows.
com.aliyun.datahub.exception.AuthorizationFailureException: The signatrue is Time-Expired. (Status Code: 401; Error Code: Unauthorized; Request ID: 2018090316021120b99a0a737b3629)
at com.aliyun.datahub.model.serialize.JsonErrorParser.parse(JsonErrorParser.java:41)
at com.aliyun.datahub.model.serialize.GetProjectResultJsonDeser.deserialize(GetProjectResultJsonDeser.java:19)
at com.aliyun.datahub.model.serialize.GetProjectResultJsonDeser.deserialize(GetProjectResultJsonDeser.java:14)
at com.aliyun.datahub.DatahubClient.getProject(DatahubClient.java:123)
at com.aliyun.datahub.DatahubClient.getProject(DatahubClient.java:104)
at com.aliyun.datahub.wrapper.Project$Builder.build(Project.java:15)
at com.aliyun.datahub.wrapper.Project$Builder.build(Project.java:20)
at com.aliyun.datahub.flume.sink.DatahubWriter.<init>(DatahubWriter.java:55)
at com.aliyun.datahub.flume.sink.DatahubSink.start(DatahubSink.java:144)
at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:45)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
flume配置文件
a1.sources =r1
a1.sinks =k1
a1.channels =c1
a1.sources.r1.type =exec
a1.sources.r1.command =cat /opt/data/test_basic.log
a1.sinks.k1.type =com.aliyun.datahub.flume.sink.DatahubSink
a1.sinks.k1.datahub.accessID =明确是正确的ID
a1.sinks.k1.datahub.accessKey =明确是正确的accessKey
a1.sinks.k1.datahub.endPoint =http://dh-cn-hangzhou.aliyuncs.com
a1.sinks.k1.datahub.project =test_demo
a1.sinks.k1.datahub.topic =test_topic
a1.sinks.k1.batchSize =1
a1.sinks.k1.serializer =DELIMITED
a1.sinks.k1.serializer.delimiter =,
a1.sinks.k1.serializer.fieldnames =id,url,urlname,title,chset,scr,col,ip,country,province,city,county,clientDate
a1.sinks.k1.serializer.charset =UTF-8
a1.sinks.k1.shard.number =3
a1.sinks.k1.shard.maxTimeOut =60
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 1000
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
启动flume命令
bin/flume-ng agent -n a1 -c conf -f conf/datahub_basic.conf -Dflume.root.logger=INFO,console
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。