环境:Ubuntu 16.04 jdk:1.8 canal环境:一个canal server,两个instance, canal版本:canal.kafka-1.1.0
出现问题原因:
1、删除zk上/otter znode 目的:重置instance的binlog file和Position
详细日志: 2018-10-23 16:05:48.880 [pool-6-thread-2] ERROR com.alibaba.otter.canal.kafka.producer.CanalKafkaStarter - process error! java.lang.NullPointerException: null at com.alibaba.otter.canal.store.memory.MemoryEventStoreWithBuffer.calculateSize(MemoryEventStoreWithBuffer.java:555) ~[canal.store-1.1.0.jar:na] at com.alibaba.otter.canal.store.memory.MemoryEventStoreWithBuffer.doGet(MemoryEventStoreWithBuffer.java:322) ~[canal.store-1.1.0.jar:na] at com.alibaba.otter.canal.store.memory.MemoryEventStoreWithBuffer.tryGet(MemoryEventStoreWithBuffer.java:261) ~[canal.store-1.1.0.jar:na] at com.alibaba.otter.canal.server.embedded.CanalServerWithEmbedded.getEvents(CanalServerWithEmbedded.java:478) ~[canal.server-1.1.0.jar:na] at com.alibaba.otter.canal.server.embedded.CanalServerWithEmbedded.getWithoutAck(CanalServerWithEmbedded.java:310) ~[canal.server-1.1.0.jar:na] at com.alibaba.otter.canal.server.embedded.CanalServerWithEmbedded.getWithoutAck(CanalServerWithEmbedded.java:273) ~[canal.server-1.1.0.jar:na] at com.alibaba.otter.canal.kafka.producer.CanalKafkaStarter.worker(CanalKafkaStarter.java:121) [canal.kafka-1.1.0.jar:na] at com.alibaba.otter.canal.kafka.producer.CanalKafkaStarter.access$000(CanalKafkaStarter.java:26) [canal.kafka-1.1.0.jar:na] at com.alibaba.otter.canal.kafka.producer.CanalKafkaStarter$1.run(CanalKafkaStarter.java:70) [canal.kafka-1.1.0.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_181] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_181] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181] 2018-10-23 16:05:48.881 [pool-6-thread-2] INFO com.alibaba.otter.canal.kafka.producer.CanalKafkaStarter - ## the canal consumer user_cards is running now ...... 2018-10-23 16:05:48.881 [pool-6-thread-2] ERROR com.alibaba.otter.canal.kafka.producer.CanalKafkaStarter - process error! java.lang.NullPointerException: null at com.alibaba.otter.canal.store.memory.MemoryEventStoreWithBuffer.calculateSize(MemoryEventStoreWithBuffer.java:555) ~[canal.store-1.1.0.jar:na] at com.alibaba.otter.canal.store.memory.MemoryEventStoreWithBuffer.doGet(MemoryEventStoreWithBuffer.java:322) ~[canal.store-1.1.0.jar:na] at com.alibaba.otter.canal.store.memory.MemoryEventStoreWithBuffer.tryGet(MemoryEventStoreWithBuffer.java:261) ~[canal.store-1.1.0.jar:na] at com.alibaba.otter.canal.server.embedded.CanalServerWithEmbedded.getEvents(CanalServerWithEmbedded.java:478) ~[canal.server-1.1.0.jar:na] at com.alibaba.otter.canal.server.embedded.CanalServerWithEmbedded.getWithoutAck(CanalServerWithEmbedded.java:310) ~[canal.server-1.1.0.jar:na] at com.alibaba.otter.canal.server.embedded.CanalServerWithEmbedded.getWithoutAck(CanalServerWithEmbedded.java:273) ~[canal.server-1.1.0.jar:na] at com.alibaba.otter.canal.kafka.producer.CanalKafkaStarter.worker(CanalKafkaStarter.java:121) [canal.kafka-1.1.0.jar:na] at com.alibaba.otter.canal.kafka.producer.CanalKafkaStarter.access$000(CanalKafkaStarter.java:26) [canal.kafka-1.1.0.jar:na] at com.alibaba.otter.canal.kafka.producer.CanalKafkaStarter$1.run(CanalKafkaStarter.java:70) [canal.kafka-1.1.0.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_181] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_181] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
我暴力的删除的了zk中的znode,重启canal的时候他会重新加载instance和canal server的配置么?
原提问者GitHub用户1241407808
用最新的v1.1.1的版本, canal.deployer部署一下kafka的同步, 已知问题修复了. ps. 新版本已经合并了canal.kafka工程到canal.deployer中, 主要修改一下canal.properties里的canal.serverMode=kafka即可
原回答者GitHub用户agapple
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。