canal1.1.3使用dynamicTopic时ack失败导致zookeeper cursor不会
环境信息
canal version 1.1.3 mysql version 5.7.20
问题描述
使用Canal 1.1.3最新代码 ，启用dynamicTopic配置，使用zookeeper实现高可用 启动canal server一直报错：
2019-01-16 13:42:53.121 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:3 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:3 is not the firstly:1 2019-01-16 13:42:53.121 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:3 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:3 is not the firstly:1 2019-01-16 13:42:53.239 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:4 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:4 is not the firstly:1 2019-01-16 13:42:53.239 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:4 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:4 is not the firstly:1 2019-01-16 13:42:53.376 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:5 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:5 is not the firstly:1 2019-01-16 13:42:53.376 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:5 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:5 is not the firstly:1 2019-01-16 13:42:53.396 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:6 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:6 is not the firstly:1 2019-01-16 13:42:53.396 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:6 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:6 is not the firstly:1 2019-01-16 13:42:53.418 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:7 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:7 is not the firstly:1 2019-01-16 13:42:53.418 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:7 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:7 is not the firstly:1 2019-01-16 13:42:53.432 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:8 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:8 is not the firstly:1 2019-01-16 13:42:53.432 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:8 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:8 is not the firstly:1 2019-01-16 13:42:53.447 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:9 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:9 is not the firstly:1 2019-01-16 13:42:53.447 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:9 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:9 is not the firstly:1 2019-01-16 13:42:53.456 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:10 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:10 is not the firstly:1 2019-01-16 13:42:53.456 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:10 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:10 is not the firstly:1 2019-01-16 13:42:53.479 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:11 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:11 is not the firstly:1 2019-01-16 13:42:53.479 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:11 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:11 is not the firstly:1
ack失败导致zookeeper cursor不会更新
原提问者GitHub用户flackyang
写回答
-
-
在Canal 1.1.3中，动态topic的Zookeeper实现需要配置正确，并且所连接的Zookeeper集群正常运行。从你提供的信息来看，Canal Server启动时存在错误，建议根据错误提示，尝试以下方法解决问题：
-
检查Zookeeper集群是否正常可用。尝试连接Zookeeper集群并且使用命令行进行数据操作，例如获取Zookeeper节点的信息。如果Zookeeper连接存在问题，需要检查Zookeeper配置、网络连接和Zookeeper集群状态等，并且确保Canal Server所连接的Zookeeper集群能够正常运行。
-
检查dynamicTopic的Zookeeper配置参数是否正确。在Canal Server配置文件中，需要指定Zookeeper集群所在的IP地址、端口号等信息，确保Canal Server能够正确连接到Zookeeper集群。同时，检查Zookeeper的命名空间、父节点等信息是否设置正确。
-
确认Canal Server中dynamicTopic的相关配置是否正确。dynamicTopic的相关配置连接了Zookeeper集群，根据指定的数据源配置动态创建binlog的topic，可以动态监听binlog，且动态监听到的binlog会写入对应动态创建的topic中。在Canal Server配置文件中，需要使用类似下面的配置对dynamicTopic进行设置：
canal.instance.dynamicConfig=true canal.instance.zookeeper.address=127.0.0.1:2181 canal.instance.zookeeper.sessionTimeout=6000 canal.instance.zookeeper.connectTimeout= 6000 canal.instance.zookeeper.namespace=example canal.instance.zookeeper.config.mode=full- canal.instance.dynamicConfig=true: 表示开启Canal Server的动态配置特性。
- canal.instance.zookeeper.address: 表示Zookeeper的IP地址和端口号。
- canal.instance.zookeeper.sessionTimeout: 表示Zookeeper的会话超时时间，单位为毫秒。
- canal.instance.zookeeper.connectTimeout: 表示Zookeeper的连接超时时间，单位为毫秒。
- canal.instance.zookeeper.namespace: 表示Zookeeper的命名空间。
- canal.instance.zookeeper.config.mode: 表示Zookeeper的配置模式，使用full表示全量模式（每次启动配置都更新），使用lite表示简洁模式（只有当配置改变时才更新）。
2023-05-08 15:17:17赞同 展开评论 打赏 -
版权声明：本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。
