canal.adapter控制台可以打印出dml日志,但就是同步不到ES,我在mysql数据库做增删改查操作数据都可以读取到日志,但就是不会同步到ES索引,下面是application.yml配置 :
server: port: 8081 spring: jackson: date-format: yyyy-MM-dd HH:mm:ss time-zone: GMT+8 default-property-inclusion: non_null
canal.conf: mode: tcp #tcp kafka rocketMQ rabbitMQ flatMessage: true zookeeperHosts: syncBatchSize: 1000 retries: -1 timeout: accessKey: secretKey: consumerProperties:
canal.tcp.server.host: 127.0.0.1:11111 canal.tcp.zookeeper.hosts: canal.tcp.batch.size: 500 canal.tcp.username: canal.tcp.password:
kafka.bootstrap.servers: 127.0.0.1:9092 kafka.enable.auto.commit: false kafka.auto.commit.interval.ms: 1000 kafka.auto.offset.reset: latest kafka.request.timeout.ms: 40000 kafka.session.timeout.ms: 30000 kafka.isolation.level: read_committed kafka.max.poll.records: 1000
rocketmq.namespace: rocketmq.namesrv.addr: 127.0.0.1:9876 rocketmq.batch.size: 1000 rocketmq.enable.message.trace: false rocketmq.customized.trace.topic: rocketmq.access.channel: rocketmq.subscribe.filter:
rabbitmq.host: rabbitmq.virtual.host: rabbitmq.username: rabbitmq.password: rabbitmq.resource.ownerId:
srcDataSources: pprsPermitDS: url: jdbc:mysql://127.0.0.1:3306/canal-test?useUnicode=true username: canal password: canal canalAdapters:
instance: example # canal instance Name or mq topic name groups: groupId: g1 outerAdapters: name: logger name: es7 hosts: http://127.0.0.1:9200 properties: mode: rest cluster.name: elasticsearch 下面是es7目录下的映射文件product.yml :
dataSourceKey: pprsPermitDS destination: example groupId: g1 esMapping: _index: canalproduct _id: _id _type: doc upsert: true sql: "SELECT p.id as _id, p.title, p.subtitle, p.pic FROMproduct p" commitBatch: 3000
下面是canal.deployer/conf/example/instance.properties配置:
canal.instance.gtidon=false
canal.instance.master.address=127.0.0.1:3306 canal.instance.master.journal.name= canal.instance.master.position= canal.instance.master.timestamp= canal.instance.master.gtid=
canal.instance.rds.accesskey= canal.instance.rds.secretkey= canal.instance.rds.instanceId=
canal.instance.defaultDatabaseName = canal-test
canal.instance.dbUsername=canal canal.instance.dbPassword=canal canal.instance.connectionCharset = UTF-8
canal.instance.enableDruid=false
canal.instance.filter.regex=...
canal.instance.filter.black.regex=
canal.mq.topic=example canal.mq.partition=0
原提问者GitHub用户haifengkok
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。