环境信息 canal version

deployer : 1.1.6 adapter : 1.1.6

mysql version

mysql : 8.0.13 elasticsearch : 7.17.3

问题描述

实现mysql 8的*库的xxx表增量同步至es7索引xxx

步骤重现

adapter application.yml部分配置：

srcDataSources: dbconfig1: url: jdbc:mysql://...:/?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=UTC username: * password: * canalAdapters: - instance: example # canal instance Name or mq topic name groups: - groupId: g1 outerAdapters: - name: logger - name: es7 hosts: http://...:* # 127.0.0.1:9200 for rest mode properties: mode: rest # or rest security.auth: : # only used for rest mode cluster.name: *

增量同步表的配置文件es7/xxx.yml：

dataSourceKey: dbconfig1 destination: example groupId: g1 esMapping: _index: xxx _id: id upsert: true

pk: id

sql: "select id,alias,plus_code,kexin_plus_code,code,code_sqjw,is_settle,is_temp,is_build,is_house,has_monitoring,level,name,full_name,landmark,parent_id,parent_ids,start_date,stop_date,type,village_city_type,weight,clazz,last_update_coordinate_time,remarks,del_flag,sort,temp,old_id from xxx" commitBatch: 1500

修改xxx表一条数据

实际执行情况

日志出现NullPointerException异常

2022-05-26 15:43:26.214 [pool-2-thread-1] INFO c.a.o.canal.client.adapter.logger.LoggerAdapterExample - DML: {"data":[{"id":"vvvvvvvvvvvvvvvvvv","alias":"vvvvv3aa","center_latitude":null,"center_longitude":null,"plus_code":null,"range_points":null,"kexin_center_latitude":null,"kexin_center_longitude":null,"kexin_plus_code":null,"kexin_range_points":null,"code":null,"code_sqjw":null,"is_settle":null,"is_temp":null,"is_build":null,"is_house":null,"has_monitoring":0,"level":0,"name":"vvvv","full_name":"vvvvvvvvvvvvvvvvvvvvvvvvvvvvvv","landmark":"","parent_id":"0","parent_ids":"0,","start_date":null,"stop_date":null,"type":12,"village_city_type":null,"weight":null,"clazz":null,"last_update_coordinate_time":null,"create_by":"1","create_date":1653550251000,"update_by":"1","update_date":1653550256000,"remarks":null,"del_flag":"1","sort":null,"temp":null,"old_id":null}],"database":"","destination":"example","es":1653551004000,"groupId":"g1","isDdl":false,"old":[{"alias":"vvvvv3a"}],"pkNames":["id"],"sql":"","table":"xxx","ts":1653551006057,"type":"UPDATE"} 2022-05-26 15:43:26.215 [pool-2-thread-1] ERROR c.a.o.canal.client.adapter.es.core.service.ESSyncService - sync error, es index: xxx, DML : Dml{destination='example', database='', table='xxx', type='UPDATE', es=1653551004000, ts=1653551006057, sql='', data=[{id=vvvvvvvvvvvvvvvvvv, alias=vvvvv3aa, center_latitude=null, center_longitude=null, plus_code=null, range_points=null, kexin_center_latitude=null, kexin_center_longitude=null, kexin_plus_code=null, kexin_range_points=null, code=null, code_sqjw=null, is_settle=null, is_temp=null, is_build=null, is_house=null, has_monitoring=0, level=0, name=vvvv, full_name=vvvvvvvvvvvvvvvvvvvvvvvvvvvvvv, landmark=, parent_id=0, parent_ids=0,, start_date=null, stop_date=null, type=12, village_city_type=null, weight=null, clazz=null, last_update_coordinate_time=null, create_by=1, create_date=2022-05-26 15:30:51.0, update_by=1, update_date=2022-05-26 15:30:56.0, remarks=null, del_flag=1, sort=null, temp=null, old_id=null}], old=[{alias=vvvvv3a}]} 2022-05-26 15:43:26.220 [pool-2-thread-1] ERROR c.a.otter.canal.adapter.launcher.loader.AdapterProcessor - java.lang.NullPointerException java.lang.RuntimeException: java.lang.NullPointerException at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:112) ~[na:na] at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:60) ~[na:na] at com.alibaba.otter.canal.client.adapter.es.core.ESAdapter.sync(ESAdapter.java:104) ~[na:na] at com.alibaba.otter.canal.client.adapter.es.core.ESAdapter.sync(ESAdapter.java:83) ~[na:na] at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.batchSync(AdapterProcessor.java:140) ~[client-adapter.launcher-1.1.6.jar:na] at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.lambda$null$1(AdapterProcessor.java:98) ~[client-adapter.launcher-1.1.6.jar:na] at java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:891) ~[na:1.8.0_242] at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.lambda$null$2(AdapterProcessor.java:95) ~[client-adapter.launcher-1.1.6.jar:na] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_242] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_242] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_242] at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_242] Caused by: java.lang.NullPointerException: null at com.alibaba.otter.canal.client.adapter.es7x.support.ES7xTemplate.getESDataFromDmlData(ES7xTemplate.java:321) ~[na:na] at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.singleTableSimpleFiledUpdate(ESSyncService.java:814) ~[na:na] at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.update(ESSyncService.java:208) ~[na:na] at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:97) ~[na:na] ... 11 common frames omitted 2022-05-26 15:43:26.220 [Thread-4] ERROR c.a.otter.canal.adapter.launcher.loader.AdapterProcessor - Outer adapter sync failed! Error sync and rollback, execute times: 1

原提问者GitHub用户yuyoyth