自行搭建k8s集群中遇到pod/ccm无法读取ccm token的问题

自己搭建的一个k8s集群，没有使用阿里云的ack服务。 根据github文档部署的ccm部分，但是文档中的镜像源registry-vpc.cn-hangzhou.aliyuncs.com/acs无法拉取镜像（后续测试公网域名可以使用）

部署情况：

kube镜像源: registry.aliyuncs.com/google_containers/ k8s version: 1.18.12 ccm version: v1.9.3.339-g9830b58-aliyun

遇到的问题： ccm pod 一直重启

# kubectl logs -n kube-system --tail 100 -f cloud-controller-manager-6hd6x

I1231 02:28:06.861965       1 clientmgr.go:140] wait for token ready
E1231 02:28:29.656303       1 clientmgr.go:116] token retrieve: failed to load ak, cloud config is nil
I1231 02:28:29.657715       1 clientmgr.go:140] wait for token ready
E1231 02:29:23.636956       1 clientmgr.go:116] token retrieve: failed to load ak, cloud config is nil
I1231 02:29:23.637005       1 clientmgr.go:140] wait for token ready
panic: token not ready timed out waiting for the condition

goroutine 1 [running]:
k8s.io/cloud-provider-alibaba-cloud/cloud-controller-manager.init.0.func1(0x0, 0x0, 0x7ffcbcb63ad7, 0x8, 0xc000118708, 0x1)
	/go/src/k8s.io/cloud-provider-alibaba-cloud/cloud-controller-manager/alicloud.go:159 +0x66a
k8s.io/cloud-provider.GetCloudProvider(0x7ffcbcb63ad7, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/go/pkg/mod/k8s.io/kubernetes/staging/src/k8s.io/cloud-provider@v0.0.0-20200325144952-9e991415386e/plugins.go:86 +0xcf
k8s.io/cloud-provider.InitCloudProvider(0x7ffcbcb63ad7, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/go/pkg/mod/k8s.io/kubernetes/staging/src/k8s.io/cloud-provider@v0.0.0-20200325144952-9e991415386e/plugins.go:134 +0x504
k8s.io/cloud-provider-alibaba-cloud/cmd/cloudprovider/app.(*ServerCCM).initialization(0xc00003c400, 0x3e, 0x0)
	/go/src/k8s.io/cloud-provider-alibaba-cloud/cmd/cloudprovider/app/ccm.go:154 +0x94
k8s.io/cloud-provider-alibaba-cloud/cmd/cloudprovider/app.(*ServerCCM).Start(0xc00003c400, 0xc000186070, 0x29cb492)
	/go/src/k8s.io/cloud-provider-alibaba-cloud/cmd/cloudprovider/app/ccm.go:189 +0x2f
k8s.io/cloud-provider-alibaba-cloud/cmd/cloudprovider/app.Run(0xc00003c400, 0xc0000ae000, 0x0)
	/go/src/k8s.io/cloud-provider-alibaba-cloud/cmd/cloudprovider/app/ccm.go:238 +0x43
main.main()
	/go/src/k8s.io/cloud-provider-alibaba-cloud/cmd/cloudprovider/cloudprovider-alibaba-cloud.go:48 +0xec

问题排查经过

1.根据文档配置的context，检查pod内context文件状况。 检查accesskey信息是否正确注入

文件存在 文件权限所有用户可读 变量注入成功

[root@cn-huhehaote ~]# kubectl exec -n kube-system -it cloud-controller-manager-6hd6x -- sh
/ # env | grep ACCESS
ACCESS_KEY_ID=LTAI4***************How9
ACCESS_KEY_SECRET=LjN2u********************C71y6
/ # ls -l /var/run/secrets/kubernetes.io/serviceaccount/token
lrwxrwxrwx    1 root     root            12 Dec 31 02:26 /var/run/secrets/kubernetes.io/serviceaccount/token -> ..data/token
/ # ls -l /var/run/secrets/kubernetes.io/serviceaccount/..data/token 
-rw-r--r--    1 root     root           978 Dec 31 02:26 /var/run/secrets/kubernetes.io/serviceaccount/..data/token
/ # cat /var/run/secrets/kubernetes.io/serviceaccount/..data/token
eyJhbGciOiJSUzI1NiIsImtpZCI6Im5aSklJYl9GdHNwMGFJQzVBUXdsdURIR09HWElsb2FTSnVPNExkZnF6UmcifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJjbG91ZC1jb250cm9sbGVyLW1hbmFnZXItdG9rZW4tNngyNmgiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiY2xvdWQtY29udHJvbGxlci1tYW5hZ2VyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiM2U1MDBhMWItZDkyNy00ODU0LWJmYjItZWI0NmFmZWEyZGEwIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmNsb3VkLWNvbnRyb2xsZXItbWFuYWdlciJ9.EfgAHK6xk2aCM6tDy3PUqMgBrTKY9xbUCqjHr5ooSLWFc2NEUsMGK7EmV2awv07n4_UqGMTXcAsH5l7HsgNa1COmGX8vgt1x3g6yFQkfw-ZBxZiHqIpr3-xfaoCXjD1Aa0peX6NsLZkNYq43mOQzIpwhevg5s9xst-_8drjudL1ZBn0LOycKXNIuAipRCXrFLIcDJ6VtZp7dFTCwYLOEr97o6kuNj12g-2KHyNvqjxvYsQPrlMMUKXcC1wOW_SuLdtlXabNWo1ZGE9cQFEJy2u3EPL_JWgo80qmq3KyTPB50Ge-SJC6563-xPV0kAjZDCq6sKQ2M7ClDF7lZp1e_2w

2.检查secret内容，与pod内的token文件做对比

文件内容一致

[root@cn-huhehaote ~]# echo $(kubectl get secret -n kube-system $(kubectl get sa -n kube-system cloud-controller-manager -o jsonpath={.secrets[0].name}) -o jsonpath={.data.token}) | base64 --decode
eyJhbGciOiJSUzI1NiIsImtpZCI6Im5aSklJYl9GdHNwMGFJQzVBUXdsdURIR09HWElsb2FTSnVPNExkZnF6UmcifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJjbG91ZC1jb250cm9sbGVyLW1hbmFnZXItdG9rZW4tNngyNmgiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiY2xvdWQtY29udHJvbGxlci1tYW5hZ2VyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiM2U1MDBhMWItZDkyNy00ODU0LWJmYjItZWI0NmFmZWEyZGEwIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmNsb3VkLWNvbnRyb2xsZXItbWFuYWdlciJ9.EfgAHK6xk2aCM6tDy3PUqMgBrTKY9xbUCqjHr5ooSLWFc2NEUsMGK7EmV2awv07n4_UqGMTXcAsH5l7HsgNa1COmGX8vgt1x3g6yFQkfw-ZBxZiHqIpr3-xfaoCXjD1Aa0peX6NsLZkNYq43mOQzIpwhevg5s9xst-_8drjudL1ZBn0LOycKXNIuAipRCXrFLIcDJ6VtZp7dFTCwYLOEr97o6kuNj12g-2KHyNvqjxvYsQPrlMMUKXcC1wOW_SuLdtlXabNWo1ZGE9cQFEJy2u3EPL_JWgo80qmq3KyTPB50Ge-SJC6563-xPV0kAjZDCq6sKQ2M7ClDF7lZp1e_2w