flink基于k8s安装,hadoop原生安装集群,将任务放到flink上执行的时候,报错不能使用checkpoint,详细报错如下: Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find a file system implementation for scheme 'hdfs'. The scheme is not directly supported by Flink and no Hadoop file system to support this scheme could be loaded. For a full list of supported file systems, please see https://ci.apache.org/projects/flink/flink-docs-stable/ops/filesystems/. at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:530) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:407) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.core.fs.Path.getFileSystem(Path.java:274) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.runtime.state.filesystem.FsCheckpointStorageAccess. (FsCheckpointStorageAccess.java:64) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.runtime.state.filesystem.FsStateBackend.createCheckpointStorage(FsStateBackend.java:527) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.streaming.runtime.tasks.StreamTask. (StreamTask.java:337) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.streaming.runtime.tasks.StreamTask. (StreamTask.java:304) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.streaming.runtime.tasks.SourceStreamTask. (SourceStreamTask.java:76) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.streaming.runtime.tasks.SourceStreamTask. (SourceStreamTask.java:72) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_292] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_292] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_292] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_292] at org.apache.flink.runtime.taskmanager.Task.loadAndInstantiateInvokable(Task.java:1515) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:727) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_292] Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Hadoop is not in the classpath/dependencies. at org.apache.flink.core.fs.UnsupportedSchemeFactory.create(UnsupportedSchemeFactory.java:55) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:526) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:407) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.core.fs.Path.getFileSystem(Path.java:274) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.runtime.state.filesystem.FsCheckpointStorageAccess. (FsCheckpointStorageAccess.java:64) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.runtime.state.filesystem.FsStateBackend.createCheckpointStorage(FsStateBackend.java:527) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.streaming.runtime.tasks.StreamTask. (StreamTask.java:337) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.streaming.runtime.tasks.StreamTask. (StreamTask.java:304) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.streaming.runtime.tasks.SourceStreamTask. (SourceStreamTask.java:76) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.streaming.runtime.tasks.SourceStreamTask. (SourceStreamTask.java:72) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_292] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_292] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_292] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_292] at org.apache.flink.runtime.taskmanager.Task.loadAndInstantiateInvokable(Task.java:1515) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:727) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) ~[flink-dist_2.12-1.13.0.jar:1.13.0] at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_292]
请求大佬看一下,主要是这两个集群,一个是k8s安装的,一个不是。
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。