Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OzoneFileSystem #676

Open
tothandor opened this issue Nov 29, 2023 · 0 comments
Open

OzoneFileSystem #676

tothandor opened this issue Nov 29, 2023 · 0 comments

Comments

@tothandor
Copy link

I try to use an Ozone filesystem with this HDFS Sink Connector, but I get the following error, which is weird since OzoneFileSystem is a subclass of Hadoop FileSystem.

I could use this very filesystem with CLI commands like hdfs dfs -ls ofs://... or through libhdfs from anywhere else (eg. pyarrow.fs.HadoopFilesystem.from_uri("ofs://..."))), so I would suppose that it should be usable natively in Java.

I am not a Java developer, but I tried my best to interpret the sources of the involved libraries to find out what is the problem, but I can't figure it out. Maybe this is not the problem of this connector, but it seems to me, that the other HdfsStorage constructor should be invoked, to make it work.

Caused by: java.lang.ClassCastException: org.apache.hadoop.fs.ozone.OzoneFileSystem cannot be cast to org.apache.hadoop.fs.FileSystem
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3260)
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:121)
        at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3310)
        at org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:3284)
        at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:528)
        at io.confluent.connect.hdfs.storage.HdfsStorage.<init>(HdfsStorage.java:60)

Please help me to find this out!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant