hue xxx.gz file Open in Brower Failed to decompress file.

[liwei2916]

Is there an existing issue for this?

• I have searched the existing issues

Description

I want to view the content of the gz file in HDFS through hue, but unfortunately, hue reported an error. The error message is Failed to decompress file

Steps To Reproduce

python 3.8

Logs

[11/Apr/2024 22:10:25 -0700] views ERROR Could not decompress file at "/datawarehouse/alpha/temp/base_obj_fulltext/part-00002.gz": Compressed file ended before the end-of-stream marker was reached
Traceback (most recent call last):
File "/usr/share/hue/apps/filebrowser/src/filebrowser/views.py", line 1007, in _read_gzip
contents = decompress_gzip(fhandle.read())
File "/usr/lib/python3.8/gzip.py", line 548, in decompress
return f.read()
File "/usr/lib/python3.8/gzip.py", line 292, in read
return self._buffer.read(size)
File "/usr/lib/python3.8/gzip.py", line 498, in read
raise EOFError("Compressed file ended before the "
EOFError: Compressed file ended before the end-of-stream marker was reached
[11/Apr/2024 22:10:25 -0700] exceptions_renderable ERROR Potential trace: [<FrameSummary file /usr/share/hue/apps/filebrowser/src/filebrowser/views.py, line 1007 in _read_gzip>, <FrameSummary file /usr/lib/python3.8/gzip.py, line 548 in decompress>, <FrameSummary file /usr/lib/python3.8/gzip.py, line 292 in read>, <FrameSummary file /usr/lib/python3.8/gzip.py, line 498 in read>]
[11/Apr/2024 22:10:25 -0700] middleware INFO Processing exception: Failed to decompress file.: Traceback (most recent call last):
File "/usr/share/hue/apps/filebrowser/src/filebrowser/views.py", line 1007, in _read_gzip
contents = decompress_gzip(fhandle.read())
File "/usr/lib/python3.8/gzip.py", line 548, in decompress
return f.read()
File "/usr/lib/python3.8/gzip.py", line 292, in read
return self._buffer.read(size)
File "/usr/lib/python3.8/gzip.py", line 498, in read
raise EOFError("Compressed file ended before the "
EOFError: Compressed file ended before the end-of-stream marker was reached

During handling of the above exception, another exception occurred:

Hue version

hue 4.11

[liwei2916]

After verification, gz files larger than 1MB are not allowed, while files smaller than 1MB can be browsed for their content

What is the reason for this? Can someone help answer it

Is there any configuration that can solve it in hue.ini

[bjornalm]

Hi @liwei2916 and thanks for reaching out. @athithyaaselvam do you know why we have this limit on compressed files?

[liwei2916]

@bjornalm If compiling Hue with Python 3, opening a compressed file larger than 1MB triggers an error. I'll temporarily revert to version 4.8.0 and use Python 2.7

anyone has encountered the same problem？

[athithyaaselvam]

Hi @liwei2916, I tried to reproduce this but seems like its working for me (built on the master branch). I opened a 17mb .tgz file. Although there is a config in filebrowser, which might prevent it

[filebrowser]
  # A limit on a file size (bytes) that can be uploaded to a filesystem.
  # A value of -1 means there will be no limit.
  max_file_size_upload_limit=-1 

[liwei2916]

Hi @athithyaaselvam May I ask if you are using Python 3 or Python 2 for compilation? My environment is Releases 4.11.0 and Python 3.8. I have added the max_file_size_upload_limit=-1 configuration, but it still doesn't work

I still have the same error when using the gethue/hue: 4.11.0 image

I am currently using version 4.8.0, compiled through Python 2.7, and there are no restrictions on opening compressed files

[athithyaaselvam]

Hi @liwei2916, I'm using Python3.8 for compilation. Opening the file fails only for ".gz" format, as a quick workaround, try to change the compression format. I tried with ".tgz, .zip" files (It works). There is a restriction on the decompression for ".gz" files in Py3 Hue, Looking into it.

[liwei2916]

Okay, thank you @athithyaaselvam ， Will future versions be supported?

[github-actions]

This issue is stale because it has been open 30 days with no activity and is not labeled "Prevent stale". Remove "stale" label or comment or this will be closed in 10 days.