Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix UnicodeDecodeError #378

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ehrenfeu
Copy link

Log files like e.g. the Blitz-0.log are containing references to files being uploaded by users, i.e. there is no reasonable assumption that can be made over what people will put into their file names.

Parsing the logs e.g. via "omero admin diagnostics" will fail with a UnicodeDecodeError when hitting lines referring to such file names.

Force-setting the encoding to "utf-8" here fixes this problem for us.

Log files like e.g. the `Blitz-0.log` are containing references to
files being uploaded by users, i.e. there is no reasonable assumption
that can be made over what people will put into their file names.

Parsing the logs e.g. via "omero admin diagnostics" will fail with a
UnicodeDecodeError when hitting lines referring to such file names.

Force-setting the encoding to "utf-8" here fixes this problem for us.
@ehrenfeu
Copy link
Author

For the record, the unicode sequence that was contained in the filename that triggered this issue now was \xe2\x80\x8e, but we have seen this repeatedly before.

@sbesson sbesson self-requested a review July 27, 2023 20:28
@sbesson
Copy link
Member

sbesson commented Jul 28, 2023

Thanks @ehrenfeu, do you have by any chance the full stack trace of the original exception?
For reference, #236 (released in OMERO.py 5.8.0) aimed to deal at a very similar issue. The original fix also forcing the encoding to utf-8 but as we tried to cover additional scenarios, the code eventually moved to using the surrogateescapehandler error handler to decode unsupported bytes.

@ehrenfeu
Copy link
Author

Thanks @sbesson - unfortunately I am already off now, but I might be able to produce a stack trace after returning from vacation. Feel free to remind me if you don't get one by the end of August.

@sbesson
Copy link
Member

sbesson commented Sep 4, 2023

@ehrenfeu ping

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants