-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
http logger should respect the encoding of its log outputs #12179
Comments
I forked the codebase and put in the small change and everything seems OK for my use case though I'm too concerned about unforeseen side effects and unclear on how to test etc. to open a PR. |
Hi, Thanks for the report. I would say that the call should be just And just open the log in text mode here twisted/src/twisted/web/http.py Lines 3304 to 3309 in c465c46
Don't worry to much about breaking things. If you send a PR with good test coverate and for changes which pass on the existing tests, the PR should be good to merge. As a side note, I don't know why the Twisted HTTP is not using the standard logging infrastructure, and instead uses a custom file. I am using the server in production, but I have just overwritten the factory not to handle the logs in a separate file. Thanks again! |
Whew. OK, there are a couple of mixed-up issues here.
|
|
thanks - i tested out the changes from with #12198 and my problems are solved |
Describe the incorrect behavior you saw
At this line Twisted is assuming that
logFile
has an encoding of"utf8"
but that isn't always the case... I'm currently on macOS redirecting my logs tostdout
through the system logs which use"utf-16-le"
(viastd-nslog
) , and so when Twisted goes to log a basic http request it crashes on some log lines withbuiltins.UnicodeDecodeError: 'utf-16-le' codec can't decode byte 0x0a in position 160: truncated data
.If I change that line to
everything works fine, but I shouldn't have to do that given that the encoding is stored in the
logFile
variable at the time the call is made. Here's the output ofvars(logFile)
:So
twisted.python.log
already knows that thelogFile
is in a different encoding but it goes ahead and forces the wrong encoding anyways. It seems to me - correct me if I'm wrong - that changing the line tomight fix this fairly trivially?
It's also possible there's some order of log instantiation that would fix this but I have tried many things as detailed here and I always get the same result, even when i replace the twisted logger with a logger that does nothing.
Describe how to cause this behavior
std-nslog
to reroutesys.stdout
andsys.stderr
through macOS system logsStack Trace
Describe the correct behavior you'd like to see
Twisted's log mechanism should respect the encoding of the writeable object its trying to write to.
Testing environment
The text was updated successfully, but these errors were encountered: