Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI mistune on Windows defaults to cp1251 encoding on Russian locale, produces error #379

Open
JinEnMok opened this issue Feb 25, 2024 · 1 comment

Comments

@JinEnMok
Copy link

JinEnMok commented Feb 25, 2024

Apparently, there's a bug in the Windows + Python combo on certain locales which results in a wonky encoding choices by the interpreter.

In my case, when I ran python -m mistune ... on a file that contained the character ★ (U+2605 "Black Star"), it produced an error along the lines of:

[lines omitted for clarity]
   File "encodings\cp1251.py", line [xxx], in encode
UnicodeEncodeError: 'charmap' codec can't encode characters in position [yyy-zzz]: character maps to <undefined>

My system language is Russian, so its choice of encoding is understandable to a degree. This appears to be a similar problem, and applying that along with this advice fixed it for me.

Maybe there's something to be adjusted in mistune to force it (?) to use UTF-8?

  • Windows 10
  • Python 3.12.2
  • mistune 3.0.2
@lepture
Copy link
Owner

lepture commented Mar 30, 2024

@JinEnMok A pull request is welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants