Error if code includes unicode character

[rebeccamccabe]

The comments of my code use the unicode character Σ.
When I go to run > run line profiler, I get the following error:

Traceback (most recent call last):
  File "C:\Users\rgm222\Anaconda3\envs\decider-10-7-23\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\rgm222\Anaconda3\envs\decider-10-7-23\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\rgm222\Anaconda3\envs\decider-10-7-23\lib\site-packages\kernprof.py", line 409, in <module>
    main(sys.argv[1:])
  File "C:\Users\rgm222\Anaconda3\envs\decider-10-7-23\lib\site-packages\kernprof.py", line 395, in main
    prof.print_stats(output_unit=options.unit,
  File "C:\Users\rgm222\Anaconda3\envs\decider-10-7-23\lib\site-packages\line_profiler\line_profiler.py", line 169, in print_stats
    show_text(lstats.timings, lstats.unit, output_unit=output_unit,
  File "C:\Users\rgm222\Anaconda3\envs\decider-10-7-23\lib\site-packages\line_profiler\line_profiler.py", line 460, in show_text
    show_func(fn, lineno, name, stats[fn, lineno, name], unit,
  File "C:\Users\rgm222\Anaconda3\envs\decider-10-7-23\lib\site-packages\line_profiler\line_profiler.py", line 433, in show_func
    stream.write(txt)
  File "C:\Users\rgm222\Anaconda3\envs\decider-10-7-23\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u03a3' in position 109: character maps to <undefined>

Removing the unicode character solves the issue.

Versions:
spyder 5.5.0
spyder-line-profiler 0.3.2

[jitseniesen]

Hi @rebeccamccabe, sorry for not saying something when you filed your report and thanks for taking the time to write it up.

I tried to reproduce the error but everything works fine on my computer. My guess is that it depends on the operating system, your computer is Windows and mine is Linux, so I will have a go at it on Windows later.

Some initial thoughts, mainly written down so that I remember later: the error occurs in line_profiler and there is nothing about Spyder in the error message. When Spyder runs the line profiler, it does so in a separate Python process. Perhaps that process outputs to a pipe or something like that using the cp1252 encoding and the error occurs when it encounters the Σ character. There is a similar traceback at https://stackoverflow.com/questions/56936927/unicodeencodeerror-when-using-line-profiler so it's possible the real error is in the line profiler. Nevertheless, we should try to work around it. Maybe turn on UTF8 mode in Python?

[jitseniesen]

I managed to reproduce the error in our test environment on Windows.