Writing UTF-8 ics files on Windows

[mo-gm]

my.zip

Hey there,
I am fairly new to github and python, so please be gentle with me :)
I am having trouble with creating an .ics-file with your ics.py-library. As i am german, there are many letters like ü, ä, ö and ß. I am trying to create .ics-files from emails wich are partly in german.
I managed to create the file and fill it with a description, title etc. unfortunately the special characters i mentioned above are simply missing in the file when created. Do you know what i am doing wrong? i basically followed you example here:
https://icspy.readthedocs.io/en/stable/index.html (starting from "Create a new calendar and add events")

I would be glad if you could help me. If this request is in the wrong place here, please let me know.

from ics import Calendar, Event
c = Calendar()
e = Event()
e.name = "My cool event"
e.begin = '2014-01-01 00:00:00'
e.description = 'Ich begrüße euch zu Beispiel, ähnlich dem, welches ich selbst verwende. Hier noch ein Wort mit ö: Döner.'
c.events.add(e)
c.events
with open('my.ics', 'w') as my_file:
    my_file.writelines(c)

I did not use my original file, but reduced it to the core. Now it is basically your example with a german description with as much special characters as i could fit in these to sentences.

My Python Version is 3.8.2
My Windows Version is Windows 10 Home
Ics.py Version is 0.7
(I think everything is up to date)

//edit: Code Example for future reader of this thread
//edit: I added a zip file containing the .ics file i produced because i could not add it without zipping it

[N-Coder]

Hey!
Yep, seems to be the right place - although I haven't used the new discussions feature before this seems to be suited far better than opening an issue.

Could you provide a minimal working example for what you are trying to do, which also reproduces the error?
Also some info about your environment (python version, operation system, version of ics.py) would be helpful.
In general, including these two points when discussing software online speeds things up a lot, as it allows others to faster reproduce your issue. ;)
If you have a relevant .ics file at hand, you can edit it with a text editor and redact lines containing private info by simply stripping them out or replacing their value with gibberish, so that you can also include it here.

Viele Grüße

[mo-gm]

Hey N-Coder,
thank you for your kind reply. I updated my original post for future readers with the same problem. I also added a zip file with the produces ics-file. it is zipped because github would not let me upload it as *.ics

This is my code:

from ics import Calendar, Event
c = Calendar()
e = Event()
e.name = "My cool event"
e.begin = '2014-01-01 00:00:00'
e.description = 'Ich begrüße euch zu Beispiel, ähnlich dem, welches ich selbst verwende. Hier noch ein Wort mit ö: Döner.'
c.events.add(e)
c.events
with open('my.ics', 'w') as my_file:
    my_file.writelines(c)

I did not use my original file, but reduced it to the core. Now it is basically your example with a german description with as much special characters as i could fit in these to sentences.

My Python Version is 3.8.2
My Windows Version is Windows 10 Home
Ics.py Version is 0.7

[C4ptainCrunch]

It looks like the problem might only appear on windows, i just tried your sample on Linux and it looks like it works as expected.

~> ipython
Python 3.9.0 

In [1]: from ics import Calendar, Event
   ...: c = Calendar()
   ...: e = Event()
   ...: e.name = "My cool event"
   ...: e.begin = '2014-01-01 00:00:00'
   ...: e.description = 'Ich begrüße euch zu Beispiel, ähnlich dem, welches ich selbst verwende. Hier noch ein Wort mit ö: Dön
   ...: er.'
   ...: c.events.add(e)
   ...: c.events
   ...: with open('my.ics', 'w') as my_file:
   ...:     my_file.writelines(c)
   ...: 

In [2]: c
Out[2]: <Calendar with 1 event and 0 todo>

In [3]: str(c)
Out[3]: 'BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:ics.py - http://git.io/lLljaA\r\nBEGIN:VEVENT\r\nDESCRIPTION:Ich begrüße euch zu Beispiel\\, ähnlich dem\\, welches ich selbst verwende. Hier noch ein Wort mit ö: Döner.\r\nDTSTART:20140101T000000Z\r\nSUMMARY:My cool event\r\nUID:f5732d20-e949-482f-9d06-248198a055aa@f573.org\r\nEND:VEVENT\r\nEND:VCALENDAR'

~> cat my.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:ics.py - http://git.io/lLljaA
BEGIN:VEVENT
DESCRIPTION:Ich begrüße euch zu Beispiel\, ähnlich dem\, welches ich selbst verwende. Hier noch ein Wort mit ö: Döner.
DTSTART:20140101T000000Z
SUMMARY:My cool event
UID:f5732d20-e949-482f-9d06-248198a055aa@f573.org
END:VEVENT
END:VCALENDAR

Could you try changing the write part with these examples and tell us if any of them works ?

with open('my.ics', 'w', encoding="utf8") as my_file:
    my_file.writelines(c)

with open('my.ics', 'w') as my_file:
    my_file.write(str(c))

with open('my.ics', 'wb') as my_file:
    my_file.write(str(c).encode('utf-8'))

[N-Coder]

Hmm, that is weird. If I run your example, I get a correct .ics file with all the umlauts (yeah, that is really the english word for that ... germanisms are weird):

BEGIN:VCALENDAR
VERSION:2.0
PRODID:ics.py - http://git.io/lLljaA
BEGIN:VEVENT
DESCRIPTION:Ich begrüße euch zu Beispiel\, ähnlich dem\, welches ich selbst verwende. Hier noch ein Wort mit ö: Döner.
DTSTART:20140101T000000Z
SUMMARY:My cool event
UID:2605b094-5b57-4b5f-b7ec-a44bf0c84758@2605.org
END:VEVENT
END:VCALENDAR

If I open the .ics file from your .zip, I can also see the umlauts, but there is now one empty line after each actual line of the file:

my.zip/my.ics with doubled line breaks

BEGIN:VCALENDAR

VERSION:2.0

PRODID:ics.py - http://git.io/lLljaA

BEGIN:VEVENT

DESCRIPTION:Ich begrüße euch zu Beispiel\, ähnlich dem\, welches ich selbst verwende. Hier noch ein Wort mit ö: Döner.

DTSTART:20140101T000000Z

SUMMARY:My cool event

UID:cbf5ac87-d5fa-46b4-9631-5901400a6abb@cbf5.org

END:VEVENT

END:VCALENDAR

As I'm on linux and we can't control the line ending mode when you write the file via writelines that is no too big surprise and also shouldn't be any problem with other ics applications.

So the umlauts are actually there, no matter which OS generated the file.
Maybe it is the editor with which you open the file that can't handle them or guesses the encoding wrong (maybe try Notepad++, Sublime Text, VS Code or anything that isn't the dumb M$ Notepad).
Python 3 is all UTF-8, so there's not much that we could do wrong on that side.

Hint: You can enclose code blocks in triple backticks (```) to have it properly formatted as code - if you also specify the language, you'll also get nice syntax highlighting. I did that for you. ;)

[mo-gm]

with open('my.ics', 'w', encoding="utf8") as my_file:
    my_file.writelines(c)

with open('my.ics', 'wb') as my_file:
    my_file.write(str(c).encode('utf-8'))

These two did actually work! Big thank you to both of you!

@N-Coder : The umlaut-thing made me smile :) I tried the "insert code"-button in the github repy section, but it only made two ticks (``), but thank you for showing that to me. As a text-editor i am using sublime text 3. to open the .ics file i used ms outlook.

@C4ptainCrunch : That was a quick and easy solution, thank you!

@both: Thank you for solving my issue in that short time! I am now even more thrilled to get better at python and i will try to get used to github (i promise ;) )

[N-Coder]

Ahh C4ptainCrunch was right, open doesn't always use UTF-8, but tries to guess the user's preference from their locale.
I just checked, your file actually uses the encoding 'Western (Windows 1252)'.
I guess we should fix this example, as the RFC actually says that we MUST generate the files in UTF-8.

[mo-gm]

/edit: Found me old answer. but i can not mark @C4ptainCrunch s reply as answer. So here it is marked for future readers:

So it worked with these two code-snippets:

with open('my.ics', 'w', encoding="utf8") as my_file:
    my_file.writelines(c)

with open('my.ics', 'wb') as my_file:
    my_file.write(str(c).encode('utf-8'))