Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Line length in Fido texts is based on bytes, not characters #136

Open
punktniklas opened this issue May 9, 2021 · 0 comments
Open

Line length in Fido texts is based on bytes, not characters #136

punktniklas opened this issue May 9, 2021 · 0 comments
Labels
Milestone

Comments

@punktniklas
Copy link
Owner

When Fido texts are read from disk they are broken up into lines (since the Fido text format is based on paragraphs, not lines). The line length used is 79 when reading texts and 75 when quoting. This is counted in bytes, which was the same as the number of characters when everything was converted to ISO Latin 1. Now however it's possible that the text is in a multi byte format such as UTF-8. This means that if all characters in a line use two bytes the effective line length will be less than 40. And for characters that use even more bytes (such as many asian languages) it will be even shorter.

The reading logic should be updated to count characters (Unicode code points), not bytes.

@punktniklas punktniklas added the bug label May 9, 2021
@punktniklas punktniklas added this to the Soonish milestone May 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant