fix(node): allow all diacritical marks in slugs #2389
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2d58651 limited the removal of nonspacing characters to combining diacritical marks targeting the Latin alphabet because removing marks for other alphabets results in undesirable changes to the semantics of the filename. Even for the Latin alphabet, though, removing these accents is not necessarily desirable and can change meaning. Filesystem support for Unicode is widespread. If users can type diacritical marks for their language into their node titles, they can likely type the same marks when managing files at a command line shell. Furthermore, multilingual users will still end up with non-ASCII-compatible filenames if their node titles happen to include letters from other languages. Therefore, be consistent across alphabets by default, and preserve all alphanumeric Unicode characters in filenames, regardless of whether those characters happen to be Latin-based. `string-glyph-compose` is retained, so NFC normalization still happens. While we’re at it, tidy the function as a whole. Ref: org-roam#1460
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
2d58651 limited the removal of nonspacing characters to combining diacritical marks targeting the Latin alphabet because removing marks for other alphabets results in undesirable changes to the semantics of the filename.
Even for the Latin alphabet, though, removing these accents is not necessarily desirable and can change meaning. Filesystem support for Unicode is widespread. If users can type diacritical marks for their language into their node titles, they can likely type the same marks when managing files at a command line shell. Furthermore, multilingual users will still end up with non-ASCII-compatible filenames if their node titles happen to include letters from other languages.
Therefore, be consistent across alphabets by default, and preserve all alphanumeric Unicode characters in filenames, regardless of whether those characters happen to be Latin-based.
string-glyph-composeis retained, so NFC normalization still happens. While we’re at it, tidy the function as a whole.Ref: #1460
Cc @khinsen (#230): Any thoughts on this? The above PR has already introduced NFC normalization and diacritics, but not for the Latin alphabet. I’m curious how your dual-boot setup looks where you’re running into normalization issues between macOS and Linux. Are you mounting a filesystem other than HFS or APFS in macOS that doesn’t have an
nfcmount option?