Hyphens created with incorrect unicode character, causing compatibility issues

[cyberbrix]

Is there an existing issue for this?

• I have searched the existing open and closed issues

Current Behavior

When creating hyphens, the wrong encoding is used. (Should be Hyphen-Minus.) This creates compatibility issues with items which have a hyphen in the name. I believe this is related to #3371.

Subsequently, when a "-" from other systems is used, Lidarr doesn't recognize them.

Expected Behavior

hyphen-minus should be used in place of hyphen

Steps To Reproduce

import an artist with a hyphen in the name. (blink182, lin-manuel miranda, B-52s, etc.) Use fake tracks if needed for proper effect.

Attempt to view them in an ubuntu/bash shell

ls -d blink*
It will show 'blink□182' with a square box. (if you copy/paste into any other OS, it displays it.)

If you edit the artist name and use the proper hyphen, Lidarr no longer recognizes the content.

Environment

- OS: Ubuntu 20.04.6 LTS
- Lidarr: 2.2.5.4141
- Docker Install: No
- Using Reverse Proxy: No
- Browser: Any
- Database: Sqlite 3.31.1

What branch are you running?

Master

Trace Logs?

Trace Logs have been provided as applicable. Reports may be closed if the required logs are not provided.

• I have read and followed the steps in the wiki link above and provided the required trace logs - the logs contain trace - that are relevant and show this issue.

[mynameisbogdan]

You need to provide trace logs in text format.

Also share the output of locale.

[cyberbrix]

LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

lidarr.trace.txt

I took the tracks, put them in a folder for manual import, and then imported them. After found, I went to the folder, and literally replaced the hyphen with a hyphen (in Windows). Went back to the path, did a refresh and it didn't see the files anymore.

[mynameisbogdan]

You're editing filenames outside of Lidarr?

Why do you think this is an issue caused by Lidarr?

[cyberbrix]

No, the folder/file names are created by Lidarr using the incorrect encoding. That creates certain characters, such as "Hyphen", rather than "Hyphen-Minus". If you read up on https://en.wikipedia.org/wiki/ISO/IEC_8859-1 and subsequently,
https://en.wikipedia.org/wiki/Hyphen-minus. I think it would be solved by the aforementioned #3371

Lidarr doesn't use U+002D for hyphen, it uses U+2010. All other systems generally use U+002D, and are compatible

[mynameisbogdan]

I took the tracks, put them in a folder for manual import, and then imported them. After found, I went to the folder, and literally replaced the hyphen with a hyphen (in Windows). Went back to the path, did a refresh and it didn't see the files anymore.

I have a hard time understanding what you're doing here when you said you're editing file paths and now you saying you don't.

[cyberbrix]

The files and paths were created by Lidarr. I replaced a hyphen with a hyphen.  This because Lidarr is looking for the wrong hyphen, it thinks the files are gone.  

[mynameisbogdan]

In Windows as in using a NFS mount to view and edit paths, correct? Maybe it's a samba character encoding issue in your case?

I hope you understand that your steps to reproduce aren't quite as described.

[cyberbrix]

The screenshot I shared was from Linux. 

[cyberbrix]

Just re-imported the files. They were previously in a folder named blink-182,

Now: