[web] Make search insensitive to differences in punctuation

[hacketiwack]

Current situation
When searching for a song containing punctuation characters, the search string must match exactly the punctuation characters.

Example
Let's admit that you have a song named L’Hymne à l’amour in your database and you want to search for it.
By entering the search term l'hymne you won't get any result as the punctuation characters in the database are ’ (U+2019) instead of ' (U+0027).
It would be awesome to be able to search with whatever punctuation characters.

For example, the search in Firefox takes that into account. Therefore, when searching for ’ or ', the results are the same.
Added to that, there are other punctuation characters that affect the search: e.g., «, », ”, “, ”, ꞉, ：

[ejurgensen]

Yes, I guess this has relation to #1390, and I imagine the solution here is also some modification of the db collation. However, not sure how to actually do that, much of what is in done in that part of sqlext.c is black magic to me.

[hacketiwack]

@ejurgensen indeed, it is certainly related. I will try to find some more information on how to do that.
So far, my searches have given me no valuable results.
I'll keep you posted.

[hacketiwack]

I thought about this again, and I was wondering if using uc_is_punct could be used to ignore punctuation characters.

Maybe we could add a test on this line to ignore punctuation characters like below.

if ( !uc_is_punct(uString) && sqlite3Fts5UnicodeFold(uString, 1) != sqlite3Fts5UnicodeFold(uPattern, 1) )

However, I'm not sure what negative effects this would have.