Can't correctly update name that has diacritics mark

[renintw]

Describe the bug

See https://wordpress.slack.com/files/U01ES2FFK42/F071EVB3JGZ/image.png

To reproduce

Steps to reproduce the behavior: Update user name in user profile.

Expected behavior

Should be correctly updated with the diacritics mark.

[renintw]

Findings:

On WordCamp sites or networks, user names containing diacritic marks cannot be saved correctly (for example, ś turns into ?). Furthermore, if the same user's name is updated on wp.org sites, it is saved and displayed correctly there. However, when returning to WordCamp sites, it again displays incorrectly (this time ś turns into garbled characters like Å›).

Also tested this through the local env and the sandbox, the local env has no such issue at all, and on the sandbox, I saw that the encoding format is explicitly specified in the wp-config:

// Force utf8mb4 since HyperDB won't admit it supports it.
define('DB_CHARSET', 'utf8mb4');
define('DB_COLLATE', 'utf8mb4_general_ci');

However, it still does not update correctly. So I proceeded to look at the DB, but I'm not sure if the WordCamp server is currently using the DB I see in phpMyAdmin, as the most recent WordCamp sites I can see date back to 2018, and I also didn't find any tables related to the central site or wc users.

any thoughts on the suggested next actions? @pkevan @dd32

I do see this script: https://wordcamp.svn.wordpress.org/bin/php/single-use/core/utf8mb4.php, should we try to run that?

[dd32]

There's a ticket already for this somewhere in the WordCamp backlog.

The bug isn't that you can't save properly, it's that you can edit the profile in the first place. No user details should be changed from WordCamp sites.

The profile should not be editable via WordCamp, BuddyPress, or bbPress. Or probably any wp-admin page on WordPress.org (Except for specially privileged accounts).

This is primarily (but not only) because editing users from those sites doesn't properly invalidate the memcache values, as they don't have access to it (although buddypress does now).

(I've got a diff sitting on my sandbox somewhere for this, I'll see if I can find it and commit it to the wporg-mu-plugins security file)

[dd32]

#590

[renintw]

Thanks, Dion, but it seems that even if we restrict users from updating their information outside of their profile, the character issue at WordCamp still needs to be resolved, right? I just tested updating a profile with diacritic marks, and the characters still display incorrectly on WordCamp.

I saw you mentioned needing to replace HyperDB with a WordPress.org variant. Is it tricky to do a complete test for it?

[dd32]

Not really.. the benefits don't outweigh the downsides.

The core issue is the database connection charset. WordCamp uses utf8mb4, WordPress.org uses utf8 in latin1. Reading generally works, but writing causes problems.

Changing the HyperDB version in use from the plugin version that's used on WordCamp, to the one used on WordPress.org, will fix it (I did some work on this for the wporg+BPorg use case, as BPorg also uses utf8mb4), allowing it to run the correct charsets in the right datasets, but it's not a straight copy-paste.

If you'd like to replace HyperDB with the wporg version and configuration (minus all the datasets that aren't needed) go for it, but any attempts to workaround it other ways is a waste of time IMHO..