Link preview fails to decode UTF-8.

[dhoepfl]

Steps to reproduce the problem

See this post.

The link preview says “Lüneburg Aktuell · Apr 8” instead of “Lüneburg Aktuell · Apr 8”.

Expected behaviour

Incorrect encoding in one tag should not break others.

Actual behaviour

See above: “Lüneburg” instead of “Lüneburg”.

Detailed description

I suspect the incorrectly truncated UTF-8 sequence at the end of the og:description tag triggers this error: <meta property="og:description" content="Aufruf gegen den Landesparteitag der AfD Niedersachsen am 20. April in Unterlüß Gemeinsam gegen rechts – für eine vielfältige Gesellschaft! Die niedersächsische AfD h�...">

Mastodon instance

layer8.space, and on original page on norden.social

Mastodon version

v4.2.8

Browser name and version

Firefox 125.0.1

Operating system

Windows 10

Technical details

No response

[PuercoPop]

Reading the code, each tag is processed independently

mastodon/app/lib/link_details_extractor.rb

Lines 182 to 184 in 35b517c

	def provider_name
	html_entities.decode(structured_data&.publisher_name || opengraph_tag('og:site_name'))
	end

I checked the spec file and there is no data using utf8. Trying to reproduce locally would involve updating the spec

mastodon/spec/lib/link_details_extractor_spec.rb

Line 210 in 35b517c

<meta property="og:site_name" content="Pet News">

---

The linked toot's site has the following OpenGran site_name tag:
<meta property="og:site_name" content="Lüneburg Aktuell">