Move symbolic font glyphs to private use area if they don't have unicode mappings (issue 2594, bug 789074, bug 865644) #7705

Snuffleupagus · 2016-10-09T11:58:28Z

This is a tentative patch, which builds upon PR #7482 (that commit is included here with the correct author), that attempts to address issue #2594, bug 789074, bug 865644.

According to #2594 (comment) and #2594 (comment) it seems like this kind of simple hack might actually work, and I'm opening the PR to be able to run tests on (especially) the Linux bot.

timvandermeij · 2016-10-13T19:50:49Z

I checked most of the test failures (not all of them), but those looked like improvements to me. It has been reported that this patch fixes the original issue.

/cc @yurydelendik or @brendandahl for review

brendandahl · 2016-10-17T18:18:51Z

It looks like we're moving a lot of glyphs to the private use area, even ones that probably shouldn't be.
Some random pdf's I looked at:
issue5501.pdf
before patch

fonts.js:920 originalCharCode=65 (A)
fonts.js:931 unicode=         65 (A)
fonts.js:920 originalCharCode=69 (E)
fonts.js:931 unicode=         69 (E)

after patch

fonts.js:920 originalCharCode=65 (A)
fonts.js:932 unicode=         65 (A)
fonts.js:958 remapping to=    57380 ()
fonts.js:920 originalCharCode=69 (E)
fonts.js:932 unicode=         69 (E)
fonts.js:958 remapping to=    57381 ()

artofwar.pdf

originalCharCode=65 (A)
fonts.js:933 unicode=         65 (A)
fonts.js:921 originalCharCode=66 (B)
fonts.js:933 unicode=         66 (B)

after

originalCharCode=65 (A)
fonts.js:933 unicode=         65 (A)
fonts.js:959 remapping to=    57359 ()
fonts.js:921 originalCharCode=66 (B)
fonts.js:933 unicode=         66 (B)
fonts.js:959 remapping to=    57360 ()

Also, see my comment in #2594 (comment)

Snuffleupagus · 2016-10-17T19:30:13Z

/botio test

Snuffleupagus · 2016-10-18T08:28:38Z

/botio test

Snuffleupagus · 2016-10-18T09:32:46Z

/botio test

yurydelendik · 2016-10-18T16:47:00Z

my 2¢: we don't have complete charCode->glyphNames mapping (based on encoding and differences). I'm thinking we shall address this first and after that adjustMapping will use glyphNames/unicode to determine if char code is at right location and will build FreeType2 autofit-friendly toFontChar?

Snuffleupagus · 2016-10-19T13:37:09Z

/botio test

…ode mappings.

…mapping and text selection for mathematic fonts (issue 2594)

Snuffleupagus · 2016-10-27T10:43:42Z

The latest version of this PR is in all likelihood my last attempt at fixing #2594, since it's an issue that I'm personally not affected by.

Here, besides the original patch by @brendandahl which already fixes the majority of the problem, I'm also amending src/core/glyphlist.js with the (common) names used in AMSmath Type1 (La)TeX fonts.
This is based on lists of TeX glyph names, and reasonable Unicode values, that I found while searching the web. Hopefully this should be enough to avoid mapping these math symbols to regular characters, thus preventing FreeType from using the wrong glyphs when computing hint values, and should also improve text-selection at the same time.

If this is a reasonable approach, we probably need a couple of Linux users (currently affected by the issue) to test this to ensure that it's in fact a sufficient solution.

pdfjsbot · 2016-11-10T11:40:29Z

From: Bot.io (Linux)

Received

Command cmd_test from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.21.233.14:8877/c75a6e95f23893f/output.txt

pdfjsbot · 2016-11-10T11:40:29Z

From: Bot.io (Windows)

Received

Command cmd_test from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.22.172.223:8877/b5bba0611348d95/output.txt

pdfjsbot · 2016-11-10T12:06:08Z

From: Bot.io (Windows)

Failed

Full output at http://107.22.172.223:8877/b5bba0611348d95/output.txt

Total script time: 25.65 mins

Font tests: Passed
Unit tests: Passed
Regression tests: FAILED

Image differences available at: http://107.22.172.223:8877/b5bba0611348d95/reftest-analyzer.html#web=eq.log

pdfjsbot · 2016-11-10T12:20:03Z

From: Bot.io (Linux)

Failed

Full output at http://107.21.233.14:8877/c75a6e95f23893f/output.txt

Total script time: 39.57 mins

Font tests: Passed
Unit tests: Passed
Regression tests: FAILED

Image differences available at: http://107.21.233.14:8877/c75a6e95f23893f/reftest-analyzer.html#web=eq.log

pdfjsbot · 2016-11-10T12:21:54Z

From: Bot.io (Linux)

Received

Command cmd_preview from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.21.233.14:8877/768d76e3834ac61/output.txt

pdfjsbot · 2016-11-10T12:24:53Z

From: Bot.io (Linux)

Success

Full output at http://107.21.233.14:8877/768d76e3834ac61/output.txt

Total script time: 2.99 mins

Published

mozilla/pdf.js#7705

brendandahl · 2016-11-29T00:49:23Z

r+ pending confirmation of fix

timvandermeij · 2016-12-10T19:19:47Z

The fix has been confirmed, so I'll take care of landing this.

timvandermeij · 2016-12-10T19:19:53Z

/botio-linux preview

pdfjsbot · 2016-12-10T19:19:53Z

From: Bot.io (Linux)

Received

Command cmd_preview from @timvandermeij received. Current queue size: 0

Live output at: http://107.21.233.14:8877/1ccea58dc194a13/output.txt

pdfjsbot · 2016-12-10T19:22:02Z

From: Bot.io (Linux)

Success

Full output at http://107.21.233.14:8877/1ccea58dc194a13/output.txt

Total script time: 2.14 mins

Published

timvandermeij · 2016-12-10T19:22:59Z

/botio test

pdfjsbot · 2016-12-10T19:22:59Z

From: Bot.io (Linux)

Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://107.21.233.14:8877/cf88fd2fe81c120/output.txt

pdfjsbot · 2016-12-10T19:22:59Z

From: Bot.io (Windows)

Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://107.22.172.223:8877/c9019a03bb8c0da/output.txt

pdfjsbot · 2016-12-10T19:49:07Z

From: Bot.io (Windows)

Failed

Full output at http://107.22.172.223:8877/c9019a03bb8c0da/output.txt

Total script time: 26.13 mins

Font tests: Passed
Unit tests: Passed
Regression tests: FAILED

Image differences available at: http://107.22.172.223:8877/c9019a03bb8c0da/reftest-analyzer.html#web=eq.log

pdfjsbot · 2016-12-10T19:50:27Z

From: Bot.io (Linux)

Failed

Full output at http://107.21.233.14:8877/cf88fd2fe81c120/output.txt

Total script time: 27.46 mins

Font tests: Passed
Unit tests: Passed
Regression tests: FAILED

Image differences available at: http://107.21.233.14:8877/cf88fd2fe81c120/reftest-analyzer.html#web=eq.log

timvandermeij · 2016-12-10T20:03:41Z

/botio makeref

pdfjsbot · 2016-12-10T20:03:41Z

From: Bot.io (Linux)

Received

Command cmd_makeref from @timvandermeij received. Current queue size: 0

Live output at: http://107.21.233.14:8877/d57a8875608d51b/output.txt

pdfjsbot · 2016-12-10T20:03:41Z

From: Bot.io (Windows)

Received

Command cmd_makeref from @timvandermeij received. Current queue size: 0

Live output at: http://107.22.172.223:8877/efe11a00014626c/output.txt

pdfjsbot · 2016-12-10T20:29:33Z

From: Bot.io (Windows)

Success

Full output at http://107.22.172.223:8877/efe11a00014626c/output.txt

Total script time: 25.85 mins

Lint: Passed
Make references: Passed
Check references: Passed

pdfjsbot · 2016-12-10T20:30:16Z

From: Bot.io (Linux)

Success

Full output at http://107.21.233.14:8877/d57a8875608d51b/output.txt

Total script time: 26.58 mins

Lint: Passed
Make references: Passed
Check references: Passed

timvandermeij · 2016-12-10T20:30:40Z

Thank you for the patch!

Move symbolic font glyphs to private use area if they don't have unicode mappings (issue 2594, bug 789074, bug 865644)

Snuffleupagus added the font-conversion label Oct 9, 2016

timvandermeij mentioned this pull request Oct 13, 2016

Move symbolic font glyphs to private use area if they don't have unic… #7482

Closed

timvandermeij assigned brendandahl and yurydelendik Oct 13, 2016

brendandahl mentioned this pull request Oct 17, 2016

Formulas are painted too big. #2594

Closed

brendandahl and others added 2 commits October 26, 2016 16:39

Move symbolic font glyphs to private use area if they don't have unic…

8d036fa

…ode mappings.

Add TeX-specific glyph names to glyphlist.js to improve both glyph …

81b9d55

…mapping and text selection for mathematic fonts (issue 2594)

briangough pushed a commit to overleaf/web that referenced this pull request Nov 16, 2016

try out new pdfjs font fix

c0bdb1d

mozilla/pdf.js#7705

Snuffleupagus changed the title ~~Move symbolic font glyphs to private use area if they don't have unicode mappings (issue 2594)~~ Move symbolic font glyphs to private use area if they don't have unicode mappings (issue 2594, bug 865644) Nov 22, 2016

Snuffleupagus changed the title ~~Move symbolic font glyphs to private use area if they don't have unicode mappings (issue 2594, bug 865644)~~ Move symbolic font glyphs to private use area if they don't have unicode mappings (issue 2594, bug 789074, bug 865644) Nov 22, 2016

timvandermeij merged commit 00a006e into mozilla:master Dec 10, 2016

Snuffleupagus deleted the issue-2594 branch December 10, 2016 21:28

Snuffleupagus mentioned this pull request Apr 1, 2017

Math rendering problems #8224

Closed

movsb pushed a commit to movsb/pdf.js that referenced this pull request Jul 14, 2018

Merge pull request mozilla#7705 from Snuffleupagus/issue-2594

45a6b04

Move symbolic font glyphs to private use area if they don't have unicode mappings (issue 2594, bug 789074, bug 865644)

Snuffleupagus mentioned this pull request Feb 7, 2019

Unicode detection error #10247

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move symbolic font glyphs to private use area if they don't have unicode mappings (issue 2594, bug 789074, bug 865644) #7705

Move symbolic font glyphs to private use area if they don't have unicode mappings (issue 2594, bug 789074, bug 865644) #7705

Snuffleupagus commented Oct 9, 2016 •

edited

timvandermeij commented Oct 13, 2016

brendandahl commented Oct 17, 2016

Snuffleupagus commented Oct 17, 2016

Snuffleupagus commented Oct 18, 2016

Snuffleupagus commented Oct 18, 2016

yurydelendik commented Oct 18, 2016

Snuffleupagus commented Oct 19, 2016

Snuffleupagus commented Oct 27, 2016 •

edited

pdfjsbot commented Nov 10, 2016

pdfjsbot commented Nov 10, 2016

pdfjsbot commented Nov 10, 2016

pdfjsbot commented Nov 10, 2016

pdfjsbot commented Nov 10, 2016

pdfjsbot commented Nov 10, 2016

brendandahl commented Nov 29, 2016

timvandermeij commented Dec 10, 2016

timvandermeij commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

timvandermeij commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

timvandermeij commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

timvandermeij commented Dec 10, 2016

Move symbolic font glyphs to private use area if they don't have unicode mappings (issue 2594, bug 789074, bug 865644) #7705

Move symbolic font glyphs to private use area if they don't have unicode mappings (issue 2594, bug 789074, bug 865644) #7705

Conversation

Snuffleupagus commented Oct 9, 2016 • edited

timvandermeij commented Oct 13, 2016

brendandahl commented Oct 17, 2016

Snuffleupagus commented Oct 17, 2016

Snuffleupagus commented Oct 18, 2016

Snuffleupagus commented Oct 18, 2016

yurydelendik commented Oct 18, 2016

Snuffleupagus commented Oct 19, 2016

Snuffleupagus commented Oct 27, 2016 • edited

pdfjsbot commented Nov 10, 2016

From: Bot.io (Linux)

Received

pdfjsbot commented Nov 10, 2016

From: Bot.io (Windows)

Received

pdfjsbot commented Nov 10, 2016

From: Bot.io (Windows)

Failed

pdfjsbot commented Nov 10, 2016

From: Bot.io (Linux)

Failed

pdfjsbot commented Nov 10, 2016

From: Bot.io (Linux)

Received

pdfjsbot commented Nov 10, 2016

From: Bot.io (Linux)

Success

Published

brendandahl commented Nov 29, 2016

timvandermeij commented Dec 10, 2016

timvandermeij commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

From: Bot.io (Linux)

Received

pdfjsbot commented Dec 10, 2016

From: Bot.io (Linux)

Success

Published

timvandermeij commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

From: Bot.io (Linux)

Received

pdfjsbot commented Dec 10, 2016

From: Bot.io (Windows)

Received

pdfjsbot commented Dec 10, 2016

From: Bot.io (Windows)

Failed

pdfjsbot commented Dec 10, 2016

From: Bot.io (Linux)

Failed

timvandermeij commented Dec 10, 2016

pdfjsbot commented Dec 10, 2016

From: Bot.io (Linux)

Received

pdfjsbot commented Dec 10, 2016

From: Bot.io (Windows)

Received

pdfjsbot commented Dec 10, 2016

From: Bot.io (Windows)

Success

pdfjsbot commented Dec 10, 2016

From: Bot.io (Linux)

Success

timvandermeij commented Dec 10, 2016

Snuffleupagus commented Oct 9, 2016 •

edited

Snuffleupagus commented Oct 27, 2016 •

edited