[troika-three-text] IOS Safari Loading Issues #88
Comments
atlmtw
changed the title
|
Thanks for reporting this! You're not the first to describe issues like this in iOS Safari, but I wasn't able to reproduce it previously. I'll try with your site and hopefully I'll be able to reproduce it. Are you using it via |
|
Hey @lojjic I'm using this package directly with three.js. And the reason I have three-utils is that I also use your createDerivedMaterial function in some cases. Thanks for checking on this! |
|
I was able to reproduce the issue(s) loading your website on an iPhone 8. I haven't yet had time to start debugging but I just wanted to acknowledge that the bug is reproducible. |
|
+1 |
|
I'm trying to dig into this issue, and I'm having trouble reproducing it now. Same iPhone 8 as before. I can't reproduce it on designsdalena.com anymore, but it's possible that site has changed to no longer use troika-three-text or worked around it in another way. So I'm trying to create a minimal testcase using multiple fonts, and can't get it to fail: Is anyone else able to reproduce this somewhere that I could use to test/debug? |
|
Hey @lojjic To confirm, I did switch to something else. Although I do enjoy the experience of using troika more. |
|
@atlmtw Thanks for the confirmation. I don't suppose you have an old version of your site in source control that you'd be able to send me? I'm struggling to reproduce this and yours seemed like a reliable repro. |
|
Hey @lojjic Btw, love the work 💯 |
|
@amitrajput1992 No I still haven't been able to construct a reproducible test case. Do you have one I could use? |
|
@lojjic This is what I see on BrowserStack. This is on Iphone 8 + Safari v13 This one on my iPhone 11 + Safari 14 Could it be something to do with the fact that font files are downloaded in the web-worker? We know safari can be a pain with web workers |
|
@amitrajput1992 Thanks, I can indeed replicate the issue on that page. I'll see if I can debug from there and/or replicate it in a minimized local testcase.
I've heard others say this too, and the usage of a worker is definitely a plausible culprit, but I haven't been able to find any details about known bugs with workers in Safari. Do you have specifics there you can share? |
|
@lojjic
I don't know about the exact issues here, partly because I can't find these anywhere (or people don't log these), but this is something that I've noticed while tinkering around. react360 formerly react-vr (abandoned now) used to runs the whole react code inside a web worker and the updates to the main thread were too damn slow. It would easily take a click action anywhere around 300ms to 500ms to propagate to my click listener in the worker thread and often would miss a few clicks too. I feel this could be something to do with the structured clone algorithm that is used when transmitting information from the worker to the main thread. Although I haven't been able to find a solid proof around this limitation on iOS |
|
I think I should first focus on these artifacts, which don't always appear but sometimes do:
In those cases the layout and SDF generation is obviously completing and returning to the main thread, but some of the SDFs appear to have inside/outside fills reversed in places. I can see how that could happen if the path data for those characters is incomplete, like only having half its path segments. If something is happening during font load where the font's arraybuffer is truncated or corrupted, I think that could possibly result in this observed symptom. Likewise it could also result in totally empty results if truncated at certain places. I'll investigate that as a possibility (maybe XHR giving a partial arraybuffer response?) but first step is getting this into a reproducible testcase that I can modify and run locally. |
|
That makes sense. arraybuffer corruption could be a culprit here. Also, If it helps, I'm using troika using drei with the below versions: |
|
Running on the main thread is possible, but would result in a pretty crappy experience blocking the main thread for potentially several seconds. That should be a last resort only. Has your test page changed? It seems like it's only using a single font for all the different text objects now, at least on iOS...? I still see garbled SDFs with that single font sometimes though, which implies this may be exacerbated by multiple fonts but isn't limited to that. |
|
I did add a fallback for iOS devices to use only a single font for the entire app. This is my production env, so can't have broken things there. It is troublesome to find out that this is still happening with a single font too 🤦 |
|
Hmm it actually seems like I can only get the bug to occur with your new single font when connected to Safari devtools, and it's pretty consistently bugged then. Not sure if that gives us any clue or not. |
|
Well, a little more progress... I've verified that I'm able to force the same partial glyph artifacts seen above by truncating the path commands for individual glyphs. I haven't yet been able to determine whether that is due to the original font data being incomplete (I think that would cause only one partial glyph, not many, so it seems unlikely), or if something is causing the for-loops to exit early (this seems impossible, but maybe there's some strange JIT bug). Either way, I'm getting stuck with Safari's USB connection devtools not being able to set breakpoints in the relevant code. @amitrajput1992 Would it be at all possible to get your test app as source files I can build/run locally, so I can try swapping out the troika code to debug? |
|
@lojjic While I won't be able to share the original app code, let me set up a storybook repo with a similar structure that I use in my production app and add a test render for this. Will send over the link shortly. |
|
@lojjic |
|
@amitrajput1992 Thanks for the testcase! I can replicate it there after fixing the CORS errors loading the fonts from your gmetri site by serving them from the storybook. However... then my Safari devtools just full-on crashes trying to connect to it! 🤦🤦🤦🤦🤦🤦 So I can't even add console.log statements and see them. This bug really doesn't want to get fixed, huh? Feeling frustrated; I'll try to come back to this with a fresher mind tomorrow. Let me know if you have any ideas. |
|
Hey @lojjic, I don't have a mac system with me right now, but I tested this on browserstack with local forwarding. Looks like the sdf data logged inside the web-worker vs main thread is different. It could be due to the serialization-deserialization process between thread communication, but not entirely sure. I'll keep debugging this further. |
Dumb suggestion to throw message from webworker after each line of code and console.log it, since you guys are able to reproduce bug. Picture on my iPhone 6/11: |
Not a dumb suggestion at all! And that's exactly what I was trying to do, but Safari's devtools crashes immediately for me when pointing to a local editable instance so I'm not even able to see the console.log output. :( |
|
Are you trying to open a localhost url in the connected iPhone? How does port forwarding work in this case? |
I'm accessing the dev server from the iPhone via local network IP address. I've also tried piping it through https://localhost.run. In both cases Safari devtools crashes as soon as I open it. The page itself renders just fine (though with the bug sometimes.) |
|
I read on a few blogs that this can happen in 2 scenarios:
This is a long running thread on similar lines, but no resolution https://developer.apple.com/forums/thread/92290 |
It is possible to substitute default console.log function with something like this console.log = (msg) => document.getElementById("my_ios_console").innerHTML += msg;but you need to create that div on html page or from JS script <div id="my_ios_console" style="position: absolute; top:0; left: 0; background: white"></div>this should show all console messages on main thread |
|
Thanks @munrocket, that could work, I'll give it a try. |
|
Hey All, Sorry I've been so away from this thread. Idk if this would help with debugging, but the recent Xcode 13 versions (in beta) simulator has been able to run my 3D localhost stuff fine! I ran into the issues y'all talked about before where it kept crashing with earlier versions. |
|
@lojjic Any luck with this? |
No, unfortunately, I've not been able to devote much time to this recently. |
|
there are any possibility to switch off web workers? just for check |
|
Hello everyone, I've stumbled upon this same issue in a project (for which I cannot share code unfortunately) and eventually found my way to this issue ticket right here. Since the last update on this was more than a month ago, I decided today to mess with my project's dependency code to hopefully pinpoint what exactly is happening and which lines of code are contributing to this UX disaster. Before I go any further, I want to highlight that the information and tweak below is not in any way advised, and is shared here purely to help in the search of an actual solution. The information below is NOT a solution. You've been warned. First. Yes, as mentioned earlier in the discussion, this seems indeed to be the fault of the WebWorker in iOS Safari in specific. Firefox (win10), Chrome (win10), Opera (win10), Safari (macOS big sur), and others do not present this issue and the fonts are properly loaded 100% of the time. Safari (iOS), however, runs into some sort of race condition between multiple WebWorkers (I have not identified if this is completely true nor which async calls are interfering with each other). Second. This alleged race condition (or whatever it is) is causing the buffer containing the font data being loaded to produce a few NaN values when accessed via readShort function in Troika's Typr dependency. So, is the issue actually in Typr? Perhaps. I'm not certain. However, these few NaN values cascade up the call stack ruining literally everything and finally causing the glyph to show completely wrong. Third. After pinpointing the exact location I needed (this readShort function in Typr/bin.s), I tweaked with it in quite a few ways to understand what is going on. When I simply used a console.log(Typr._bin.t.int16[0]), the application would print a few NaN that should never be (careful trying this out because the entire application may hang printing stuff - this function is called thousands of times depending on the use case). However, as expected, pausing the application anywhere inside this function (with debugger keyword or breakpoint or even accessing the console) causes the value to correct itself and not produce NaN (which led me to believe in a race condition). That means you cannot inspect the issue with a debugger in conventional ways. Fourth and finally. This is the part I advise against if not for the purpose of finding the actual solution to this problem. Note that I wrote above that if even the console is used inside the readShort function, the NaN disappear. So my ingenious hacker mindset thought the brilliant idea of including this snippet before the return statement of readShort: And it worked! All text now shows fine in iOS Safari as well as all other browsers I tested before. Problem solved~... kinda, in the worst way possible. It turns out that the brief window the application creates to access the console resolves the alleged race condition. And note that it only does so when connected to the console. In conclusion. This is where I'm at. The terrible workaround works, but I still seek an actual solution to this just as everyone here should too. It turned out that the issue may or may not be in Troika, since it may have been in Typr all along, or even iOS's implementation of WebWorker (who knows). In any case, I hope all this information helps in the investigation and we all see this one through to the end. Reference call stack: |
|
And @lojjic, regarding the troubleshoot with iOS Safari debugging via USB using a MacOS Safari: |
|
OMG @malulleybovo I returned from vacation and saw your findings and wow that was a wonderful surprise! 😃 Thank you so much for digging into this. Just knowing that readShort is producing NaNs is a huge step forward in maybe finally understanding this issue, which as you know I'd been totally stuck on for quite some time. It didn't help that I changed jobs and lost access to the iOS device I was using. My next question would be can we determine why the Typr._bin.t.int16[0] read is producing a NaN? It seems it must be getting an incorrect value in one of the buffers (either the font's The fact you can insert the console.log() there to avoid the bug is curious. I'm not sure if that indicates a race condition, or perhaps accessing the console takes it out of JIT mode. I'd hope for the former, as that sounds easier to detect and work around. |
|
@lojjic congrats on the job change! I just did some more digging into this issue right now and I got more interesting and weird news about this. Going back to the readShort code snippet I shared before, I tried peeking on the array values (with shared array buffer) and found the most bizarre thing I've seen in my software engineering career so far. readShort : function(buff, p)
{
//if(p>=buff.length) throw "error";
var a = Typr._bin.t.uint8;
a[1] = buff[p]; a[0] = buff[p+1];
/***** Right here, I peeked at Typr._bin.t.int16, Typr._bin.t.int8, and Typr._bin.t.uint8 ******/
return Typr._bin.t.int16[0];
},While peeking at the first occurence of NaN in readShort within my application, I found out that buff[p]=255 and buff[p+1]=6 (both valid uint8 values). With that in mind I peeked at the values of Typr._bin.t.uint8 and found it had [6, 255, 0, ...] as expected. Then I peeked at Typr._bin.t.int16 and found the erroneous [NaN, 0, ...] instead of [-250, 0, ...]. Lastly, I peeked at Typr._bin.t.int8 and... it was also wrong... it was [6, NaN, 0, ...] instead of [6, -1, 0, ...]. Typr glyf library uses one shared ArrayBuffer on multiple Arrays of different type (Uint8Array, Int8Array, Int16Array, etc.). This occurrence showed me that in iOS Safari (only), after altering one of these arrays, the values on the others might not get updated immediately. No clue why, but I found a resolved bug report involving ArrayBuffer in iOS Safari in recent history which makes this more believable... proven to be flawed once, may well be proven flawed twice (ref: (https://bugs.webkit.org/show_bug.cgi?id=194268)[https://bugs.webkit.org/show_bug.cgi?id=194268]). Having uncovered this, I decided to try an alternate implementation of the readShort : function(buff, p)
{
var a = buff[p + 1] + (buff[p] << 8);
if (a > 0b0111111111111111) {
a = (~a) + 1;
}
a = (a < 0 ? -1 : 1) * (a & 0b1111111111111111);
return a;
},And there you have it. All text with the font now shows correctly always (even without being connected to devtools - refer to my previous comment about the console.log thing to understand this disclaimer) This alternative solution fixed the problem in iOS Safari (tested on iOS 15.0.2), and continues to work in the previous browsers I tested on before just like it did before, such as Chrome v95.0.4638.54 (win10), Firefox v93.0 (win10), Opera v80.0.4170.63 (win10), and MacOS Safari (MacOS Big Sur v11.3.1). *If anyone can optimize my code snippet above, please feel free~ In the end, it looks to me like this issue is not caused by troika. Troika seems to in fact be suffering the consequences of a deeper issue. So I would personally move this issue to Typr instead. But what do I know... test it out for yourselves and lets argue if this really is the root of the problem. ;) |
|
I think @malulleybovo deserves an award or something! 🥳 This is amazing, narrowing it down and coming up with an alternate implementation that avoids the issue! Thank you sooo much! I'm happy to integrate your It seems like there's something wrong/dangerous with the typed-arrays-sharing-a-buffer pattern in general. It's a pretty curious pattern now that I think about it. It seems like |
|
Thanks for the compliment @lojjic~ I'm glad I was helpful. Your proposed solution seemed promising so I just tested it out and guess what, it also works (on all browsers I listed before)! My application depends on Typr's glyf script, which uses Typr's readInt8, readShort, readUshort, and readBytes. Although I've included your full solution for testing purposes, I've only tested it on these functions. And everything works by the looks of it. For a more in depth look at the effectiveness of this solution, I would test out the other scenarios. But I lack concrete examples to test these on (besides just unit testing). I believe Typr's readFixed, readF2dot14, readUshorts, readUint64, readASCII, readUnicode, readUTF8, readBytes, and readASCIIArray from bin.js wouldn.t need to be changed since they don't directly use the typed arrays. So only the functions in your gist would need to be altered within Typr. Plus, along with this change, Typr's shared ArrayBuffer and typed arrays will become obsolete. If more devs can test and give the approval to this, we'll be even more confident in the solution. This is since I do have a limited number of test cases and test devices at my disposal and there's a small chance the test result is biased. |
Amazing!!! I can hardly believe it. 🎉 🥳 I'll give this some more testing to see if I can validate the other functions, and do a basic performance comparison. Unless something nasty shows up I'll get this integrated asap. I'm pretty confident including this in a troika-three-text release so people can try it out; the community will let us know if any issues crop up with it. Once it's been out in the wild a bit I'll submit it upstream to Typr. |
|
Performance seems comparable, even a bit faster in some cases. Extra win! 😄 I've verified the other functions work too. I had to modify the |
See #88. This switches to a private fork of Typr.ts which has an experimental fix for the issue.
|
I've published troika-three-test version Anyone who is able (@amitrajput1992? @strangerintheq? @atlmtw?), I would greatly appreciate testing with this version to verify that it fixes the issue in your specific applications. I will attempt to do the same either with Browserstack or finding an iPhone to borrow. Thanks in advance! |
|
Hey @lojjic it's good to hear that there's a fix for it. Let me test this quickly and get back to you. |
|
@amitrajput1992 Hi, have you had a chance to test the alpha yet? I want to get this released and would love the extra validation. :) |
|
@lojjic Hey, I just got the time to test this out. Looks like it's working now!! Checkout the changes here: https://amitrajput1992.github.io/r3f-experiments/?path=/story/testers--text-tester |
|
I've released version |
Hello!
First off I would like to say that this package has been WAY easier to use than other things in the past. So thank you very much for putting this out!
Anyways I've recently put this on a site I'm still working on and noticed some inconsistencies on mobile. Sometimes everything would load but other times things would be missing or incomplete. Screenshots below.
Would you happen to know what may be going on? Or if I am doing anything wrong?
The big font with "welcome" as the text is an .otf file (restgold.otf)
The small text with "Hi my name is..." as the text is a .ttf file (Raleway-Medium.ttf)
If you need the font files let me know!
Device: IPhone7
IOS: 14.1
Browser: Safari
Package Details:
"three": "^0.122.0",
"troika-three-text": "^0.35.0",
"troika-three-utils": "^0.35.0",
The text was updated successfully, but these errors were encountered: