setting privacy.firstparty.isolate breaks script storage

[kekkc]

Hi all,

if anyone reads it before upgrading to FF 68: create a backup of your scripts.

After the automatic update, all my scripts were gone. Exceptions are still included (so GM seems pull some of my past settings), but all scripts are missing.

Where is GM storing the data currently? Within the folder browser-extension-data, the GM file was not changed. Still investigating, but this seems to be totally weird.

[arantius]

So I saw this and checked my version now: 67. But it had an update to apply, so I did. Now I'm on 68 and all my scripts are in place just fine.

Scripts are stored in IndexedDB. There's no simple answer to this, there's random IDs involved (beyond our control).

[kekkc]

OK so it seems it's no general issue.

The only thing I changed after the upgrade was to update settings > security > custom > block identification trackers (don't know the exact english translation). But changing it back didn't bring back my scripts :/

Will search for the IndexedDB location (I know the ID from about:support, but not the directory) & some of my backups.

[kekkc]

"profile > storage > default > about+newtab^firstPartyDomain=about.MYGMID > idb > 3312185054sbndi_pspte.sqlite" --> file not changed since a month according to file timestamp

"profile > storage > permanent > moz-safe-about+home^firstPartyDomain=about.MYGMID > idb > 818200132aebmoouht.sqlite" --> file not changed since a month according to file timestamp

But also neither of those files contains my scripts any more.

[daleeidd]

This just happened to me too. Firefox 68.0 on macOS. Confirmed issue with different profiles.

[kekkc]

This just happened to me too. Firefox 68.0 on macOS. Confirmed issue with different profiles.

Found out that simply copying an old profile to FF 68 will not restore your scripts. I had to download FF 67 Portable, copy my old profile backup there & export the scripts out of FF 67 (didn't try reimporting to FF68). So it seems that FF stores the scripts somewhere inside the profile folder, but FF68 cannot read this or doesn't migrate correctly from FF67.

Update: GM saves the scripts within "profile\storage\default\moz-extension+++MYGMID\idb\XXX.sqlite". This is a different ID than shown in about:support. Seems all scripts are still there, now it would just be interesting why FF68 cannot read them anymore for some configurations ;)

[kekkc]

Update 2: I can reproduce regularly, too with my backup profile. Unfortunately I cannot share this profile and I was unable to reproduce with a clean FF portable installation.

@daleeidd Do you have a more detailed description or "empty" profile that you can share to make this reproducible for arantius, too?

I think we should investigate. This is the worst case and should not happen, even if it's only for 1% of the users.

[Sxderp]

This sounds like the add-on id may have changed between updates.

[kekkc]

Checked that within about:debugging: shows still the same addon ID & internal UUID.

[kekkc]

OK, I found it.

The problem is the setting "privacy.firstparty.isolate;true". Usually GM stores it's settings in these 2 folders:

• profile\storage\default\moz-extension+++MYGMID
• profile\storage\default\moz-extension+++MYGMID^userContextId=MYCONTID

If 1st party isolation is enabled and you upgrade to FF 68, a 3rd folder is created:

• profile\storage\default\moz-extension+++MYGMID^firstPartyDomain=MYGMID

If you disable 1st party isolation ("privacy.firstparty.isolate;false") and then delete the 3rd created folder mentioned above, your scripts will become visible again.

[daleeidd]

@kekkc Thank you. Just followed your instructions to recover the scripts for export. Good job!

[arantius]

Great diagnostics. I'm not sure what to do about this from within Greasemonkey, though?

[arantius]

https://bugzilla.mozilla.org/show_bug.cgi?id=1564593

[kekkc]

Somehow had the hope that there's a workaround in GM possible. All my other extensions that use the same storage (e.g. https://addons.mozilla.org/de/firefox/addon/textnotes/?src=search ) didn't cause FF to create a new folder during upgrade, even with 1st party isolation turned on.

[arantius]

That extension does not use IndexedDB.

[lostfictions]

Can confirm that disabling first-party isolation (I had it enabled via this add-on, a simple toggle) and restarting FF caused my scripts to show up again. I didn't need to delete any folders.

Once I'd safely exported the scripts, re-enabling first-party isolation and then re-importing the scripts resolved the issue, so FPI doesn't seem to break Greasemonkey in a persistent or permanent way. It's maybe worth noting that (if I recall correctly) this is the same thing that happens to all (non-extension) browsing data when a user initially enables first-party isolation, so it's possible that FF68 has just extended normal FPI behaviour to apply to extensions as well.

[lostfictions]

Poking around a bit, this bug mentions add-on breakage with FPI as an aside -- couldn't find another confirmed bug specifically for this issue at a glance.

[kekkc]

Thx, mentioned the bug at bugzilla. Maybe https://bugzilla.mozilla.org/show_bug.cgi?id=1564593 should also reference the FPI (FPI was introduced as part of the TorBrowser Uplift and was as big to Mozilla as the UserScript API)

[tomrittervg]

Just a note that we (Mozilla) have seen this. I think the most relevant Bugzilla bug 1554805.

Unfortunately we don't have anyone actively working on this at the moment but maybe we can find some cycles to fix this. Unfortunately the 'fix' is likely to delete everything again; but at least enabling/disabling FPI won't switch out the storage for the extension...

[b16r05]

Not the place for me to whine as I don't have GM installed, but, same thing happened to me and nothing works. I had FPI installed, but disabled. On upgrade to FF 68, both onetab and stylus lost data.

I carried on about this here: openstyles/stylus#747

@kekkc what FF 67 Portable solution did you use? I have tried Portable apps FF and FF ESR, both say that I must not use an old FF profile... and I have no idea on how to override that dialog box. Somehow, update to FF 68 did something to my profile and FF 66.0.4 and older keeps saying to use a new profile.

[kekkc]

@b16r05 was on vacation, you probably figured it out already. I just copied the profile folder and didn't start FF 67 Portable before. I remember the same error message, but I think this method worked at the end.

[func0der]

I can NOT confirm this behavior. I have it (FF68) disabled.
I can neither see my scripts, nor can I create new ones or install anything.
It just does nothing.

When I install new scripts, it says 'undefined' in the final dialog.
When I click on "add new script", it puts the following error messages in the console:

IndexedDB UnknownErr: ActorsParent.cpp:581
Error opening user-scripts DB! <unavailable> user-script-registry.js:57:15
undefined
Error: undefined

I tried uninstalling it, deleting the folders storage\default\moz*, renamed the gm_scripts folder, re-installing it: It still does not work.

[kekkc]

Seems this was fixed in FF71 https://bugzilla.mozilla.org/show_bug.cgi?id=1554805

[hklene]

On upgrade from FF70 to FF71 I lost all my scripts and stylus-styles for the second time in 2019. privacy.firstparty.isolate is enabled. But this time I made a backup of my profile before upgrading.

I could downgrade my installation temporarily back to 70, unpack my backup and export the scripts and styles, that I redid since the other data-loss earlier.

This teaches me, to keep an external copy of all my work. Then I can check them into some version control and work on them in a proper editor. Can you recommend some hassle-free way to automatically deploy my externally changed files into GM+Firefox other than copy paste?