Closed Bug 1672314 Opened 4 years ago Closed 4 years ago

Light and Dark themes missing from about:addons and Customize Mode after upgrading to Firefox 82

Categories

(Toolkit :: Add-ons Manager, defect, P1)

Product:
Toolkit
Component:
Add-ons Manager
Version:
Firefox 82
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
84 Branch
Tracking Flags:
Tracking Status
firefox-esr78 --- unaffected
firefox81 --- unaffected
firefox82 --- wontfix
firefox83 + wontfix
firefox84 --- fixed

People

(Reporter: jscher2000, Assigned: mixedpuppy)

References

(Regression)

Details

(Keywords: regression)

Attachments

(2 files)

addonStartup.json.lz4
1.49 KB, application/octet-stream
Details
Bug 1672314 handle invalid addon startup data properly during startup
47 bytes, text/x-phabricator-request
Details | Review

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0

Steps to reproduce:

Check Themes section of about:addons or Themes button in Customize Mode.

Actual results:

Cannot find Light or Dark themes.

Curiously they are not listed when I execute this in the Browser Console:

var tlist = AddonManager.getAddonsByTypes(["theme"]);
tlist.then((themes) => {console.log(themes.map(t => t.name));});

But they are physically present on

resource://builtin-themes/

Expected results:

Light and Dark themes should be available options. (They are in 83.0b2.)

I notice this doesn't affect my newer test profiles, so it might be caused by some legacy files or preferences.

It came up on Reddit, so more testers might appear here: https://www.reddit.com/r/firefox/comments/jf3xd3/what_happened_to_the_default_themes_firefox_820/

My main profile dates back to Nov. 2017 (Firefox 57) and I have this non-default preference value set:

extensions.update.autoUpdateDefault = false

I went to about:addons, clicked the gear button, and clicked Check for Updates,. Firefox reported None found.

I exited/restarted Firefox and the Light, Dark and Default themes were back.

I don't know whether those things are causally linked, but I suspect they might be.

I encountered this issue after upgrading to Firefox 82 (Firefox from ArchLinux's repository). The console errors are consistent, the disappearance of items in about:addons is not. But I personally witnessed that the default themes were gone, except for Alpenglow.

mozregression with the STR below (and a custom bisect script) points to bug 1660557 as the likely regressor.

I'm moving this over to the Add-ons Manager component because the fix is probably to fixup the database in some kind of migration.

STR:

  1. Create profile directory with Firefox 81, e.g. /tmp/profdir81
  2. Start Firefox >= 82 with that profile directory, firefox --no-remote -profile /tmp/profdir81
  3. Open the global browser console and scroll to the top.

Expected:

  • no significant errors.

Actual:

  • logs below.

The global JS console (Ctrl-Shift-J) contains the following errors related to this bug:

1603810249736	addons.xpi-utils	WARN	updateMetadata: Add-on firefox-compact-dark@mozilla.org is invalid: Error: File resource:///modules/themes/dark/ does not contain a valid manifest(resource://gre/modules/addons/XPIInstall.jsm:669:11) JS Stack trace: loadManifest@XPIInstall.jsm:669:11
awaitPromise@XPIProvider.jsm:228:15
syncLoadManifest@XPIInstall.jsm:747:24
updateMetadata@XPIDatabase.jsm:2930:32
updateExistingAddon@XPIDatabase.jsm:3168:23
processFileChanges@XPIDatabase.jsm:3255:31
checkForChanges@XPIProvider.jsm:2999:55
startup@XPIProvider.jsm:2457:12
callProvider@AddonManager.jsm:230:31
_startProvider@AddonManager.jsm:601:17
startup@AddonManager.jsm:825:14
startup@AddonManager.jsm:3530:26
observe@addonManager.js:87:29
1603810249737	addons.xpi-utils	WARN	updateMetadata: Add-on default-theme@mozilla.org is invalid: Error: File resource://gre/modules/themes/default/ does not contain a valid manifest(resource://gre/modules/addons/XPIInstall.jsm:669:11) JS Stack trace: loadManifest@XPIInstall.jsm:669:11
awaitPromise@XPIProvider.jsm:228:15
syncLoadManifest@XPIInstall.jsm:747:24
updateMetadata@XPIDatabase.jsm:2930:32
updateExistingAddon@XPIDatabase.jsm:3168:23
processFileChanges@XPIDatabase.jsm:3255:31
checkForChanges@XPIProvider.jsm:2999:55
startup@XPIProvider.jsm:2457:12
callProvider@AddonManager.jsm:230:31
_startProvider@AddonManager.jsm:601:17
startup@AddonManager.jsm:825:14
startup@AddonManager.jsm:3530:26
observe@addonManager.js:87:29
1603810249737	addons.xpi-utils	WARN	updateMetadata: Add-on firefox-compact-light@mozilla.org is invalid: Error: File resource:///modules/themes/light/ does not contain a valid manifest(resource://gre/modules/addons/XPIInstall.jsm:669:11) JS Stack trace: loadManifest@XPIInstall.jsm:669:11
awaitPromise@XPIProvider.jsm:228:15
syncLoadManifest@XPIInstall.jsm:747:24
updateMetadata@XPIDatabase.jsm:2930:32
updateExistingAddon@XPIDatabase.jsm:3168:23
processFileChanges@XPIDatabase.jsm:3255:31
checkForChanges@XPIProvider.jsm:2999:55
startup@XPIProvider.jsm:2457:12
callProvider@AddonManager.jsm:230:31
_startProvider@AddonManager.jsm:601:17
startup@AddonManager.jsm:825:14
startup@AddonManager.jsm:3530:26
observe@addonManager.js:87:29
1603810249791	addons.xpi-utils	WARN	updateMetadata: Add-on firefox-alpenglow@mozilla.org is invalid: Error: File resource:///modules/themes/alpenglow/ does not contain a valid manifest(resource://gre/modules/addons/XPIInstall.jsm:669:11) JS Stack trace: loadManifest@XPIInstall.jsm:669:11
awaitPromise@XPIProvider.jsm:228:15
syncLoadManifest@XPIInstall.jsm:747:24
updateMetadata@XPIDatabase.jsm:2930:32
updateExistingAddon@XPIDatabase.jsm:3168:23
processFileChanges@XPIDatabase.jsm:3255:31
checkForChanges@XPIProvider.jsm:2999:55
startup@XPIProvider.jsm:2457:12
callProvider@AddonManager.jsm:230:31
_startProvider@AddonManager.jsm:601:17
startup@AddonManager.jsm:825:14
startup@AddonManager.jsm:3530:26
observe@addonManager.js:87:29

and followed shortly by

1603810252318	addons.xpi-utils	WARN	updateMetadata: Add-on default-theme@mozilla.org is invalid: Error: File resource://gre/modules/themes/default/ does not contain a valid manifest(resource://gre/modules/addons/XPIInstall.jsm:669:11) JS Stack trace: loadManifest@XPIInstall.jsm:669:11
awaitPromise@XPIProvider.jsm:228:15
syncLoadManifest@XPIInstall.jsm:747:24
updateMetadata@XPIDatabase.jsm:2930:32
updateExistingAddon@XPIDatabase.jsm:3168:23
processFileChanges@XPIDatabase.jsm:3255:31
getNewSideloads@XPIProvider.jsm:3035:28
1603810252343	addons.xpi-utils	WARN	updateMetadata: Add-on firefox-compact-dark@mozilla.org is invalid: Error: File resource:///modules/themes/dark/ does not contain a valid manifest(resource://gre/modules/addons/XPIInstall.jsm:669:11) JS Stack trace: loadManifest@XPIInstall.jsm:669:11
awaitPromise@XPIProvider.jsm:228:15
syncLoadManifest@XPIInstall.jsm:747:24
updateMetadata@XPIDatabase.jsm:2930:32
updateExistingAddon@XPIDatabase.jsm:3168:23
processFileChanges@XPIDatabase.jsm:3255:31
getNewSideloads@XPIProvider.jsm:3035:28
1603810252400	addons.xpi-utils	WARN	updateMetadata: Add-on firefox-compact-light@mozilla.org is invalid: Error: File resource:///modules/themes/light/ does not contain a valid manifest(resource://gre/modules/addons/XPIInstall.jsm:669:11) JS Stack trace: loadManifest@XPIInstall.jsm:669:11
awaitPromise@XPIProvider.jsm:228:15
syncLoadManifest@XPIInstall.jsm:747:24
updateMetadata@XPIDatabase.jsm:2930:32
updateExistingAddon@XPIDatabase.jsm:3168:23
processFileChanges@XPIDatabase.jsm:3255:31
getNewSideloads@XPIProvider.jsm:3035:28
Status: UNCONFIRMED → NEW
Component: Untriaged → Add-ons Manager
Ever confirmed: true
Keywords: regression
Product: Firefox → Toolkit
Regressed by: 1660557
Has Regression Range: --- → yes
status-firefox81: --- → unaffected
status-firefox82: --- → affected
status-firefox83: --- → affected
status-firefox84: --- → affected
status-firefox-esr78: --- → unaffected
Summary: Light and Dark themes missing from about:addons and Customize Mode → Light and Dark themes missing from about:addons and Customize Mode after upgrading to Firefox 82

From looking at the stack traces, it's obvious that there is no migration to the new extensions from bug 1660557 at startup.
The new extensions are registered too late... at final-ui-startup, whereas the add-on database migration happens "synchronously" before any UI occurs at all.

The expected migration logic from bug 1660557 could never have worked. I have yet to check for the best solution here. Whether it's a migration of paths of built-in themes in the database, and/or whether we need to (somehow) support the installation/update of a (new) (builtin) (theme) add-on after corruption of the previous install.

Gijs can you look at this regression?

Flags: needinfo?(gijskruitbosch+bugs)

(In reply to Julien Cristau [:jcristau] from comment #5)

Gijs can you look at this regression?

My impression is that Rob is already doing this, and he has a much better understanding of this problemspace than I do.

Flags: needinfo?(gijskruitbosch+bugs) → needinfo?(rob)

Shane said that he's going to look into this.

Assignee: nobody → mixedpuppy
Status: NEW → ASSIGNED
Flags: needinfo?(rob)

Tracking for a potential uplift to 83beta.

tracking-firefox83: --- → +

Alex,

Can you help with STR here? Some of us are having a harder time reproducing this issue consistently. Thanks!

Flags: needinfo?(acornestean)

Although the STR in comment 3 is able to consistently trigger the console errors, it cannot be used to reproduce the disappeared themes.
I pulled my profile from a recent backup and am able to reproduce the issue. So by removing files from the profile directory, I am able to pinpoint the specific issue further:

baseline: eliminating user data:

  • Starting with old profile, last accessed with Firefox 81
  • Using a path name different from the original profile directory - no difference
  • previous + removing IndexedDB (storage/) and databases (*.sqlite) - no difference
  • previous + removing prefs.js - no difference (=baseline for later)

baseline2: without extension files

  • baseline + removing *.json (including extensions.json) - no difference
  • baseline + removing extensions/*.xpi - no difference
  • baseline + remove *.json and extensions/ - no difference (=baseline2 for later)

tests: without userdata and extensions

  • baseline2 + removing addonStartup.json.lz4 - themes are back. (also tried with just baseline, same result)
  • previous + addonStartup.json.lz4 from backup - reproduced (usually, when Firefox 82 is restarted, the themes are back).

final test:

  • empty directory with addonStartup.json.lz4 from backup - reproduced

Now, to reduce it further I use mozlz4a.py to decompress and re-compress the file.
addonStartup.json.lz4 has the following keys:

  • app-profile
  • app-system-defaults
  • app-builtin (this contains the built-in themes)

Reductions:

  • Keeping ONLY app-builtin, after removing app-profile and app-system-defaults - can NOT reproduce
  • Keeping ONLY app-builtin, but prepending a dummy key such as "dummy": {},- REPRODUCED ( why...? I also tried short and long keys, same result)
  • Keeping ONLY app-builtin, but prepending "app-profile": {} - can NOT reproduce (strange, same as dummy key except key name is "known")
  • Keeping ONLY app-builtin, but prepending "app-profile": {"addons":{}} - can NOT reproduce
  • Keeping ONLY app-builtin, but prepending "app-profile": {"addons":{"@dummy":{}}} - REPRODUCED

I will attach a test case to reproduce the disappearing themes AND the console errors from comment 3.

Flags: needinfo?(acornestean)
Attached file addonStartup.json.lz4 —

Minimal reduction, see comment 10.

STR:

  1. Create empty directory, profdir.
  2. Put this file, addonStartup.json.lz4 in there.
  3. Start Firefox with this directory, e.g. firefox --no-remote -profile profdir --offline about:addons
  4. Click on Themes.

Expected (Firefox 81):

  • about:addons shows list of default themes (default, dark, alpenglow, light)

Actual (Firefox 82):

  • about:addons only shows Alpenglow theme.
Severity: -- → S3
Priority: -- → P1
Severity: S3 → S2
Pushed by scaraveo@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/09a20cd2699f handle invalid addon startup data properly during startup r=rpl
Pushed by scaraveo@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/639345ca2c28 handle invalid addon startup data properly during startup r=rpl

https://hg.mozilla.org/mozilla-central/rev/639345ca2c28

Status: ASSIGNED → RESOLVED
Closed: 4 years ago
status-firefox84: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → 84 Branch

@rpl and I ran through any scenarios that would make us want to uplift this and found no reason, so it will ride the train. Any users on 82 are already affected, and a restart fixes the problem. The issue only affects builtin addons if they change the path they are located at. The next ESR will get both the path change and the fix at the same time.

Flags: needinfo?(mixedpuppy)
Blocks: 1682300
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: