1639705 - staged upgrade uninstalls addon, resulting in data loss

Status: RESOLVED FIXED

(Toolkit :: Add-ons Manager, defect, P2)

Description

[[e-mail]]

This is an issue which has been reported in uBlock Origin (uBO) issue tracker: https://github.com/uBlockOrigin/uBlock-issues/issues/1054.

Steps to reproduce:

1. Install uBO 1.27.0, click https://addons.mozilla.org/firefox/downloads/file/3568893/ublock_origin-1.27.0-an+fx.xpi.
2. Restart the browser
3. Go to about:addons => "Manage Extensions Shortcuts"
4. Assign a keyboard shortcut to uBO's "Relax blocking mode", for example, Alt-Z
5. Restart the browser
6. Result: as expected, the assigned shortcut is still there
7. Go to about:addons and click "Check for Updates"
8. Result: uBO's three-dot menu shows that an update is available
9. Restart the browser
10. Result: unexpected, the assigned shortcut is gone

The reporter on uBO's issue tracker wisely noticed:

The only thing I know of that makes uBO different from every other extension is that its updates don't get applied instantly when they are downloaded. Instead you have to restart the browser or explicitly disable and then re-enable uBO

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:mixedpuppy, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 3

[Rob Wu [:robwu]]

I'll investigate.

Comment 5

[[e-mail]]

To be clear, it seems deferring the restart of the extension upon update is central to the issue here -- i.e. it works fine when allowing the extension to restart immediately upon update (through https://github.com/gorhill/uBlock/wiki/Advanced-settings#extensionupdateforcereload). I can reproduce when restart is deferred (default behavior), and the issue disappears when restart is not deferred (extensionUpdateForceReload set to true).

Comment 6

[Rob Wu [:robwu]]

When the extension uses the runtime.onUpdateAvailable, updates are not immediately installed but staged for the next restart. This is implemented as uninstall+install rather than update. Due to the uninstall step, some changes from extensions are unexpectedly reverted after an update:

• Shortcuts are reset to the defaults ref (this bug)
• There is an attempt to revert the default search setting and "Your New Tab has changed" notification reappears ref (search settings are not correctly reverted because the search module is not initialized at this point and an error is thrown before the search setting are removed.
• "Developer Tools installed by add-ons" is reset (shown by default) ref
• The "Access Your Hidden Tabs" prompt re-appears when the tabs.hide API is used ref
• "Your New Tab has changed." notification re-appears ref
• Any extension API that changes a pref through the ExtensionPreferencesManager would have their changes reverted ref. This doesn't happen in practice because the module doesn't appear to have been loaded at early startup. In the light of this bug, this is good, because
  otherwise there would have been dataloss (e.g. bug 1549204).

We should fix this by doing an update.

Output of Firefox Beta 78.0b2

1591350755139	addons.xpi	DEBUG	Install of https://addons.mozilla.org/firefox/downloads/file/3574596/ublock_origin-1.27.6-an+fx.xpi?src=dp-btn-primary completed.
1591350755142	addons.xpi	DEBUG	Loading bootstrap scope from jar:file:///tmp/testprof/prof/extensions/[e-mail]!/
1591350755142	addons.xpi	DEBUG	Calling bootstrap method install on [e-mail] version 1.27.6
1591350755142	addons.xpi	DEBUG	Calling bootstrap method startup on [e-mail] version 1.27.6
1591350755383	addons.xpi	DEBUG	removeTemporaryFile: https://addons.mozilla.org/firefox/downloads/file/3574596/ublock_origin-1.27.6-an+fx.xpi?src=dp-btn-primary removing temp file /tmp/tmp-fe6.xpi
1591350757018	addons.manager	DEBUG	Registering upgrade listener for [e-mail]
1591350779452	addons.update-checker	DEBUG	Requesting https://versioncheck.addons.mozilla.org/update/VersionCheck.php?reqVersion=2&id=[e-mail]&version=1.27.6&maxAppVersion=null&status=userEnabled&appID={ec8030f7-c20a-464f-9b0e-13a3a9e97384}&appVersion=78.0&appOS=Linux&appABI=x86_64-gcc3&locale=en-US&currentAppVersion=78.0&updateType=97&compatMode=normal
1591350779463	addons.productaddons	INFO	sending request to: https://aus5.mozilla.org/update/3/GMP/78.0/20200602134412/Linux_x86_64-gcc3/en-US/beta/Linux%205.6.8-arch1-1%20(GTK%203.24.20%2Clibpulse%2013.0.0)/default/default/update.xml
1591350779711	addons.productaddons	INFO	Completed downloading document
1591350780076	addons.update-checker	DEBUG	Found an update entry for [e-mail] version 1.27.10
1591350780100	addons.xpi	DEBUG	Download started for https://addons.cdn.mozilla.net/user-media/addons/607454/ublock_origin-1.27.10-an+fx.xpi?filehash=sha256%3Ad40f84113e7d7fa6289bdb8192fd9477abc4ed6d5dd40de55ddc79bed1ea070c to file /tmp/tmp-suk.xpi
1591350780138	addons.xpi	DEBUG	Download of https://addons.cdn.mozilla.net/user-media/addons/607454/ublock_origin-1.27.10-an+fx.xpi?filehash=sha256%3Ad40f84113e7d7fa6289bdb8192fd9477abc4ed6d5dd40de55ddc79bed1ea070c completed.
1591350780482	addons.xpi	INFO	add-on [e-mail] has an upgrade listener, postponing upgrade until restart
1591350780486	addons.xpi	DEBUG	Addon [e-mail] will be installed as a packed xpi
1591350780496	addons.xpi	DEBUG	Staged install of [e-mail] from https://addons.cdn.mozilla.net/user-media/addons/607454/ublock_origin-1.27.10-an+fx.xpi?filehash=sha256%3Ad40f84113e7d7fa6289bdb8192fd9477abc4ed6d5dd40de55ddc79bed1ea070c ready; waiting for restart.

Then after a restart

1591350885889	addons.xpi	DEBUG	checkForChanges
1591350885889	addons.xpi	DEBUG	Loaded add-on state: {"app-profile":{"addons":{"[e-mail]":{"dependencies":[],"enabled":true,"lastModifiedTime":1591350755000,"loader":null,"path":"[e-mail]","rootURI":"jar:file:///tmp/testprof/prof/extensions/[e-mail]!/","runInSafeMode":false,"signedState":2,"signedDate":1590062556000,"telemetryKey":"uBlock0%40raymondhill.net:1.27.6","version":"1.27.6","startupData":{"persistentListeners":{"webRequest":{"onBeforeRequest":[[{"incognito":null,"tabId":null,"types":null,"urls":["moz-extension://c8c9e035-f8f5-4650-86f8-787f38677fbb/web_accessible_resources/*"],"windowId":null},["blocking"]],[{"incognito":null,"tabId":null,"types":null,"urls":["http://*/*","https://*/*","ws://*/*","wss://*/*"],"windowId":null},["blocking"]],[{"incognito":null,"tabId":null,"types":["csp_report"],"urls":["http://*/*","https://*/*"],"windowId":null},["blocking","requestBody"]]],"onHeadersReceived":[[{"incognito":null,"tabId":nul…

1591350885892	addons.xpi	DEBUG	scanForChanges changed: false, state: {}
1591350886288	addons.xpi	DEBUG	Processing install of [e-mail] in app-profile
1591350886289	addons.xpi	DEBUG	Loading bootstrap scope from jar:file:///tmp/testprof/prof/extensions/[e-mail]!/
1591350886289	addons.xpi	DEBUG	Calling bootstrap method uninstall on [e-mail] version 1.27.6
1591350886290	addons.xpi	DEBUG	XPIStates adding add-on [e-mail] in {"addons":{"[e-mail]": ...
1591350886291	addons.xpi	DEBUG	Updating XPIState for {"id":"[e-mail]", ...

1591350886291	addons.xpi-utils	DEBUG	Synchronously loading the add-ons database
1591350886291	addons.xpi-utils	DEBUG	Starting async load of XPI database /tmp/testprof/prof/extensions.json
1591350886292	addons.xpi-utils	DEBUG	Finished async read of XPI database, parsing...
1591350886293	addons.xpi-utils	DEBUG	Successfully read XPI database
1591350886294	addons.xpi-utils	DEBUG	Add-on [e-mail] modified in app-profile
1591350886294	addons.manager	DEBUG	Registering startup change 'changed' for [e-mail]
1591350886295	addons.xpi	DEBUG	Loading bootstrap scope from jar:file:///tmp/testprof/prof/extensions/[e-mail]!/
1591350886295	addons.xpi	DEBUG	Calling bootstrap method install on [e-mail] version 1.27.10
1591350886295	addons.xpi-utils	DEBUG	Make addon app-profile:[e-mail] visible

Comment 7

[Shane Caraveo (:mixedpuppy)]

It seems like uninstall is being called for a staged extension[1] when an update is occurring. We should call update, which will call uninstall, but skip the uninstall event[2]. It seems that installStagedAddon should adopt some of the logic from startInstall[3].

The distinction between install and update seems to be handled everywhere else, just not in this path.

[1] https://searchfox.org/mozilla-central/rev/35b97af64a55d1d30caa4d6e9fabc1a7fbabc509/toolkit/mozapps/extensions/internal/XPIInstall.jsm#3853
[2] https://searchfox.org/mozilla-central/rev/fac90408bcf52ca88a3dcd2ef30a379b68ab24e2/toolkit/mozapps/extensions/internal/XPIProvider.jsm#2100
[3] https://searchfox.org/mozilla-central/rev/35b97af64a55d1d30caa4d6e9fabc1a7fbabc509/toolkit/mozapps/extensions/internal/XPIInstall.jsm#1731

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:mixedpuppy, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 9

[Shane Caraveo (:mixedpuppy)]

Attachment: Bug 1639705 do not uninstall if upgrading an addon via stagged install

Comment 10

[Jeffery Small]

Just wanted to confirm that I once again lost my uBlock Origin shortcuts when Firefox automatically updated to 78.0.2 yesterday (as per bug 1641846 ). This is on Xubuntu 19.10 desktop. Things had been stable for a while until this event.

Comment 11

[tero gusto]

I can confirm this: I also lost my uBlock Origin shortcuts the other day, after not experiencing this for a while.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/442e71a3ede0
do not uninstall if upgrading an addon via stagged install r=aswan

Comment 13

[Razvan Maries]

Backed out for perma failures.

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&searchStr=windows%2C7%2Cshippable%2Copt%2Cmochitests%2Ctest-windows7-32-shippable%2Fopt-mochitest-browser-chrome-e10s%2Cbc5&revision=442e71a3ede062fed707ed3934b5cb78c05bad52&selectedTaskRun=cczkpyeESDq--4aVJVWlCw.0

Log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=310287257&repo=autoland&lineNumber=2557

Backout: https://hg.mozilla.org/integration/autoland/rev/e785ebabf7e142898e31a6c81a4d43d44eff39e3

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c684e903d22a
do not uninstall if upgrading an addon via stagged install r=aswan

Comment 15

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/c684e903d22a

Comment 16

[Jeffery Small]

Firefox 78.0.2
Xubuntu 19.10

I found this bug to be resolved a month or so ago, but it is back. The past two times I rebooted, my extension keyboard shortcuts (specifically for uBlock Origin, but probably for all of them) are missing and must be manually reset.

This was originally reported in bug:

https://bugzilla.mozilla.org/show_bug.cgi?id=1641846

Comment 17

[Shane Caraveo (:mixedpuppy)]

(In reply to Jeffery Small from comment #16)

Firefox 78.0.2
Xubuntu 19.10

I found this bug to be resolved a month or so ago, but it is back. The past two times I rebooted, my extension keyboard shortcuts (specifically for uBlock Origin, but probably for all of them) are missing and must be manually reset.

The version that this is fixed in is Firefox 80.