691402 - Addons partially removed on upgrade

Status: RESOLVED WORKSFORME

(Toolkit :: Add-ons Manager, defect)

Description

[a_geek]

I recently upgraded to 7.0 and lost a number of addons, then reinstalled some from addons.mozilla.org, then, today, found that a number of addons were disabled again. Seeing that, I exited FF7 and installed the upgrade to 7.0.1 (systemwide), then restarted FF again. But my addons are still missing. Then I also installed this "addon recovery tool" addon and followed the instructions, but my addons are still hidden/not there. The addons are not listed in the "Extensions" section of "about:support", and they are not present in the extensions directory of the profile. But addons.sqlite in the profile still lists them. It would be nice if FF would recover all the addons automatically.

This is with FF 7.0.1 (Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1).

Comment 1

[Dave Townsend [:mossop]]

We'll need some more information to help us track this one down:

1. Can you remember what add-ons got lost in the upgrade to 7.0?

2. When you say some got disabled again do you mean they vanished from the add-ons manager again? Can you remember which add-ons went this time?

3. Could you attach a copy of extensions.log from your profile folder.

Comment 2

[Jorge Villalobos [:jorgev] (he/him)]

A number of add-on developers have forwarded similar reports from their users. Only one has been able to give us an extensions.log, and it has multiple entries like the following:

2011-07-18 11:24:50 - safeInstallOperation: failed to back up file: C:\Documents and Settings\user\Application Data\Mozilla\Firefox\Profiles\gwbv7hm4.default\extensions\{3d7eb24f-2740-49df-8937-200b1cc08f8a}\chrome\flashblock.jar to: C:\Documents and Settings\user\Application Data\Mozilla\Firefox\Profiles\gwbv7hm4.default\extensions\{3d7eb24f-2740-49df-8937-200b1cc08f8a}-trash\chrome ... rolling back file moves and aborting installation.
2011-07-18 11:24:50 - ExtensionManager:_finishOperations - failure, catching exception - lineno: 1597 - file: undefined - [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIFile.moveTo]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: file:///C:/Program%20Files/Mozilla%20Firefox/components/nsExtensionManager.js :: moveFile :: line 1597"  data: no]

Comment 3

[Dave Townsend [:mossop]]

(In reply to Jorge Villalobos [:jorgev] from comment #2)
> A number of add-on developers have forwarded similar reports from their
> users. Only one has been able to give us an extensions.log, and it has
> multiple entries like the following:
> 
> 2011-07-18 11:24:50 - safeInstallOperation: failed to back up file:
> C:\Documents and Settings\user\Application
> Data\Mozilla\Firefox\Profiles\gwbv7hm4.default\extensions\{3d7eb24f-2740-
> 49df-8937-200b1cc08f8a}\chrome\flashblock.jar to: C:\Documents and
> Settings\user\Application
> Data\Mozilla\Firefox\Profiles\gwbv7hm4.default\extensions\{3d7eb24f-2740-
> 49df-8937-200b1cc08f8a}-trash\chrome ... rolling back file moves and
> aborting installation.
> 2011-07-18 11:24:50 - ExtensionManager:_finishOperations - failure, catching
> exception - lineno: 1597 - file: undefined - [Exception... "Component
> returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIFile.moveTo]" 
> nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame ::
> file:///C:/Program%20Files/Mozilla%20Firefox/components/nsExtensionManager.
> js :: moveFile :: line 1597"  data: no]

Those messages are back from July and I'm guessing they are unrelated to this problem which leaves us stuck with no way to reproduce and no suggestion of what might be happening.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Vlad or Virgil, could one of you please check if you can reproduce this issue?

Comment 5

[Jorge Villalobos [:jorgev] (he/him)]

There are some posts about this problem here: https://www.facebook.com/newtabking. You might be able to contact some of those users directly.

Comment 6

[Ike Ferber]

After booting the computer and starting Firefox, the add-ons manager disables  
Foxvox as incompatible with 7.0.1. I then do Check for updates and I get No 
updates found but now the Foxvox entry says "Foxvox will be enabled after you restart Firefox" I click Restart now and Firefox comes up immediately with Foxvox running. Hope this helps to solve the issue.

Comment 7

[Tony Mechelynck [:tonymec]]

(In reply to Ike Ferber from comment #6)
> After booting the computer and starting Firefox, the add-ons manager
> disables  
> Foxvox as incompatible with 7.0.1. I then do Check for updates and I get No 
> updates found but now the Foxvox entry says "Foxvox will be enabled after
> you restart Firefox" I click Restart now and Firefox comes up immediately
> with Foxvox running. Hope this helps to solve the issue.

IIRC, it is a known fact that if the update-check finds only a version-compatibility update with no change to the add-on code, it will (erroneously, or at least misleadingly) say "No updates found" while in fact it updates the version compatibility data and, for auto-disabled addons, if the new maxVersion is >= the current appVersion, stages the addon for enabling at next restart.

Comment 9

[Bridget Almas]

I am seeing this with my extension as well (https://addons.mozilla.org/en-US/firefox/addon/alpheios-basic-libraries/) and only on Ubuntu.  I am not seeing any extensions.log in my profile directory and with all the various debugging prefs on, there aren't any interesting errors in the error console.

I took a look at the contents of the extensions.sqlite database, and interestingly, the max version for my extension in the targetApplication table is set to 6.* and not 8.*, so that would seem to be why the browser keeps disabling it, but I don't know why it isn't being set properly to 8.* as it should be.

Comment 10

[Dave Townsend [:mossop]]

(In reply to Bridget Almas from comment #9)
> I am seeing this with my extension as well
> (https://addons.mozilla.org/en-US/firefox/addon/alpheios-basic-libraries/)
> and only on Ubuntu.  I am not seeing any extensions.log in my profile
> directory and with all the various debugging prefs on, there aren't any
> interesting errors in the error console.
> 
> I took a look at the contents of the extensions.sqlite database, and
> interestingly, the max version for my extension in the targetApplication
> table is set to 6.* and not 8.*, so that would seem to be why the browser
> keeps disabling it, but I don't know why it isn't being set properly to 8.*
> as it should be.

The maxVersion in your extension's install.rdf is 6.* so this would suggest Firefox is hitting problems syncing compatibility with AMO. If you can reproduce can you try the following:

1. Install Firefox 6.0
2. Enable extensions.logging.enabled in about:config
3. Install the extension and restart Firefox to enable it
4. Upgrade Firefox to 7.0 or 8.0
5. Check that the extension is disabled and if so copy any messages from the error console that appear extension related, they should all start "LOG addons" or "LOG addons.*".

Then try the same again, but this time between steps 3 and 4 do an update check for the extension and see if that changes the results

Comment 11

[Bridget Almas]

I'm not sure if this is helpful, but after posting the last comment, I tried manually updating the extensions.sqlite database, setting the compatibility to 8.* and flipping the active and and app disabled flags (to from 0 and 1 to 1 and 0 respectively).  This caused the add-ons menu to show the extension as enabled, but it wasn't actually.  I then uninstalled the extension, and reinstalled it.  The new entry in extensions.sqlite for the extension now had the max update version as 8.*, and it has retained it (and the extension remains enabled) after restarting.

I'm not sure if this confirms your theory or not. I'll see if I can try the steps you suggest as well.

Comment 12

[Bridget Almas]

Where can I find an installable Firefox 6.0 Linux build that won't automatically update itself? I downloaded one from http://www.mozilla.org/en-US/products/download.html?product=firefox-6.0.1&os=linux&lang=en-US but it updated itself before I had a chance to do anything.

Comment 13

[Jorge Villalobos [:jorgev] (he/him)]

You can disable automatic updates in the Preferences window, in the Advanced tab, under Updates.

Comment 14

[Bridget Almas]

Hmm. Okay, well, I did the following:

1) removed my profile directory
2) installed FF 6
3) installed my add-on
4) saw that the maxVersion in extensions.sqlite db was set to 6.*.
5) updated Firefox to 7.01.
I didn't get any errors about extension incompatibility, my extension remained enabled, and the maxVersion in the extensions.sqlite database was now correctly set to 8.

Comment 15

[Bridget Almas]

Also, as a follow-up, the first approach, of editing the sqlite database didn't hold up after a restart (i.e. after restarting the extension was disabled by the browse again), but the 2nd approach of downgrading Firefox, reinstalling, etc. did. I'm not sure if it's something in the profile directory (since I used a fresh profile directory for the 2nd approach) that's triggering the unwelcome behavior or not.

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Vlad, it would be great if we could figure out what's going on here once the 3rd party work has been finished.

Comment 17

[u279076]

Dave, how can we proceed with this given Bridget's recent comments? Should we be tracking this for Firefox 8?

Comment 18

[Dave Townsend [:mossop]]

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #17)
> Dave, how can we proceed with this given Bridget's recent comments? Should
> we be tracking this for Firefox 8?

So far there is not enough information to go on. We need to be able to reproduce this to narrow down what is causing it. There isn't any evidence that this started with Firefox 7 either.

Comment 19

[Bridget Almas]

I can reproduce it pretty reliably, so if there is anything more I can do to help narrow this down, please let me know.

Comment 20

[u279076]

Bridget, can you see if this issue reproduces for you on a build previous to Firefox 7?

Comment 21

[Dave Townsend [:mossop]]

Comment 14 suggested you couldn't reproduce, was I misunderstanding?

Comment 22

[Bridget Almas]

@Anthony: I don't think it was occurring before FF 7 but I will double check and confirm that.

@Dave, actually I realized today that it is still occurring, which makes me wonder if there is something going on when there are multiple extensions installed or maybe upon OS upgrades. In the initial test I did above, I had only the Alpheios extensions installed.  I have since reinstalled the other extensions I use regularly (Firebug, Live HTTP Headers, and Poster) and I also did an Ubuntu OS upgrade (to 11.10).  I can't be sure at what point it was reintroduced.  So I think need to go back and test a bit more methodically, adding additional extensions one at a time, etc.. I have a separate laptop I can do that on, so I'll get back to you when I've done that.

Comment 23

[Bridget Almas]

Ok, well, I think all of the above is irrelevant, at least for my extension, for which the problem seems to be related to the fact that my extension uses a binary. It is not limited to Ubuntu but IS limited to Firefox 7.  The following can only be reproduced with Firefox 7 and not earlier versions, but it can be reproduced reliably with Firefox 7 on Mac, Windows and Ubuntu: 

To reproduce:

Install Firefox 7.01
Install the Alpheios Basic Libraries extension
Install the Alpheios Greek Tools
Restart Firefox
See that the Alpheios Toolbar is there
Close and Restart FF, reboot the OS, etc. as many times as you want, and each time see that the Alpheios Toolbar is still there (meaning the extension hasn't been disabled).
Now actually interact with the extension:
On the Alpheios Toolbar, click the Alpheios menu and toggle on
On the Alpheios Toolbar, click the Alpheios menu and toggle off
Restart FF
Alpheios Basic Libraries have been disabled as incompatible with FF 7.01

I am not seeing any errors in the console, but I'm guessing this probably has something to do with the binary executable the tools launch.  There was some talk on the extension developers list serve about plans to disable extensions with binaries after FF upgrades if they hadn't been explicitly set to support the new FF release. A number of us (myself included) objected to this, but perhaps this policy was implemented in Firefox 7? My extension does actually work fine in Firefox 7 and I don't really see any cause for it to be disabled. I'm fairly certain the use of the binary is causing this because if instead of Alpheios Greek Tools you install and try to use the Alpheios Arabic tools, which DON'T make use of the binary from the Alpheios Basic Libraries, the Alpheios Basic Libraries don't get disabled. It's only when you launch the binary executable that they get disabled on reboot.

Comment 24

[Dave Townsend [:mossop]]

So this is actually not related to a Firefox upgrade?

Can you attach copies of extensions.sqlite from your profile before and after the problem occurs

Comment 25

[Bridget Almas]

Attachment: extensions.sqlite after problem occurs

Comment 26

[Bridget Almas]

Attachment: extensions.sqlite before problem occurs

Comment 27

[Bridget Almas]

It think it's related to a Firefox upgrade in so far as
- it didn't happen in Firefox 6 and it does in Firefox 7
- it is being affected by the part of the Firefox code that is trying to automatically determine whether or not allow the application (i.e.) Firefox to automatically enable an extension for a later release than is indicated in the extensions' install.rdf

I've attached the extensions.sqlite database from before and after the problem. You can see that before the problem, in the addon table the extension is enabled and in the targetApplication table, the maxVersion is 8.*. And before the problem, in the addon table the extension is flagged as 'app disabled' and the maxVersion in the targetApplication table is now 6.*, which is what is in the install.rdf.

Comment 28

[Bridget Almas]

sorry, typo in the above, I meant to say *after* the problem, the extension is now flagged as app disabled and max version set back to 6.*

Comment 29

[Dave Townsend [:mossop]]

What is happening is the add-ons manager is detecting some change to the add-on "Alpheios Basic Libraries". Specifically the database shows that the file modification times of the files for the add-on changed from "Fri Oct 21 2011 13:35:19 GMT-0700 (PDT)" to "Fri Oct 21 2011 13:35:45 GMT-0700 (PDT)". This change would cause the add-ons manager to assume the add-on had changed and so effectively reinstall it and since that happens during startup we don't have time to contact AMO for updated compatibility information so we only use the data in install.rdf to determine compatibility.

Is it possible that the binary in this extension is modifying files when you call it? That would certainly cause this though I'd expect it to happen all the way back to Firefox 4 in that case.

Comment 30

[Bridget Almas]

Yes, not the binary but the extension code itself does modify a configuration file for the binary when you enable it for the first time. This probably explains some of the difficulty I had in narrowing it down, because the problem may then only occur if the extension didn't modify this file before the upgrade to 7.  I'm pretty sure this didn't cause a problem prior to FF 7 though.

Comment 31

[Dave Townsend [:mossop]]

If there is a change here between Firefox 6 and 7 then it might be interesting to learn just what caused that, but in general we don't support extensions modifying their own files and that is what is causing the issue here

Comment 32

[Simon Kornblith]

Our binary extension (Zotero WinWord Integration) does not modify itself, but we've seen a couple reports of this as well:

http://forums.zotero.org/discussion/20315/
http://forums.zotero.org/discussion/20112/#Item_9

I can't reproduce this. It also seems like this affects our PythonExt-based extension is affected, but in this case the cause is probably that PythonExt creates *.pyo files on first run.

It would be really nice extensions didn't get disabled when the modification time changes, but failing that, it looks like the only (reliable) way to avoid

Comment 33

[Simon Kornblith]

this problem is to create a new build for new Firefox versions.

...apologies for hitting the submit button too quickly...

Comment 34

[Virgil Dicu [:virgil] [QA]]

WFM. Not able to reproduce the initial bug report, all addons are displayed as expected.
Wonder if this might not have to do with the AMO bug which incorrectly listed extensions versions as compatible.

As per New Tab King, by reading the reports on facebook, seems their STR would look something like this:
1. Start Firefox 6.0.1 
2. Install New tab King v5.0.9
3. Upgrade to 7.0.1

4. The extension got disabled as incompatible at a Firefox random restart

I can't install the extension in F7 on AMO because it is listed as incompatible with Firefox 7 (although listed differently on AMO), so I suspect this might be invalid from that point of view. Something might have changed for that version of the extension.

Comment 35

[Mike Kaply [:mkaply]]

We're getting random disabled reports on our add-on as well (Personas Interactive). We can find no rhyme or reason as to why it is getting disabled.

Comment 36

[Mihai Morar, (:MihaiMorar)]

I done a couple of investigations, following STR from Comment 0 and Comment 10:

1. I'd installed a version of Nightly 22.a1 (2013-04-01)
2. I'd installed a couple of addons from addons.mozilla.org, including New Tab King extension 5.1.14, having extensions.logging.enabled pref enabled and disabled too.
3. I'd restarted FF and Updated to latest version: 20130611081224

Actual results:
At step 3 after restarting FF (before update) all addons are displayed and enabled in Addons Manager. After FF got updated, they are all still displayed and enabled.

Expected results:
All addons must be displayed and have same status as before restart / update.

Based on my investigations mentioned above I can confirm this issue is Resolved.