702506 - Add-ons get treated as new third party installs if the database is corrupted or otherwise unreadable

Status: RESOLVED INACTIVE

(Toolkit :: Add-ons Manager, defect)

Description

[Frank Jiao]

Occasionally,due to some reasons such as extensions.sqlite has been remove or locked,all of addons will be disabled and report by about:newaddon when starting Firefox.

Comment 1

[Frank Jiao]

Attachment: the screenshot pops up to the user.

We got feedback on installed addons are disabled when user start
Firefox 8.0.
After my investigation, one of situation could be extensions.sqlite is
locked or been removed, all addons will be disabled and reported by
about:newaddon when starting firefox.
Please see the attachment for the screenshot pops up to the user.

Comment 2

[Dave Townsend [:mossop]]

Essentially caused by bug 702265 though we may want to do something special for the profile folder

Comment 3

[Hong Tang]

More information:
On one of our user's machine,somehow the file extensions.sqlite always got deleted after reboot the system. Then he starts firefox and got the popups (as attached) for all of his addons.

Comment 4

[Dave Townsend [:mossop]]

This is likely to be fixed by bug 699175

Comment 6

[public]

(In reply to Hong Tang from comment #3)
> More information:
> On one of our user's machine,somehow the file extensions.sqlite always got
> deleted after reboot the system. Then he starts firefox and got the popups
> (as attached) for all of his addons.

I have this problem on all of my PCs. In my case no reboot is required, closing the Windows session and logging on again already triggers this issue.
I've filed this as 
https://bugzilla.mozilla.org/show_bug.cgi?id=704823
I'll mark it as a duplicate to this one.

Comment 7

[Dave Townsend [:mossop]]

If the people seeing this problem are able to test on the latest nightly builds then that would be really useful.

Comment 8

[public]

> If the people seeing this problem are able to test on the latest nightly

Using Nightly 11.0a1 (2011-11-24) the problem disappears for me.

Comment 9

[Frank Jiao]

(In reply to public from comment #8)
> > If the people seeing this problem are able to test on the latest nightly
> 
> Using Nightly 11.0a1 (2011-11-24) the problem disappears for me.

But, problems still happen to me in here.

(In reply to Hong Tang from comment #3)
> More information:
> On one of our user's machine,somehow the file extensions.sqlite always got
> deleted after reboot the system. Then he starts firefox and got the popups
> (as attached) for all of his addons.

Steps to reproduce
  Shutdown firefox
  Remove [profile]/extensions.sqlite
  Start firefox
  ALL OF ADDONS will be disabled

Comment 10

[public]

(In reply to Frank Jiao from comment #9)
> (In reply to public from comment #8)
> > > If the people seeing this problem are able to test on the latest nightly
> > 
> > Using Nightly 11.0a1 (2011-11-24) the problem disappears for me.
> 
> But, problems still happen to me in here.
> 
> (In reply to Hong Tang from comment #3)
> > More information:
> > On one of our user's machine,somehow the file extensions.sqlite always got
> > deleted after reboot the system. Then he starts firefox and got the popups
> > (as attached) for all of his addons.
> 
> Steps to reproduce
>   Shutdown firefox
>   Remove [profile]/extensions.sqlite
>   Start firefox
>   ALL OF ADDONS will be disabled

But this seems to be the desired behaviour. Like mentioned in the Release Notes as a new feature Firefox 8 will disabled all "third-party" add-ons by default. (And if the files are damaged or lost they are recreated).

For me the key question is: Why extensions.ini and extensions.sqlite are deleted when I reboot my system?

The more often I try to reproduce the behaviour the more weird it gets: At the beginning I could reproduce it with FF 8.0.1, 9.0b2 and 10.0a2 several times (with slightly different results, e.g. 9.0b2 does not open an extra tab for each add-on, it justs silently disables them). Later on I could not reproduce it anymore with 10.0a2.
At the beginnig it was sufficient to close my session and reopen it to trigger the problem, now I have to reboot (both FF 8.0.1).
I moved my profile to a VM, installed the same global extensions, and I'm not able to reproduce the issue there.
Actually, I can't spend more time on this. If it's fixed in the latest nightly, I hope the fix will find its way to the final release of FF 9.

Comment 11

[Dave Townsend [:mossop]]

(In reply to public from comment #10)
> (In reply to Frank Jiao from comment #9)
> > (In reply to public from comment #8)
> > > > If the people seeing this problem are able to test on the latest nightly
> > > 
> > > Using Nightly 11.0a1 (2011-11-24) the problem disappears for me.
> > 
> > But, problems still happen to me in here.
> > 
> > (In reply to Hong Tang from comment #3)
> > > More information:
> > > On one of our user's machine,somehow the file extensions.sqlite always got
> > > deleted after reboot the system. Then he starts firefox and got the popups
> > > (as attached) for all of his addons.
> > 
> > Steps to reproduce
> >   Shutdown firefox
> >   Remove [profile]/extensions.sqlite
> >   Start firefox
> >   ALL OF ADDONS will be disabled
> 
> But this seems to be the desired behaviour. Like mentioned in the Release
> Notes as a new feature Firefox 8 will disabled all "third-party" add-ons by
> default. (And if the files are damaged or lost they are recreated).

If they are truly third party. Stuff in the profile may or may not be third party, the problem is when extensions.sqlite and extensions.ini are gone we have no way to know which is which. We should probably assume profile add-ons are not third party when we can't be sure as that is the route that has the least user impact.

> For me the key question is: Why extensions.ini and extensions.sqlite are
> deleted when I reboot my system?

That's not going to be soething Firefox does (Firefox doesn't do anything at reboot time). If that is happening I'd look at other software on your machine, I know some AV tools did things like this at one point.

Comment 12

[public]

> If they are truly third party. Stuff in the profile may or may not be third
> party, the problem is when extensions.sqlite and extensions.ini are gone we
> have no way to know which is which. We should probably assume profile
> add-ons are not third party when we can't be sure as that is the route that
> has the least user impact.

Just to explain: In my case three types of extensions are present. 1. In the users profile (e.g. Zotero), 2. In the Firefox program directory (e.g. Oracle's Java Console, 3. via Windows registry (e.g. BitDefender toolbar). All three are treated as third-party.

[The same behaviour I observe with Thunderbird 8. I've placed some dictionaries from the official add-on site in "%ProgramFiles%\Mozilla Thunderbird\extensions". For Thunderbird they are third-party extensions. But Thunderbird remembers my settings and asks me only once.]

> 
> > For me the key question is: Why extensions.ini and extensions.sqlite are
> > deleted when I reboot my system?
> 
> That's not going to be soething Firefox does (Firefox doesn't do anything at
> reboot time). If that is happening I'd look at other software on your
> machine, I know some AV tools did things like this at one point.

It is strange indeed, but it must be related to Firefox. It happens with Firefox 8,9 and 10, did not happen with Firefox 5 and 7 and the effect disappears with the latest nightly. In fact there's no Firefox process running when the files are deleted, but there are ways to tell Windows to remove files on next reboot. It must be something like this.

Most likely I can't blame it on the AV program, its logfile does not show any related entry and the effect is reproducible even when the on-access AV scan is desactivated.

Comment 13

[Dave Townsend [:mossop]]

(In reply to public from comment #12)
> > If they are truly third party. Stuff in the profile may or may not be third
> > party, the problem is when extensions.sqlite and extensions.ini are gone we
> > have no way to know which is which. We should probably assume profile
> > add-ons are not third party when we can't be sure as that is the route that
> > has the least user impact.
> 
> Just to explain: In my case three types of extensions are present. 1. In the
> users profile (e.g. Zotero), 2. In the Firefox program directory (e.g.
> Oracle's Java Console, 3. via Windows registry (e.g. BitDefender toolbar).
> All three are treated as third-party.
> 
> [The same behaviour I observe with Thunderbird 8. I've placed some
> dictionaries from the official add-on site in "%ProgramFiles%\Mozilla
> Thunderbird\extensions". For Thunderbird they are third-party extensions.
> But Thunderbird remembers my settings and asks me only once.]
> 
> > 
> > > For me the key question is: Why extensions.ini and extensions.sqlite are
> > > deleted when I reboot my system?
> > 
> > That's not going to be soething Firefox does (Firefox doesn't do anything at
> > reboot time). If that is happening I'd look at other software on your
> > machine, I know some AV tools did things like this at one point.
> 
> It is strange indeed, but it must be related to Firefox. It happens with
> Firefox 8,9 and 10, did not happen with Firefox 5 and 7 and the effect
> disappears with the latest nightly. In fact there's no Firefox process
> running when the files are deleted, but there are ways to tell Windows to
> remove files on next reboot. It must be something like this.

You mean that you see the third party prompts on startup with 8, 9 and 10? Previous versions didn't have that so you wouldn't have seen anything. Here is a question. If you disable an add-on with Firefox 7, then restart your computer and start Firefox 7 again does the add-on remain disabled? If not that would be a sure sign those fixes got deleted again.

Firefox never schedules profile files for deletion on shutdown.

> Most likely I can't blame it on the AV program, its logfile does not show
> any related entry and the effect is reproducible even when the on-access AV
> scan is desactivated.

That wouldn't necessarily rule the AV tool out, you'd have to completely uninstall it to test for sure.

Comment 14

[Dapeng Wei]

Hi, one of our users has the same problem. He uses Windows 7. He found one error in window boot log.

name: Application, Source: Microsoft-Windows-WMI

He found the solution for this error from MS. There is one repair tool MicrosoftFixit50688.msi. This problem has gone after installing this patch.

Anyone who can reproduce this problem, please try it.

Comment 15

[public]

(In reply to Dave Townsend (:Mossop) from comment #13)
> 
> You mean that you see the third party prompts on startup with 8, 9 and 10?
> Previous versions didn't have that so you wouldn't have seen anything. Here
> is a question. If you disable an add-on with Firefox 7, then restart your
> computer and start Firefox 7 again does the add-on remain disabled? If not
> that would be a sure sign those fixes got deleted again.
> 
> Firefox never schedules profile files for deletion on shutdown.
> 

You're absolutely right. I've just tested this with Firefox 7 and when I rebooted the machine extensions.ini and extensions.sqlite were deleted from my profile folder. Since all extensions were automatically enabled in earlier Firefox versions I've never noticed the problem before, but it was always there.

Finally I can't blame it on Firefox and have to figure out which process is deleting these two files.

Comment 16

[Dapeng Wei]

(In reply to Dapeng Wei from comment #14)
> Hi, one of our users has the same problem. He uses Windows 7. He found one
> error in window boot log.
> 
> name: Application, Source: Microsoft-Windows-WMI
> 
> He found the solution for this error from MS. There is one repair tool
> MicrosoftFixit50688.msi. This problem has gone after installing this patch.
> 
> Anyone who can reproduce this problem, please try it.

This user replied to us that he changed those two files' permissions to read only when he tried to solved this problem. So this patch may be not the right solution.

Another user told us his finding. He had bitdefender 2010 installed in his machine. Everytime he booted his machine, two files were deleted. Then he updated to bitdefender 2011 and the problem is fixed.

Comment 17

[public]

(In reply to Frank Jiao from comment #1)
> Created attachment 574518 [details]
> the screenshot pops up to the user.
> 
> We got feedback on installed addons are disabled when user start
> Firefox 8.0.
> After my investigation, one of situation could be extensions.sqlite is
> locked or been removed, all addons will be disabled and reported by
> about:newaddon when starting firefox.
> Please see the attachment for the screenshot pops up to the user.

There's a workaround available: Create a file "mysettings.js" in "%ProgramFiles%\Mozilla Firefox\defaults\pref" (for Windows) with the follwing content:

pref("extensions.shownSelectionUI", true);
pref("extensions.autoDisableScopes", 0);

This should restore the old behaviour for all local users. See here for details:
http://forums.mozillazine.org/viewtopic.php?f=23&t=2320715&start=15

HTH

Comment 18

[public]

(In reply to Dapeng Wei from comment #16)
> (In reply to Dapeng Wei from comment #14)

> 
> Another user told us his finding. He had bitdefender 2010 installed in his
> machine. Everytime he booted his machine, two files were deleted. Then he
> updated to bitdefender 2011 and the problem is fixed.

I can confirm this: BitDefender Antivirus 2010 is deleting the two files on every reboot (without logging the action and even if the files are on the exclusion list and the on-access module is desactivated). I can reproduce this on several PCs.

Upgrading to BitDefender Antivirus 2011 solves the problem.

Comment 19

[Dapeng Wei]

Yes, we did the same thing here.
Thanks for the confirmation.

Comment 20

[Gregory Szorc [:gps]]

I just experienced something like this when testing add-on sync in a fresh profile. STR:

1) Create fresh profile
2) Set up Sync against existing profile that has add-ons
3) Wait for initial sync run to complete. See entries in about:addons
4) Restart browser
5) On restart witness new tabs opened by add-on install, new chrome in browser
6) Go to about:addons and see the "You don't have of this type installed" message
7) Restart browser a few times
8) Get a bunch of about:newaddon tabs

I've had it reproduce on 2 of 3 fresh profiles today when testing add-on sync. This is on OS X 10.7 running a Nightly built from source pulled in the last 24 hours.

Comment 21

[Gregory Szorc [:gps]]

I encountered this issue in 2/3 fresh profiles when testing bug 712542. There may be helpful comments in that bug starting at comment #12.

In related news, I fear this bug has the potential to result in disabling add-on sync because it could result in add-on state getting corrupted and propagated to other devices.

Comment 22

[Gregory Szorc [:gps]]

Just saw this on my local build of services-central on OS X (last pulled from m-c on 2012-02-15) when installing add-ons through about:addons and https://addons.mozilla.org/. Sync was configured, but there were no add-on installs performed through Sync: I was starting with an empty extensions database.

Comment 23

[Richard Newman [:rnewman]]

In the absence of another bug, I'm flagging this as blocking Bug 712542. gps, feel free to rearrange.

Comment 24

[Dave Townsend [:mossop]]

(In reply to Gregory Szorc [:gps] from comment #20)
> I just experienced something like this when testing add-on sync in a fresh
> profile. STR:
> 
> 1) Create fresh profile
> 2) Set up Sync against existing profile that has add-ons
> 3) Wait for initial sync run to complete. See entries in about:addons
> 4) Restart browser
> 5) On restart witness new tabs opened by add-on install, new chrome in
> browser
> 6) Go to about:addons and see the "You don't have of this type installed"
> message
> 7) Restart browser a few times
> 8) Get a bunch of about:newaddon tabs
> 
> I've had it reproduce on 2 of 3 fresh profiles today when testing add-on
> sync. This is on OS X 10.7 running a Nightly built from source pulled in the
> last 24 hours.

Can you enable extensions.logging.enabled and after step 4 copy the messages from the error console relating to addons. There may be a lot, they also come through stdout.

Comment 25

[Gregory Szorc [:gps]]

Attachment: Console log showing errors

The console log on application start after add-on sync initial run captures an error and shows database recreation.

Comment 26

[Dave Townsend [:mossop]]

(In reply to Gregory Szorc [:gps] from comment #25)
> Created attachment 598402 [details]
> Console log showing errors
> 
> The console log on application start after add-on sync initial run captures
> an error and shows database recreation.

Thanks this is very useful. Could you test again without Add-on Compatibility Reporter included at all, that triggers a restart in the middle of startup that has been known to cause this problem in the past.

Comment 27

[Dave Townsend [:mossop]]

(In reply to Richard Newman [:rnewman] from comment #23)
> In the absence of another bug, I'm flagging this as blocking Bug 712542.
> gps, feel free to rearrange.

Actually technically it's bug 671894 that is the real problem here, this bug is about being saner about the fallout from a corrupt DB.

Comment 28

[Gregory Szorc [:gps]]

I uninstalled Add-on Compatibility Reporter and reset my Sync data. I tried with 2 new profiles and the issue did *not* appear. The following is the log on successful application start.

*** LOG addons.xpi: startup
*** LOG addons.xpi: Skipping unavailable install location app-system-share
*** LOG addons.xpi: checkForChanges
*** LOG addons.xpi: Found updated metadata for [e-mail] in app-profile
*** LOG addons.xpi: Processing install of [e-mail] in app-profile
*** LOG addons.xpi: Found updated metadata for {d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d} in app-profile
*** LOG addons.xpi: Processing install of {d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d} in app-profile
*** LOG addons.xpi: Found updated metadata for {e4a8a97b-f2ed-450b-b12d-ee082ba24781} in app-profile
*** LOG addons.xpi: Processing install of {e4a8a97b-f2ed-450b-b12d-ee082ba24781} in app-profile
*** LOG addons.xpi: Opening database
*** LOG addons.xpi: New add-on [e-mail] installed in app-profile
*** LOG addons.xpi: New add-on {d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d} installed in app-profile
*** LOG addons.xpi: New add-on {e4a8a97b-f2ed-450b-b12d-ee082ba24781} installed in app-profile
*** LOG addons.xpi: Updating database with changes to installed add-ons
*** LOG addons.xpi: Updating add-on states
*** LOG addons.xpi: Writing add-ons list

Comment 29

[Dave Townsend [:mossop]]

Thanks, I filed bug 728466 to track the issue with ACR.

Comment 30

[BMO Automation]

Per policy at https://wiki.mozilla.org/Bug_Triage/Projects/Bug_Handling/Bug_Husbandry#Inactive_Bugs. If this bug is not an enhancement request or a bug not present in a supported release of Firefox, then it may be reopened.