563006 - Migration of addons takes time

Status: VERIFIED FIXED

(Toolkit :: Add-ons Manager, defect, P2)

Description

[Alice0775 White]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a5pre) Gecko/20100430 Minefield/3.7a5pre ID:20100430072714
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a5pre) Gecko/20100430 Minefield/3.7a5pre ID:20100430072714

Migration of addons takes time.
it takes about 10min/40addons

Reproducible: Always

Steps to Reproduce:
1. Start Minefield with 40addons(they should be working well.), Say profile "A".
2. Quit Minefield.
3. Start Minefield and Create New Profile, Say profile "B".
4. Quit all Minefield
5. Copy extensions folder in the profile "A" to Profile "B".
6. Delete extemsions.ini, extemsions.log, extemsions.sqlite in Profile "B".
7. Start Minefield with Profile "B".

Actual Results:
 Migration of addons takes time.
 it takes about 10min/40addons.
 However, CPU(C2Q@2.5GHz)  usage is 1-4% during migration.

Expected Results:
 Should be less than 1 min. as before landing Bug 550048.

Comment 1

[Alice0775 White]

And several addons are missing after migration.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 02/17 - 02/21)]

This is bad! Can you please upload all extension.* files Alice? We should check what's going on here.

Comment 3

[Alice0775 White]

Here, my extensions Folder(9MB)
http://www.saturn.dti.ne.jp/alice0775/chrome/xul/extensions.zip

Comment 4

[Alice0775 White]

In addition to the comment #3
You need to set preference as follows.
user_pref("extensions.checkCompatibility.3.7a", false);

Comment 5

[Blair McBride [:Unfocused] (UNAVAILABLE)]

Alice: Would you mind also uploading your extensions.sqlite, extensions.rdf, and extensions.ini files? It should be small enough to just attach to this bug - thanks.

Comment 6

[Alice0775 White]

Attachment: extensions.ini, rdf, etc.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 02/17 - 02/21)]

Missing extensions is already bug 562930. Lets keep this bug on the perf issue.

Comment 8

[Brian King [:kinger]]

Took about 2 minutes for me, with 52 extensions and 12 themes. Unfortunately I did not measure the exact time. I looked at system performance at that time and CPU usage was also quite low for me during migration.

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.3a5pre) Gecko/20100511 Ant.com Toolbar 1.4 Minefield/3.7a5pre (.NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729)

When it finally did start, a bunch of disabled extensions has been enabled. But I will file another bug on that.

Comment 9

[Robert Strong (they/them - no direct email)]

If you tried out the new add-ons manager (for example, in a previous nightly)and then deleted extensions.sqlite file the disabled state won't be migrated a second time.

Dave and I tracked down what is causing the long time to launch along with a likely fix today. btw: the long time to launch will also happen when migration doesn't occur such as just deleting the extensions.sqlite file.

Comment 10

[Brian King [:kinger]]

(In reply to comment #9)
> If you tried out the new add-ons manager (for example, in a previous
> nightly)and then deleted extensions.sqlite file the disabled state won't be
> migrated a second time.

I don't recall manually deleting it, but it is quite possible it happened. Is it even worth filing the bug then?

Comment 11

[Robert Strong (they/them - no direct email)]

You can try to reproduce by first adding a new boolean preference in about:config with the name extensions.logging.enabled and a value of true. Then exit Firefox, opening the prefs.js file in your profile, and delete the lines that contain the following:
extensions.bootstrappedAddons
extensions.databaseSchema
extensions.enabledAddons
extensions.installCache
extensions.pendingOperations

Then delete the extensions.sqlite file from the profile.

Launch a version of Firefox without the new add-ons manager and make a note of which extensions are disabled.

Launch a version of Firefox with the new add-ons manager.

Check if any of the add-ons that were disabled are no longer disabled.

If there were any check the error console for any messages related to add-ons that might look relevant.

I did this a few times yesterday trying to track down the cause of this bug and each time the previously disabled add-ons were still disabled after the migration. The couple of times when I forgot remove the preferences migration didn't occur and all add-ons were enabled.

Comment 12

[Johnny Stenback (:jst)]

Any updates on this bug? This is blocking the next alpha (which we'd like to do next week:ish). Who should own this?

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 02/17 - 02/21)]

(In reply to comment #12)
> Any updates on this bug? This is blocking the next alpha (which we'd like to do
> next week:ish). Who should own this?

We are waiting for bug 562756 which already has a patch and has to pass the review cycle. Hopefully this bug will be solved with it's checkin.

Comment 14

[Brian King [:kinger]]

(In reply to comment #11)
> You can try to reproduce by first adding a new boolean preference in
> about:config with the name extensions.logging.enabled and a value of true. 

<snip>

> I did this a few times yesterday trying to track down the cause of this bug and
> each time the previously disabled add-ons were still disabled after the
> migration. The couple of times when I forgot remove the preferences migration
> didn't occur and all add-ons were enabled.

I just (rather belatedly) ran this test, and despite the delay, everything works as expected. All disabled add-ons remained disabled. I suspect what happened before is that extensions data got corrupted because I killed the Firefox process during startup because I thought it hanged while what it was actually doing was taking a long time to convert (this bug).

Comment 15

[Dave Townsend [:mossop]]

I'm the owner here, but as said this should be fixed when we get bug 562756 landed so the main activity is going on there.

Comment 16

[Dave Townsend [:mossop]]

Should be fixed by bug 562756. If we don't already have a litmus test testing installing a new Firefox with a Firefox 3.6 profile then we should have one and it should verify that it starts up in a reasonable amount of time.

Comment 17

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 02/17 - 02/21)]

Dave, I would suggest to prepare a profile with extensions which have been installed with the older add-ons manager. We can store this profile as litmus data and would also be able to hopefully re-use it in a Mozmill test without requiring to install an older build first. We simply would have to collect a good selection of extensions we can use for that test.

Comment 18

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 02/17 - 02/21)]

Alice and Brian, could you both please test again, that the performance problems have been fixed? I can't see any noticeable lag when using a Minefield build the first time and having about 20 extensions installed.

Comment 19

[Alice0775 White]

I confirmed that the problem is fixed,Thanks.
Mozilla/5.0 (Windows; U; Windows NT 6.1; WOW64; en-US; rv:1.9.3a5pre) Gecko/20100530 Minefield/3.7a5pre ID:20100530040319

Comment 20

[Dave Townsend [:mossop]]

(In reply to comment #17)
> Dave, I would suggest to prepare a profile with extensions which have been
> installed with the older add-ons manager. We can store this profile as litmus
> data and would also be able to hopefully re-use it in a Mozmill test without
> requiring to install an older build first. We simply would have to collect a
> good selection of extensions we can use for that test.

That should work ok in this case. It probably makes sense to choose the top 20 add-ons from AMO or something.

Comment 21

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 02/17 - 02/21)]

Verified fixed with Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b12pre) Gecko/20110209 Firefox/4.0b12pre ID:20110209030359

While running tests for this bug report, I have also found bug 633339, and bug 633382.