633189 - Disable compatibility check for add-ons installed via Mozmill CLI

Status: VERIFIED FIXED

(Testing Graveyard :: Mozmill, defect)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b12pre) Gecko/20110210 Firefox/4.0b12pre

After dropping an extension into a fresh profile, Minefield is not running the compatibility check on next launch.

Steps to reproduce:
1. Create a fresh profile
2. Close Firefox
3. Drop an incompatible extension into the profile's extensions directory. I used Adblock Plus 1.3.3, which has a maxVersion of 4.0b9pre
4. Rename the XPI to the add-ons ID. For Adblock Plus this is {d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}.xpi
5. Launch Firefox

Expected:
A compatibility check should be run and the add-on enabled.

Actual:
No compatibility check is run. On opening the Add-ons Manager the add-on is disabled with a message that it is not compatible with the current version of Firefox.

Note: Manually triggering an update in Add-ons Manager works as expected.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Dave, this is a major problem for our Mozmill Add-ons tests. None of them will work when the maxVersion is lower as the current application version. Why don't we perform compatibility checks for new add-ons which have been dropped to the extension folder? We are registering those correctly, so nothing is broken with the installation itself, but the missing compat check leaves us in a bad state.

Comment 2

[Dave Townsend [:mossop] (PTO until Aug 21st)]

You close Firefox after creating the fresh profile? Doesn't that mean the profile isn't fresh anymore?

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Dave, the same happens when you create a new profile, immediately close the profile manager, and copy the xpi into the extensions folder. That's the way how Mozmill drops the extensions into a fresh profile. Also unpacking the extension doesn't trigger a compatibility check during the first start.

Comment 4

[Dave Townsend [:mossop] (PTO until Aug 21st)]

Don't think we can block on this, not a normal use case and I presume we can work around it by just updating the mozmill add-on's maxVersion

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Dave, this is not about the maxVersion of Mozmill but for add-ons we want to install and run tests for. We can't modify their install.rdf as we want. In the case above it's for Adblock which has a maxVersion of 4.0b9pre in its install.rdf.

Without that fix we will not being able to support add-ons tests for those which only have been marked as compatible on AMO but no new version released.

Comment 6

[Dave Townsend [:mossop] (PTO until Aug 21st)]

Ok we don't show the compatibility UI for fresh profiles because the whole point of the UI was for upgrade scenarios. I don't think that this is new, we didn't show it in this case in 3.6 either so I don't think this is something we should even fix.

The simple workaround for mozmill is to add the following line to a prefs.js in the profile before starting:

user_pref("extensions.lastAppVersion", "1.0");

This will force Firefox to believe it is an upgrade and so do the full compatibility checking.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

So that would raise another problem. In Mozmill we have disabled the compatibility update mismatch UI, which now still blocks us from getting compatibility updates from AMO. Only when I keep 'extensions.showMismatchUI' as true, we apply the compatibility update:

-                   'extensions.showMismatchUI' : False,
+                   #'extensions.showMismatchUI' : False,
+                   # Force
+                   'extensions.lastAppVersion' : '1.0',

With showMismatchUI enabled, we would run into problems for update tests, because we can't automate this dialog. One solution could be that the update tests explicitly set this preference to false, right before Firefox gets restarted.

Dave, do you have any other ideas?

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Clint, this issue will be a major pain for us. So I request to block Mozmill 1.5.2.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Dave, even with 'extensions.lastAppVersion' set, no compatibility check is performed on start-up. Is it because we have showMismatchUI disabled? It would be very important for us to know, what's the right way for us to continue with that reported issue with Mozmill. Mozmill 1.5.2 is going to be released soon, so I would really like to see a fix here.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

That's indeed what I'm seeing here, re-enabling 'extensions.showMismatchUI' checks for compatibility updates. Why isn't the front-end separated from the backend? Is disabling the compatibility check the only solution for us?

Comment 11

[Dave Townsend [:mossop] (PTO until Aug 21st)]

We can't do compatibility checks unless we show UI since we have to block startup to do it.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to comment #11)
> We can't do compatibility checks unless we show UI since we have to block
> startup to do it.

Does that mean we would have to completely disable the compatibility check then?

Comment 13

[Dave Townsend [:mossop] (PTO until Aug 21st)]

(In reply to comment #12)
> (In reply to comment #11)
> > We can't do compatibility checks unless we show UI since we have to block
> > startup to do it.
> 
> Does that mean we would have to completely disable the compatibility check
> then?

If you don't want the compatibility check UI to display then you can't have the compatibility check.

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Well, this all is kinda mess. There is really no way for us to force only compatibility updates. I can't find the right set of preferences to set to have the installed extensions available after the first start.

With "extensions.checkCompatibility" set to false we disable compatibility checks. But installing the add-on by storing the XPI file into the profile, will mark the add-on as disabled. We could turn on "extensions.showMismatchUI" if there would be a way to only get compatibility updates downloaded. But if a new version of the add-on is available we totally break our tests because Mozmill cannot handle the dialog.

So it looks like that the only solution here is to workaround this problem by modifying the maxVersion of the extension, so it fits to the current version of the application. I'm not that happy about, but it looks like there is no other way.

Comment 15

[Dave Townsend [:mossop] (PTO until Aug 21st)]

(In reply to comment #14)
> Well, this all is kinda mess. There is really no way for us to force only
> compatibility updates. I can't find the right set of preferences to set to have
> the installed extensions available after the first start.
> 
> With "extensions.checkCompatibility" set to false we disable compatibility
> checks. But installing the add-on by storing the XPI file into the profile,
> will mark the add-on as disabled.

If the compatibility checking is off then why is the extension ending up disabled? It shouldn't be.

> We could turn on "extensions.showMismatchUI"
> if there would be a way to only get compatibility updates downloaded. But if a
> new version of the add-on is available we totally break our tests because
> Mozmill cannot handle the dialog.

What about setting a custom extensions.update.url that points to a fixed file that carries whatever compatibility info you want for the installed extensions and doesn't list any updates?

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: test profile

(In reply to comment #15)
> > With "extensions.checkCompatibility" set to false we disable compatibility
> > checks. But installing the add-on by storing the XPI file into the profile,
> > will mark the add-on as disabled.
> 
> If the compatibility checking is off then why is the extension ending up
> disabled? It shouldn't be.

But it is. See the test profile. It has a user.js with the preference to disable compatibility checks, and adblock plus 1.3.3 extracted into the extensions folder. Starting this profile with a recent Firefox or Minefield build, shows the extension as disabled.

> What about setting a custom extensions.update.url that points to a fixed file
> that carries whatever compatibility info you want for the installed extensions
> and doesn't list any updates?

I would be more in favor to disable compatibility checks per default. The reason is that we had a lot of test failures when the version got bumped up and AMO hasn't been updated the appversion page to include the new versions. Also this URL would have to be a HTTP or HTTPS address, right? That would block us to run the tests offline.

Comment 17

[Dave Townsend [:mossop] (PTO until Aug 21st)]

(In reply to comment #16)
> Created attachment 512923 [details]
> test profile
> 
> (In reply to comment #15)
> > > With "extensions.checkCompatibility" set to false we disable compatibility
> > > checks. But installing the add-on by storing the XPI file into the profile,
> > > will mark the add-on as disabled.
> > 
> > If the compatibility checking is off then why is the extension ending up
> > disabled? It shouldn't be.
> 
> But it is. See the test profile. It has a user.js with the preference to
> disable compatibility checks, and adblock plus 1.3.3 extracted into the
> extensions folder. Starting this profile with a recent Firefox or Minefield
> build, shows the extension as disabled.

You have the pref wrong, you want extensions.checkCompatibility.4.0b or extensions.checkCompatibility.4.0 for release versions

Comment 18

[Dave Townsend [:mossop] (PTO until Aug 21st)]

(In reply to comment #16)
> > What about setting a custom extensions.update.url that points to a fixed file
> > that carries whatever compatibility info you want for the installed extensions
> > and doesn't list any updates?
> 
> I would be more in favor to disable compatibility checks per default. The
> reason is that we had a lot of test failures when the version got bumped up and
> AMO hasn't been updated the appversion page to include the new versions. Also
> this URL would have to be a HTTP or HTTPS address, right? That would block us
> to run the tests offline.

I think that you can set the extensions.update.url pref to any URL you like including local file but I'm not positive.

Comment 19

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Patch v1

Ok, that would make sense. Totally forgot about that part. So as long as we do not have a chance to safely create such a list on our own, I would propose we add those three prefs to our list of default prefs for a profile.

In general it will give us the availability to start Mozmill test-runs with add-ons installed. That's kinda important for our endurance tests. Tests which need the compatibility checks enabled like the update tests, can reset those prefs very easily in setupModule.

For Mozmill 2.0 we will definitely have to use a different way, as discussed already multiple times.

Comment 20

[cmtalbert]

Comment on attachment 513470 [details] [diff] [review]
Patch v1

THis is great for 1.5.2, and as Henrik noted, we definitely want to handle this better for 2.0 because as we switch to quarterly releases, this is just not gonna scale.

Comment 21

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Landed on hotfix-1.5.2:
https://github.com/mozautomation/mozmill/commit/05da4aac4dced5efb3dd281149d013a56c4927d6

Trunk patch coming up right away.

Comment 22

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Patch (master)

Patch for master. We need that for testing and waiting for any refactoring work would block us doing so.

Comment 23

[Jeff Hammel]

I thought we were going to handle master a better way (comment 20) and in fact move away from default prefs in general and handle prefs entirely differently???

Comment 24

[cmtalbert]

Comment on attachment 514280 [details] [diff] [review]
Patch (master)

Sure, then let's ensure we have a follow-on bug filed to solve this the right way for master.

Comment 25

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Landed on master:
https://github.com/mozautomation/mozmill/commit/e4a8607ebc2e9780dacab1d96d2937f77e4f246e

I'm sure we already have a bug for refactoring the handling of prefs for Mozmill 2.0. I will query and file a new one if I misremember.

Comment 26

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to comment #24)
> Sure, then let's ensure we have a follow-on bug filed to solve this the right
> way for master.

Haven't found any existing bug. So I have filed bug 636035.

Comment 27

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As given by Dave this now works with add-ons which are incompatible on trunk.