539405 - Resolve how to accommodate mozmill extensions not marked as compatible with existing build versions when running mozmill tests

Status: RESOLVED WORKSFORME

(Thunderbird :: Testing Infrastructure, defect)

Description

[Mark Banner (:standard8)]

I just realised that the reason our trunk mozmill test are failing is because the mozmill 1.2 extension isn't compatible with 3.2a1pre (although for some reason, jsbridge is).

Even if we keep mozmill up to date, we're always likely to hit the case where we bump the a version number and mozmill is no longer seen as compatible.

I think the easiest way of doing this may be to tweak "extensions.checkCompatibility" in the profile that we're setting up. Though I'm not convinced yet, and ideas welcome.

Comment 1

[Mark Banner (:standard8)]

For now, I've checked in a 'bustage' fix:

http://hg.mozilla.org/comm-central/rev/8ece14c7547b

Which is:

+        'extensions.checkCompatibility.3.2a': False,

i.e. adding a disabling of the compatibility check to the profile we use to start mozmill. Obviously we want a more permanent fix, which this bug will deal with.

Comment 2

[Dan Mosedale (:dmosedale, :dmose)]

I bet this is likely to effect more than just Thunderbird.  Adding ctalbert to the CC to see if he has any thoughts...

Comment 3

[cmtalbert]

Yeah, the reason jsbridge was compatible is that jsbridge doesn't have to be listed on AMO.  Since mozmill has to be listed on AMO, it couldn't be compatible until AMO approved 3.2a1pre as a version which happened last night as near as I can tell.  

This is going to be an going problem.  Perhaps we can solve it by having different compatibility settings on the PyPi instance of Mozmill (more broad settings there) than we do on the AMO release (there we can use the maximum of what AMO allows).

But, there is still a chance that even that way things can get out of date.  So, setting checkCompatibility to false in your automation is probably the best way to go for automation.

For daily use, I can make the above change to our PyPi build, but I am pretty hamstrung in what I can do on AMO.  Though I can push a compat change for 3.2a1pre and 3.7 up there today, if that will help.

Comment 4

[Mark Banner (:standard8)]

I think we don't need this bug any longer. We set extensions.checkCompatibility.nightly to false for tests (along with a bunch of other prefs). For the branches, our migration scripts automatically take care of changing the compatibility versions.