670146 - Get extension compatibility auto-increment on AMO to run for SeaMonkey

Status: RESOLVED WONTFIX

(addons.mozilla.org Graveyard :: Compatibility Tools, defect)

Description

[Martin F.]

While Firefox has some build-machines or similar which allow for automated checks concerning extension compatibility, Seamonkey lacks those. Seamonkey isn't very popular among extension developers (compared to FF/TB) and I expect more severe impact based on the new rapid release model where version-numbers increase faster and compatibilities like 2.0.x or 2.1.x are very short-lived and extensions are becoming "incompatible" very quickly. 

It seems important to get more extensions automatically compatible with x.y-relreases and make developers aware of Seamonkey's development/progress.
Therefore I suggest to contact AMO to request/ask for automated tests/bots/build-machines to make use of this infrastructure, too.

Comment 1

[Philip Chee]

Well we can ask, but the answer would probably say that their finite resources are better off deployed to support Firefox.

Comment 2

[Martin F.]

As they say, you don't ask, you don't get. :-)
Let's see if a little surprise may occur, it's worth a try/question.

Comment 3

[Robert Kaiser]

I don't know if this is a "testbot/build-machine" or some other kind of tool - you are assuming too many specifics there, I think. ;-)

We'd like some info on how we can get the same add-on compatibility bump that was done for FF5 and FF6 to run for SeaMonkey as well, so it's best to put this bug into the AMO product for now and see if they can either run this for us or advise us how to do it ourselves.

AMO team, can you help Callek, Philip and the SeaMonkey team with that?

Comment 4

[Tony Mechelynck [:tonymec]]

IIUC, there's some kind of automated tool at AMO doing a number of tests every six weeks on add-ons supporting the current Firefox beta (or aurora?) build, and bumping the add-on's maximum supported version (in AMO metadata, not in the install.rdf) if it does. (Still IIUC) it would be a simple setup question to let the tool know that Fx5 // Tb5 // Sm2.2, Fx6 // Tb6 // Sm2.3, Fx7 // Tb7 // Sm2.4, Fx8 // Tb8 // Sm2.5 etc.

The algorithm would (I suppose) have to be manually altered the day Sm3 comes out, which (AFAIK) won't happen soon, and will anyway be heralded by a "Sm 3.0a1" version on comm-central, something which hasn't happened yet.

For extensions which rely on a <em:targetApplication>[e-mail]</em:targetApplication> to be installable in all apps, manual action by the addon's author would of course still be required, since AMO takes no notice of that.

Comment 5

[Philip Chee]

Ref: http://forums.mozillazine.org/viewtopic.php?p=11002781#p11002781
The following appear to have been autoincremented for Firefox to 6.0.

https://addons.mozilla.org/addon/showip/
https://addons.mozilla.org/addon/epubreader/
https://addons.mozilla.org/addon/optimizegoogle/
https://addons.mozilla.org/addon/image-zoom/

It should be safe to autoincrement the SeaMonkey maxVersion to 2.4

Comment 6

[Justin Scott [:fligtar]]

Machine resources are not an issue for us and I have no problem using those to do SeaMonkey compatibility bumps. Developer resources are an issue and we can't spare any to support SeaMonkey's rapid releases. That includes the time necessary to review any patches and QA the compatibility tools to make sure they work properly for SeaMonkey, haven't broken anything in Firefox, aren't unnecessarily spamming our users, etc.

Some options I see are:
* Use the add-ons MXR to look for any add-ons that have compatibility issues and write SQL to bump the rest.
* Automatically bump all add-ons with every SeaMonkey release and deal with any that happen to be incompatible after the fact.

Comment 7

[Philip Chee]

> * Automatically bump all add-ons with every SeaMonkey release and deal with
> any that happen to be incompatible after the fact.

In terms of compatibility: All backend and toolkit changes that affect Firefox, affect SeaMonkey equally. For the front end we track Firefox pretty closely. So with an extension for Firefox 4 (and SeaMonkey 2.1) that has no problems with firefox 4+n and gets the maxVersion bumped, I'd say it would be safe to bump SeaMonkey to 2.1+0.n.
e.g. Firefox 4 -> SeaMonkey 2.1
Firefox 5 -> SeaMonkey 2.2
Firefox 6 -> SeaMonkey 2.3

I don't know how easy it would be to do this though, so it's only a suggestion.

Comment 8

[Justin Wood (:Callek)]

(In reply to comment #6)
> Machine resources are not an issue for us and I have no problem using those
> to do SeaMonkey compatibility bumps. Developer resources are an issue and we
> can't spare any to support SeaMonkey's rapid releases. That includes the
> time necessary to review any patches and QA the compatibility tools to make
> sure they work properly for SeaMonkey, haven't broken anything in Firefox,
> aren't unnecessarily spamming our users, etc.

To be clear, where is the code that runs this located (repo wise), *DO* you have the resources to let, say, a complete fork work for SeaMonkey if you don't have the resources to review changes to the existing code, etc.?

> 
> Some options I see are:
> * Use the add-ons MXR to look for any add-ons that have compatibility issues
> and write SQL to bump the rest.

This would require us to manually create SQL per release, per addon, with knowledge of the AMO DB at the time of the bump, correct?

> * Automatically bump all add-ons with every SeaMonkey release and deal with
> any that happen to be incompatible after the fact.

This sounds quite dangerous!

Comment 9

[Justin Scott [:fligtar]]

(In reply to comment #8)
> To be clear, where is the code that runs this located (repo wise)

Kumar, could you point to where the bulk validation code is?

Comment 10

[Kumar McMillan [:kumar]]

The validator code is here: https://github.com/mozilla/amo-validator

You can grep the code for 'for_appversions=' to see where compatibility tests are defined.  The checks are registered with the app GUID and a version range, e.g.

for_appversions={'{ec8030f7-c20a-464f-9b0e-13a3a9e97384}':
                             versions_after("firefox", "6.0a1")}

If we need to add Seamonkey in then it might be as simple as:

for_appversions={'{ec8030f7-c20a-464f-9b0e-13a3a9e97384}':
                             versions_after("firefox", "6.0a1"),
                 '<Seamonkey-GUID>': ['2.0.*', '2.1.*']}

Comment 11

[Justin Wood (:Callek)]

(In reply to comment #10)
> The validator code is here: https://github.com/mozilla/amo-validator
> 
> You can grep the code for 'for_appversions=' to see where compatibility
> tests are defined.  The checks are registered with the app GUID and a
> version range, e.g.
> 
> for_appversions={'{ec8030f7-c20a-464f-9b0e-13a3a9e97384}':
>                              versions_after("firefox", "6.0a1")}
> 
> If we need to add Seamonkey in then it might be as simple as:
> 
> for_appversions={'{ec8030f7-c20a-464f-9b0e-13a3a9e97384}':
>                              versions_after("firefox", "6.0a1"),
>                  '<Seamonkey-GUID>': ['2.0.*', '2.1.*']}


So skimming this code, it looks like most of this is the tests that run when an addon author requests tests, not exactly what I was looking for (the "Lets automatically bump compat for [xyz] extensions that should be good").

Where does that happen, is it here; What determins if we can, or can't. If I add SeaMonkey to the list of Firefox tests that are run, will this enable us to just blanket-enable the ones that pass [presuming of course that the remaining ones are fine]?

Comment 12

[Kumar McMillan [:kumar]]

(In reply to comment #11)
> So skimming this code, it looks like most of this is the tests that run when
> an addon author requests tests, not exactly what I was looking for (the
> "Lets automatically bump compat for [xyz] extensions that should be good").

The validator is used for both scenarios. It's used when a developer submits a new addon or manually triggers a compatibility tests.  It's also used when we run our bulk validation tasks (bug 649864) which you can kind of get a sense of here https://github.com/jbalogh/zamboni/blob/94e49f3d82d3c6c767e288d8771df2d79123ca97/apps/zadmin/tasks.py#L56

There's a lot of buried code but essentially what happens is fligar or someone logs into the admin pane and starts a validation job that says bump Firefox 7 addons to Firefox 8. In the background all FF 7 compatible addons are validated for FF 8 compatibility. If the validation passes without any errors, their max version is bumped automatically.  Otherwise, the developer gets an email with a link to the full error report.

> If I
> add SeaMonkey to the list of Firefox tests that are run, will this enable us
> to just blanket-enable the ones that pass [presuming of course that the
> remaining ones are fine]?

The first step is registering SeaMonkey tests to run in the validator.  After that, yeah, it should more or less fall into place.  An admin will fire off a specific SeaMonkey validation job then click the upgrade button when it finishes.

Comment 13

[Jens Hatlak (:InvisibleSmiley)]

I just read on Planet that now, with compatible-by-default add-ons, automatic compatibility bumps will not be done anymore. So in that sense, this bug is obsolete now. However, the validation of add-ons will continue. This part is still desired for SeaMonkey, and AFAIU the setup required for automatic bumps and automatic validation is nearly the same. So the question is: Shall we morph this bug into requesting automatic validation, or close it?

Comment 14

[Wil Clouser [:clouserw]]

Thanks for filing this.  Due to resource constraints we are closing bugs which we won't realistically be able to fix.  If you have a patch that applies to this bug please reopen.

For more info see http://micropipes.com/blog/2014/09/24/the-great-add-on-bug-triage/