Get extension compatibility auto-increment on AMO to run for SeaMonkey

RESOLVED WONTFIX

Status

RESOLVED WONTFIX
8 years ago
3 years ago

People

(Reporter: prof, Unassigned)

Tracking

Details

(Reporter)

Description

8 years ago
While Firefox has some build-machines or similar which allow for automated checks concerning extension compatibility, Seamonkey lacks those. Seamonkey isn't very popular among extension developers (compared to FF/TB) and I expect more severe impact based on the new rapid release model where version-numbers increase faster and compatibilities like 2.0.x or 2.1.x are very short-lived and extensions are becoming "incompatible" very quickly. 

It seems important to get more extensions automatically compatible with x.y-relreases and make developers aware of Seamonkey's development/progress.
Therefore I suggest to contact AMO to request/ask for automated tests/bots/build-machines to make use of this infrastructure, too.

Comment 1

8 years ago
Well we can ask, but the answer would probably say that their finite resources are better off deployed to support Firefox.
(Reporter)

Comment 2

8 years ago
As they say, you don't ask, you don't get. :-)
Let's see if a little surprise may occur, it's worth a try/question.

Comment 3

8 years ago
I don't know if this is a "testbot/build-machine" or some other kind of tool - you are assuming too many specifics there, I think. ;-)

We'd like some info on how we can get the same add-on compatibility bump that was done for FF5 and FF6 to run for SeaMonkey as well, so it's best to put this bug into the AMO product for now and see if they can either run this for us or advise us how to do it ourselves.

AMO team, can you help Callek, Philip and the SeaMonkey team with that?
Component: Release Engineering → Compatibility Tools
Product: SeaMonkey → addons.mozilla.org
QA Contact: release → compatibility
Summary: Setup testbot/build-machine for extension compatibility auto-increment on AMO → Get extension compatibility auto-increment on AMO to run for SeaMonkey
Version: Trunk → unspecified
IIUC, there's some kind of automated tool at AMO doing a number of tests every six weeks on add-ons supporting the current Firefox beta (or aurora?) build, and bumping the add-on's maximum supported version (in AMO metadata, not in the install.rdf) if it does. (Still IIUC) it would be a simple setup question to let the tool know that Fx5 // Tb5 // Sm2.2, Fx6 // Tb6 // Sm2.3, Fx7 // Tb7 // Sm2.4, Fx8 // Tb8 // Sm2.5 etc.

The algorithm would (I suppose) have to be manually altered the day Sm3 comes out, which (AFAIK) won't happen soon, and will anyway be heralded by a "Sm 3.0a1" version on comm-central, something which hasn't happened yet.

For extensions which rely on a <em:targetApplication>toolkit@mozilla.org</em:targetApplication> to be installable in all apps, manual action by the addon's author would of course still be required, since AMO takes no notice of that.
Machine resources are not an issue for us and I have no problem using those to do SeaMonkey compatibility bumps. Developer resources are an issue and we can't spare any to support SeaMonkey's rapid releases. That includes the time necessary to review any patches and QA the compatibility tools to make sure they work properly for SeaMonkey, haven't broken anything in Firefox, aren't unnecessarily spamming our users, etc.

Some options I see are:
* Use the add-ons MXR to look for any add-ons that have compatibility issues and write SQL to bump the rest.
* Automatically bump all add-ons with every SeaMonkey release and deal with any that happen to be incompatible after the fact.

Comment 7

8 years ago
> * Automatically bump all add-ons with every SeaMonkey release and deal with
> any that happen to be incompatible after the fact.

In terms of compatibility: All backend and toolkit changes that affect Firefox, affect SeaMonkey equally. For the front end we track Firefox pretty closely. So with an extension for Firefox 4 (and SeaMonkey 2.1) that has no problems with firefox 4+n and gets the maxVersion bumped, I'd say it would be safe to bump SeaMonkey to 2.1+0.n.
e.g. Firefox 4 -> SeaMonkey 2.1
Firefox 5 -> SeaMonkey 2.2
Firefox 6 -> SeaMonkey 2.3

I don't know how easy it would be to do this though, so it's only a suggestion.
(In reply to comment #6)
> Machine resources are not an issue for us and I have no problem using those
> to do SeaMonkey compatibility bumps. Developer resources are an issue and we
> can't spare any to support SeaMonkey's rapid releases. That includes the
> time necessary to review any patches and QA the compatibility tools to make
> sure they work properly for SeaMonkey, haven't broken anything in Firefox,
> aren't unnecessarily spamming our users, etc.

To be clear, where is the code that runs this located (repo wise), *DO* you have the resources to let, say, a complete fork work for SeaMonkey if you don't have the resources to review changes to the existing code, etc.?

> 
> Some options I see are:
> * Use the add-ons MXR to look for any add-ons that have compatibility issues
> and write SQL to bump the rest.

This would require us to manually create SQL per release, per addon, with knowledge of the AMO DB at the time of the bump, correct?

> * Automatically bump all add-ons with every SeaMonkey release and deal with
> any that happen to be incompatible after the fact.

This sounds quite dangerous!
(In reply to comment #8)
> To be clear, where is the code that runs this located (repo wise)

Kumar, could you point to where the bulk validation code is?
The validator code is here: https://github.com/mozilla/amo-validator

You can grep the code for 'for_appversions=' to see where compatibility tests are defined.  The checks are registered with the app GUID and a version range, e.g.

for_appversions={'{ec8030f7-c20a-464f-9b0e-13a3a9e97384}':
                             versions_after("firefox", "6.0a1")}

If we need to add Seamonkey in then it might be as simple as:

for_appversions={'{ec8030f7-c20a-464f-9b0e-13a3a9e97384}':
                             versions_after("firefox", "6.0a1"),
                 '<Seamonkey-GUID>': ['2.0.*', '2.1.*']}
(In reply to comment #10)
> The validator code is here: https://github.com/mozilla/amo-validator
> 
> You can grep the code for 'for_appversions=' to see where compatibility
> tests are defined.  The checks are registered with the app GUID and a
> version range, e.g.
> 
> for_appversions={'{ec8030f7-c20a-464f-9b0e-13a3a9e97384}':
>                              versions_after("firefox", "6.0a1")}
> 
> If we need to add Seamonkey in then it might be as simple as:
> 
> for_appversions={'{ec8030f7-c20a-464f-9b0e-13a3a9e97384}':
>                              versions_after("firefox", "6.0a1"),
>                  '<Seamonkey-GUID>': ['2.0.*', '2.1.*']}


So skimming this code, it looks like most of this is the tests that run when an addon author requests tests, not exactly what I was looking for (the "Lets automatically bump compat for [xyz] extensions that should be good").

Where does that happen, is it here; What determins if we can, or can't. If I add SeaMonkey to the list of Firefox tests that are run, will this enable us to just blanket-enable the ones that pass [presuming of course that the remaining ones are fine]?
(In reply to comment #11)
> So skimming this code, it looks like most of this is the tests that run when
> an addon author requests tests, not exactly what I was looking for (the
> "Lets automatically bump compat for [xyz] extensions that should be good").

The validator is used for both scenarios. It's used when a developer submits a new addon or manually triggers a compatibility tests.  It's also used when we run our bulk validation tasks (bug 649864) which you can kind of get a sense of here https://github.com/jbalogh/zamboni/blob/94e49f3d82d3c6c767e288d8771df2d79123ca97/apps/zadmin/tasks.py#L56

There's a lot of buried code but essentially what happens is fligar or someone logs into the admin pane and starts a validation job that says bump Firefox 7 addons to Firefox 8. In the background all FF 7 compatible addons are validated for FF 8 compatibility. If the validation passes without any errors, their max version is bumped automatically.  Otherwise, the developer gets an email with a link to the full error report.

> If I
> add SeaMonkey to the list of Firefox tests that are run, will this enable us
> to just blanket-enable the ones that pass [presuming of course that the
> remaining ones are fine]?

The first step is registering SeaMonkey tests to run in the validator.  After that, yeah, it should more or less fall into place.  An admin will fire off a specific SeaMonkey validation job then click the upgrade button when it finishes.
I just read on Planet that now, with compatible-by-default add-ons, automatic compatibility bumps will not be done anymore. So in that sense, this bug is obsolete now. However, the validation of add-ons will continue. This part is still desired for SeaMonkey, and AFAIU the setup required for automatic bumps and automatic validation is nearly the same. So the question is: Shall we morph this bug into requesting automatic validation, or close it?
Thanks for filing this.  Due to resource constraints we are closing bugs which we won't realistically be able to fix.  If you have a patch that applies to this bug please reopen.

For more info see http://micropipes.com/blog/2014/09/24/the-great-add-on-bug-triage/
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → WONTFIX
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.