692232 - Modify repack service to use new version string scheme

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Add-on Validation, defect)

Description

[Daniel Buchner [:dbuc]]

The SDK version is already being recorded on AMO in the db, this means that repacks do not need to append anything to the add-on version string that indicates what SDK version is being used. Instead, we will do the following:

In the case of a repack where the last position ends in a char, or the version string has less than 3 places: 1.foo.bar > 1.foo.bar.1 and 1.0 > 1.0.1

In the case the last position is a number and the version has 3 or more places: 1.0.0 > 1.0.1

This proposed scheme should always bump compatibility. When repacks need to be done, instead of checking version strings, we would just refer to the SDK version data that is stored in the AMO db - obviously storing this data outside a version string that can change via user manipulation at any time is best, I am as yet unable to determine why we wanted to append a guid to the version in the first place vs just adding/iterating numerical positions.

For add-ons already on AMO with the sdk string append, we need to remove that append, and add a .1 to the end of the version string. This will trigger the update system as far as I am aware.

Please advise if there is anything that would not work in this proposal. DO NOT ACT ON THIS BUG UNTIL WE HAVE CONFIRMED THE VIABILITY OF THIS FIX.

Comment 1

[Piotr Zalewa [:zalun]]

Everything which needs to be done is to change the way AMO uses the repacking API. http://flightdeck.readthedocs.org/en/latest/repackage/api.html

Comment 2

[Wil Clouser [:clouserw]]

These are our current versions with 'sdk' in them:  

  http://pastebin.mozilla.org/1347289

From what I understand (and happy to hear corrections), we'd need SQL along these lines on AMO: 

  UPDATE versions SET version=REPLACE(version,'.sdk.1.1','.1');

Comment 3

[Dave Townsend [:mossop]]

(In reply to Wil Clouser [:clouserw] from comment #2)
> These are our current versions with 'sdk' in them:  
> 
>   http://pastebin.mozilla.org/1347289
> 
> From what I understand (and happy to hear corrections), we'd need SQL along
> these lines on AMO: 
> 
>   UPDATE versions SET version=REPLACE(version,'.sdk.1.1','.1');

Updating the version in the AMO database is a first step, however unless we also update the version on the add-on's install.rdf to match Firefox will constantly believe that AMO has a newer version available and so download and install it every day. Compatibility bumps to that version would also not work in the future.

Comment 4

[Daniel Buchner [:dbuc]]

The affected add-ons currently possess the following version string scheme: 1.0.sdk.1.0

In order to correct this, we will need to modify the version string in two locations. The first location is the version string in the AMO db, the second is the version string present in the install.rdf

When Wil and I reviewed this plan, I mentioned the need to replace the version in the rdf file. He told me that the repacker was the tool to do so and that no other method presently exists for modifying that file.

Given that information, and the need to make this change in a timely fashion, Wil and I feel the repacker may be the only route that serves both requirements.

---

The proposed plan to fix the busted add-on versions in the short term is as follows:

- We would modify the repacker to output add-ons with the following change to their version string scheme: 1.0.sdk.1.0 becomes 1.0.1 (Wil advocated for dropping the sdk string from the version for the same reasons Mossop mentioned previously)

- A repack run would be initiated on the affected add-ons as soon as we verified the modified version string scheme was packed and output correctly by the repacker on test add-ons.

---

I know there was a desire to avoid the repacker for this task, but given the circumstance, it appears to be our best (possibly only) option. It should also be noted, that Piotr has already fixed the issues we experiences with the first run of the repacker. (including the carry-over of the icons and screenshots to the new repacked add-ons).

Please let me know what you think. If we get sign-off, we'll shoot for executing on this plan tomorrow afternoon (10/6/11)

Comment 5

[Jeff Balogh (:jbalogh)]

A one-off script that edits the xpi files will have fewer moving parts. And it won't trigger emails to developers or create another new version for each add-on.

Comment 6

[Jorge Villalobos [:jorgev] (he/him)]

(In reply to Jeff Balogh (:jbalogh) from comment #5)
> A one-off script that edits the xpi files will have fewer moving parts. And
> it won't trigger emails to developers or create another new version for each
> add-on.

... or move add-ons back in the review queue again (bug 689635).

Comment 7

[Dave Townsend [:mossop]]

Writing a new script carries risk too but I trust the AMO devs to reason about that and it sounds like it would have benefits here so I'm on board with either option. We should test whatever we do on some add-ons on staging and verify that Firefox sees updates for them before we do this for real.

Comment 8

[Daniel Buchner [:dbuc]]

(In reply to Jeff Balogh (:jbalogh) from comment #5)
> A one-off script that edits the xpi files will have fewer moving parts. And
> it won't trigger emails to developers or create another new version for each
> add-on.

I asked about doing something like this when talking to Wil, and while I knew it was likely possible, Wil said it might take quite a bit longer to get ready. What do you think the time frame would be on creating something like that? (We really need to execute on this tomorrow/this week).

As far as the email issue goes, I'm pretty sure the email functionality can be switched off, correct?

Comment 9

[Piotr Zalewa [:zalun]]

Editing xpi is trivial (unzip install.rdf / replace sdk.1.1 -> 1 / zip it) - I see more issues in copying these across all mirrors, but it might not an issue at all.

Comment 10

[Wil Clouser [:clouserw]]

(In reply to Daniel Buchner [:dbuc] from comment #8)
> As far as the email issue goes, I'm pretty sure the email functionality can
> be switched off, correct?

That's not an option I know of with the current UI.

Comment 11

[Daniel Buchner [:dbuc]]

The immediate fix for add-ons currently on AMO suffering version string issues, will be the execution of a targeted script on AMO, as described in bug 692524. This bug has been refocused to include only the repacker modifications necessary to support the new version string scheme proposed (and ratified by stakeholder parties) in Comment 1 above.

Comment 12

[Piotr Zalewa [:zalun]]

Can we close it already?

Comment 13

[Wil Clouser [:clouserw]]

(In reply to Piotr Zalewa [:zalun] from comment #12)
> Can we close it already?

I guess?