562360 - Compatibility information for add-ons on preview.addons.mozilla.org are not in sync with public pages

Status: RESOLVED WONTFIX

(addons.mozilla.org Graveyard :: Public Pages, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

For our Mozmill tests we are using preview.addons.mozilla.org to install extensions, themes, and search engines. That decision has been made because we do not want to falsify the download numbers of add-ons on the public page. Further we could detect regressions on AMO earlier and not after a release.

To be able to continue our testing for Firefox 3.7 alpha builds the compatibility information on preview.addons.mozilla.org has to be updated. Right now, it is not in sync with the public pages and offers a max supported version of 3.7a2pre. Please bump it up to 3.7a5pre.

Can we further have a script which automatically update the max application version at the same time when the public page gets modified? Otherwise we would run into the same problem once the Firefox version gets bumped up.

This bug blocks us from running Mozmill tests against Firefox 3.7. It would be great when we can resolve it asap. Thanks.

Comment 1

[Nick Nguyen [:osunick]]

Fligtar will figure out what to do.

Comment 2

[Nick Nguyen [:osunick]]

Henrik- can you provide a list of all the max versions you'd expect to see for the next month or so?

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

We are limiting our tests and are only using the Nightly Tester Tools at the moment. We do not have tests yet which check for incompatible add-ons, and I have no idea if and when those will come.

Instead of having to specify the max versions for the next month and again for the next alpha release, would it be possible to have the compatibility info in sync with the public page? We have to update addons.mozilla.org after each new 1.9.3a/b release. So having an automated way would be highly appreciated.

But to answer your question, our Mozmill tests will always run with the most recent Minefield nightly build.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As Dave told me, this also blocks us from uploading any Jetpack to preview.addons.mozilla.org. Can we try to get it solved temporarily for our testday today and find a more proper solution within the next days? That would be really helpful.

Comment 5

[Nick Nguyen [:osunick]]

I've added 3.7a5pre to Preview.

Comment 6

[Nick Nguyen [:osunick]]

Henrik, can you send me your AMO login so you can do this in the future?

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Nick, I would really like to leave this work up to you and your team. Can you update your internal docs to also upgrade preview when bumping the version number on AMO itself?

Comment 8

[Justin Scott [:fligtar]]

If the issue is that you're installing add-ons that aren't marked as compatible with the version of Firefox you're testing, adding new version numbers won't fix that problem. Adding new versions numbers simply allows developers the option to mark their add-ons compatible, but does not automatically do so.

So, I don't think automatically adding new version numbers to preview will fix the problem you're having. And we can't sync the entire database with preview because preview is for staging our next release, and we make database changes to its test database.

Possible solutions I see are:
* testing with extensions.checkCompatibility.* turned off
* figuring out a way to download add-ons from AMO without counting as downloads

Comment 9

[Nick Nguyen [:osunick]]

It sounds like you also want to test new add-ons written for 3.7a5pre (like atul's restartless extension).  

Since we aren't uploading these versions to AMO, and someone from the QA team is, it is reasonable and trivial to up the maxver on preview when these add-ons are tested.  Stephen Donner can help you with this.

The only reason to host test extensions on preview is to test the maxver override and whitelist service- otherwise you can just host your test extensions on a server somewhere.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Justin, would the following proposal work?

* The AMO team takes care about the max version number on preview and bumps up the version at the same time as for the public AMO website

* QA will upload their own extensions (including jetpacks) and themes for testing purposes. We will be able to update the compatibility info as soon as step 1 happened.

With that way we do not conflict with other add-ons and have full control of items we test.

Comment 11

[Maniac Vlad Florin (:vladmaniac)]

(In reply to Henrik Skupin (:whimboo) from comment #10)
> Justin, would the following proposal work?
> 
> * The AMO team takes care about the max version number on preview and bumps
> up the version at the same time as for the public AMO website
> 
> * QA will upload their own extensions (including jetpacks) and themes for
> testing purposes. We will be able to update the compatibility info as soon
> as step 1 happened.
> 
> With that way we do not conflict with other add-ons and have full control of
> items we test.

We now have our own test add-ons if this still is a Mozmill concern. Based on that we can put this to resolved fixed. Please correct otherwise

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+1]]

That bug is not related to how we use our list of extensions. It's for AMO specifically. Not sure what the current status is but I wouldn't resolve the bug as fixed now.

Comment 13

[Wil Clouser [:clouserw]]

Thanks for filing this.  Due to resource constraints we are closing bugs which we won't realistically be able to fix.  If you have a patch that applies to this bug please reopen.

For more info see http://micropipes.com/blog/2014/09/24/the-great-add-on-bug-triage/