853147 - Extend update tests to include partner builds and custom distributions

Status: RESOLVED WONTFIX

(Mozilla QA Graveyard :: Infrastructure, defect)

Description

[u279076]

In Firefox 19 we encountered an issue which prevented partner builds and builds with custom distributions from receiving updates to 19.0.1. This is a fairly critical gap in our current testing we'd like to see automated if at all possible.

In the meantime I have a Moztrap test for reference:
https://moztrap.mozilla.org/manage/case/6785/

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I don't think that this is something we can easily do. Also because it relies on a lot of external infrastructure. Anthony, where we have to fetch those builds and what has to be done to verify a successful update?

Comment 2

[u279076]

There are two different cases:
1. Partner builds
2. Custom distribution builds

The partner builds are here:
http://www.firefoxwithbing.com/
http://www.firefoxwithmsn.com/
http://downloads.yahoo.com/firefox/
Maybe we could just stage the current versions on mozqa.com and keep them there for posterity?

The custom distribution builds just use the regular releases from ftp://ftp.mozilla.org/pub/firefox/releases/ with some minor modifications in the install folder (steps 4-6).

In terms of testing, the standard test applies:
1. Download/install the source binary
2. Check for updates
3. Update is found and installs

At their core the builds are the same and they update on the same release channel, they even have the same build IDs. The partners just repack the builds with an add-on which changes some prefs. and a partner string is added to the about dialog.

Comment 3

[u279076]

This has been a long standing request. Henrik, can you please follow up here.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

So to start with custom distribution builds, where can those be found on the FTP server? Are those the funnelcake builds? Would you mind to clarify, Anthony? Thanks.

Comment 5

[u279076]

Partner Repacks are here:
ftp://ftp.mozilla.org/pub/firefox/candidates/%version%-candidates/build1/partner-repacks/

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Sylvestre, would this be anything which could be of help for us? Or should we ignore those kind of builds in regards of update testing? Thing is that it would not work out of the box and at least mozdownload would have to be adapted to be able to download such kind of builds.

Comment 7

[Sylvestre Ledru [:Sylvestre]]

Well, having some testing of the partner repack would be great. Even only a single locale on a single arch would still be an improvement.
After that, if it takes a long time to implement, we should discuss about that.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We are going to shutdown running update tests on bug 1386628. So this is nothing we want to implement.