Use --update-target-buildid for the Firefox UI updates

RESOLVED WONTFIX

Status

defect
P3
normal
RESOLVED WONTFIX
4 years ago
2 years ago

People

(Reporter: armenzg, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [qa-automation-blocked])

This is a follow up from bug 1148546.

When we test Firefox UI update tests, we simply test that we've updated and not that we've updated to the right build. Passing the extra parameter of --update-target-buildid would take care of it.

We can probably simply do something like this in here: [1]
-  retcode = self._run_test(installer_path, self.channel)
+ retcode = self._run_test(
+       installer_path=installer_path,
+       update_channel=self.channel,
+       build_id=rel_info['build_id'])

http://hg.mozilla.org/build/mozharness/file/ecd1d24223d3/scripts/firefox_ui_updates.py#l315
This is definitely a blocker for us before we can stop running the Mozmill/Marionette tests on our own.
Component: General Automation → Release Automation
QA Contact: catlee → bhearsum
Whiteboard: [qa-automation-blocked]
Ben, out of curiosity, do we test that updates point users to the right target? (with the headless tests)
(In reply to Armen Zambrano Gasparnian [:armenzg] from comment #2)
> Ben, out of curiosity, do we test that updates point users to the right
> target? (with the headless tests)

Yes, but in a different way than looking at the buildid. After applying the update we diff that against the unpacked installer. This checks that user got the right update, and also that applying the MAR results in the same appdir as a fresh install.
No longer depends on: 1148546
So I had a look into the mozharness script and the CLI option is supported! 

python scripts/firefox_ui_updates.py --firefox-ui-branch master --tools-tageloper_config.py --installer-path firefox.tar.bz2 --update-target-buildid 20150821030204

Armen, so this is only a question in how you invoke the script on the build machine right now. Can you point me to that code? Does it include the --update-target-buildid option and the appropriate buildid?
Flags: needinfo?(armenzg)
Yes, finding what value to pass is the difficult part.
I assume we want to pass the buildid of the release which is currently being built and the one we're updating people to.
However, the tools config files, IIUC, the most recent buildid they contain is the buildid of the previous release rather the one being generated (see line 1):
http://hg.mozilla.org/build/tools/file/default/release/updates/mozBeta-firefox-linux64.cfg

I also assume that not all locales and all platforms are supposed to upload to the latest release.

The code where we call fx ui tests is in here:
http://hg.mozilla.org/build/buildbotcustom/file/default/process/release.py#l1328
Notice that when we try to schedule on testers we might not use that code path.

There's buildid values flying around that file, however, I wouldn't want to use those values before talking it with bhearsum.

whimboo: does this make sense?
Flags: needinfo?(armenzg)
(In reply to Armen Zambrano Gasparnian [:armenzg] from comment #5)
> I also assume that not all locales and all platforms are supposed to upload
> to the latest release.

Wrong assumption, then. In fact all supported locales and platforms are required to update to the same build ID of the new build.
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #6)
> (In reply to Armen Zambrano Gasparnian [:armenzg] from comment #5)
> > I also assume that not all locales and all platforms are supposed to upload
> > to the latest release.
> 
> Wrong assumption, then. In fact all supported locales and platforms are
> required to update to the same build ID of the new build.

Don't we sometimes make a release only for Windows? Or certain locales we skip them on a beta because they're not ready?
I'm happy to be wrong but I thought this was the case.
Whereby it can happen that platforms can have different buildids. But then as Robert said, all locales will have the same buildid for the same platform. Btw. it may also be good to get failures for those locales which are no longer supported in the new version, or?
(In reply to Armen Zambrano Gasparnian [:armenzg] from comment #7)
> Don't we sometimes make a release only for Windows? Or certain locales we
> skip them on a beta because they're not ready?

We always build for all of them. If we do not put the updates up for some platform or locale, we actually *want* to see the failure as it's confirmation that we are off the norm (and in the correct way).
Priority: -- → P3
We never finished the project and already landed code will be removed. If we still need such a thing we would implement it via jobs in TC nowadays.
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.