Test packages on mozilla-esr52 contain outdated (over a month old) content

RESOLVED INVALID

Status

RESOLVED INVALID
a year ago
3 months ago

People

(Reporter: whimboo, Assigned: whimboo)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Assignee)

Description

a year ago
When I check the following failure:

https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-esr52&job_id=100093785

I can see in the mozmill-ci job that the test package refers to:

https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-esr52-win32/1495049059/firefox-52.1.2.en-US.win32.test_packages.json

Which then downloads:

https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-esr52-win32/1495049059/firefox-52.1.2.en-US.win32.common.tests.zip

> % grep -r "watershed" tests/firefox-ui/
> tests/firefox-ui//harness/firefox_ui_harness/testcases.py:            # Bug 604364 - We do not support watershed releases yet.
> tests/firefox-ui//harness/firefox_ui_harness/testcases.py:                             'Additional update found due to watershed release {}'.format(

What? Why does this package contain old data?
(Assignee)

Comment 1

a year ago
This is actually not only Windows XP but all platforms which get the same kind of test package. So maybe it's a fault in our pulse script, which crawls Treeherder to find the correct test package.
Summary: Mozmill-CI uses outdated test packages for Firefox ESR52 on Windows XP → Mozmill-CI uses outdated test packages for Firefox ESR52 on Windows
(Assignee)

Comment 2

a year ago
The changeset for the 52.1.2esr release is the following:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-esr52&revision=0dc7c28f20648d597b11b60e90be9a15687656aa&filter-tier=1&filter-tier=2&filter-tier=3

The build job for the Linux 32bit build points to:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-esr52&revision=0dc7c28f20648d597b11b60e90be9a15687656aa&filter-tier=1&filter-tier=2&filter-tier=3&selectedJob=99868163

https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-esr52-win32/1495049059/

So this is the exact URL where we are getting our test packages from, which means that the tests we package here are totally outdated.

In the above case the specific watershed related checks got removed about a month ago from mozilla-esr52 by a patch from myself and uplifted by Ryan via bug 1355009.

Not sure if this only affects our tests, or if other files are also affected.
Component: Firefox UI Tests → General Automation
Flags: needinfo?(catlee)
Product: Testing → Release Engineering
QA Contact: hskupin → catlee
Summary: Mozmill-CI uses outdated test packages for Firefox ESR52 on Windows → Test packages on mozilla-esr52 contain outdated (over a month old) content
Version: Version 3 → unspecified
(Assignee)

Comment 3

a year ago
I had another look at commit for the current 52.1.2esr release and found the following:

https://hg.mozilla.org/releases/mozilla-esr52/file/0dc7c28f20648d597b11b60e90be9a15687656aa/testing/firefox-ui/harness/firefox_ui_harness/testcases.py#l144

Why is that present when the following change landed about a month ago:

https://hg.mozilla.org/releases/mozilla-esr52/rev/efa5bc8349b3

Maybe I miss something? Ryan, can you also help?
Flags: needinfo?(ryanvm)
(Assignee)

Comment 4

a year ago
I talked with Julien on IRC and we figured it out. The problem here is that the uplift was done for the default branch on mozilla-esr52, while the last security builds all are based on the FIREFOX_ESR_52_1_X_RELBRANCH branch:

https://hg.mozilla.org/releases/mozilla-esr52/file/FIREFOX_ESR_52_1_X_RELBRANCH/testing/firefox-ui/harness/firefox_ui_harness/testcases.py

https://hg.mozilla.org/releases/mozilla-esr52/file/default/testing/firefox-ui/harness/firefox_ui_harness/testcases.py

As such we have to wait until the 52.2 release to not see those update tests failures anymore, or do another uplift to the relbranch which might not happen.

Anyway this bug is actually invalid, and we thankfully package the correct test files.
Status: ASSIGNED → RESOLVED
Last Resolved: a year ago
Flags: needinfo?(ryanvm)
Flags: needinfo?(catlee)
Resolution: --- → INVALID
Component: General Automation → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.