1495372 - Perma [tier2] Wm Fatal: manifest path /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_tests/web-platform/meta does not exist

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect)

Description

[Eliza Balazs [:ebalazs_]]

This started after https://bugzilla.mozilla.org/show_bug.cgi?id=1495007 landed:
https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=pending,running,success,testfailed,busted,exception,usercancel,runnable&searchStr=linux,x64,opt,source-test-wpt-manifest-upload,(wm)&selectedJob=202530458

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=202530458&repo=autoland

Error processing command. Ignoring because optional. (optional:packages.txt:comm/build/virtualenv_packages.txt)
[task 2018-10-01T06:58:39.972Z]  0:01.14 WARNING No generated manifest found
[task 2018-10-01T06:58:39.972Z]  0:01.14 INFO Could not download manifests.
[task 2018-10-01T06:58:39.972Z]  0:01.14 INFO Generating from scratch instead.
[task 2018-10-01T06:58:39.974Z]  0:01.15 INFO Updating manifests
[task 2018-10-01T06:58:39.976Z] Fatal: manifest path /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_tests/web-platform/meta does not exist
[taskcluster 2018-10-01 06:58:40.649Z] === Task Finished ===
[taskcluster 2018-10-01 06:58:40.733Z] Artifact "public/manifests.tar.gz" not found at "/builds/worker/checkouts/gecko/manifests.tar.gz"
[taskcluster 2018-10-01 06:58:41.379Z] Unsuccessful task run with exit code: 1 completed in 288.093 seconds

jgraham: Can you take a look into this? Thanks!

Comment 1

[James Graham [:jgraham]]

Attachment: Bug 1495372 - Unify wpt manifest download and update

The previous code split a lot of logic between the update and download
parts, and exposed two different mach commands. In order to simplify
things it's better to have a single entry point for manifest download
and update, and ensure that's called consistently.

Comment 2

[Intermittent Failures Robot]

30 failures in 561 pushes (0.053 failures/push) were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 3
* autoland: 27

Platform breakdown:
* linux64-asan: 4
* linux64: 26

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495372&startday=2018-10-02&endday=2018-10-02&tree=all

Comment 3

[Eliza Balazs [:ebalazs_]]

ato: Hi, This is a perma failure, your review would be helpful? Thank you!

Comment 4

[Andreas Tolfsen ❲:ato❳]

jgraham: You updated the patch, did you not land this?

Comment 5

[Andreas Tolfsen ❲:ato❳]

Sorry, I misinterpreted the needinfo from ebalazs_ to mean I needed
to look at a backout of jgraham’s patch.  I must have thought this
because I reviewed some code in a related area in WPT last week.
Sorry about this.

I’ve now reviewed the patch and I can’t spot any major obstacles
for why we shouldn’t be able to land this.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8660ad891a23
Unify wpt manifest download and update r=ato

Comment 7

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/13347 for changes under testing/web-platform/tests

Comment 8

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/8660ad891a23

Comment 9

[Cosmin Sabou [:CosminS]]

Backed out for causing win2012 bustages.

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=busted&searchStr=wind,2012&selectedJob=203201456

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=203198261&repo=autoland&lineNumber=42733

Backout link: https://hg.mozilla.org/mozilla-central/rev/65fc22a1627b7de54303275dbe2422a61080865c

22:27:35     INFO -  package-tests>   File "z:\build\build\src\testing\web-platform\manifestdownload.py", line 142, in download_manifest
22:27:35     INFO -  package-tests>
22:27:35     INFO -  package-tests>     logger.debug("Unpacking %s to %s" % (member.name, paths["manifest_path"]))
22:27:35     INFO -  package-tests>
22:27:35     INFO -  package-tests> UnboundLocalError: local variable 'member' referenced before assignment
22:27:35     INFO -  package-tests>
22:27:35     INFO -  package-tests> z:/build/build/src/testing/testsuite-targets.mk:158: recipe for target 'download-wpt-manifest' failed
22:27:35     INFO -  package-tests> mozmake.EXE[2]: *** [download-wpt-manifest] Error 1
22:27:35     INFO -  package-tests> mozmake.EXE[2]: Leaving directory 'z:/build/build/src/obj-firefox'
22:27:35     INFO -  package-tests> mozmake.EXE[2]: *** Waiting for unfinished jobs....
22:28:05     INFO -  package> mozmake.EXE[5]: Entering directory 'z:/build/build/src/obj-firefox/browser/installer'

Comment 10

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/13347
* Taskcluster (pull_request) (https://tools.taskcluster.net/task-group-inspector/#/VdT0L5OrQ8aQl7HzxGFTKA)

Comment 11

[Intermittent Failures Robot]

15 failures in 465 pushes (0.032 failures/push) were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 8
* autoland: 7

Platform breakdown:
* linux64: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495372&startday=2018-10-03&endday=2018-10-03&tree=all

Comment 12

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/13354 for changes under testing/web-platform/tests

Comment 13

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2adff75db60c
Unify wpt manifest download and update, r=ato

Comment 15

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/2adff75db60c

Comment 16

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged

Comment 17

[Intermittent Failures Robot]

57 failures in 2708 pushes (0.021 failures/push) were associated with this bug in the last 7 days.

This is the #31 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 42
* mozilla-inbound: 15

Platform breakdown:
* linux64: 53
* linux64-asan: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495372&startday=2018-10-01&endday=2018-10-07&tree=all

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/7b67ae2b52ae
[wpt PR 13354] - [Gecko Bug 1495372] Backed out changeset 8660ad891a23 (bug 1495372) for causing win2012 bustages. a=backout, a=testonly

Comment 19

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/7b67ae2b52ae