1633782 - Perma cram(try) No build detected, test metadata may be incomplete.

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, defect)

Description

[Arthur Iakab [arthur_iakab]]

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=299730632&repo=mozilla-central&lineNumber=121

[task 2020-04-28T04:48:03.099Z] $ ./mach try syntax $testargs --preset bar
[task 2020-04-28T04:48:03.099Z] + Creating config file /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_tests/web-platform/wptrunner.local.ini
[task 2020-04-28T04:48:03.099Z] + Using config path /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_tests/web-platform/wptrunner.local.ini
[task 2020-04-28T04:48:03.099Z] + https://hg.mozilla.org/mozilla-central/json-pushes?changeset=84d37ea90966593e29dfc330720713a7913a2c21&version=2&tipsonly=1
[task 2020-04-28T04:48:03.100Z] + https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.revision.cdd14a1df784642f8741b0f76041e92ba99ec9df.source.manifest-upload
[task 2020-04-28T04:48:03.100Z] + Downloading manifest from https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.revision.cdd14a1df784642f8741b0f76041e92ba99ec9df.source.manifest-upload/artifacts/public/manifests.tar.gz
[task 2020-04-28T04:48:03.100Z] + https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.revision.cdd14a1df784642f8741b0f76041e92ba99ec9df.source.manifest-upload/artifacts/public/manifests.tar.gz
[task 2020-04-28T04:48:03.100Z] + Unpacking meta/MANIFEST.json to /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_tests/web-platform/meta/MANIFEST.json
[task 2020-04-28T04:48:03.100Z] + Unpacking mozilla/meta/MANIFEST.json to /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_tests/web-platform/mozilla/meta/MANIFEST.json
[task 2020-04-28T04:48:03.101Z] Commit message:
[task 2020-04-28T04:48:03.101Z] try: -b do -p win32 -u none -t all --tag bar
[task 2020-04-28T04:48:03.101Z]
[task 2020-04-28T04:48:03.101Z] Pushed via mach try syntax
[task 2020-04-28T04:48:03.101Z]
[task 2020-04-28T04:48:03.102Z] $ ./mach try $testargs --preset bar
[task 2020-04-28T04:48:03.102Z] + Creating config file /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_tests/web-platform/wptrunner.local.ini
[task 2020-04-28T04:48:03.102Z] + Using config path /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_tests/web-platform/wptrunner.local.ini
[task 2020-04-28T04:48:03.102Z] + https://hg.mozilla.org/mozilla-central/json-pushes?changeset=84d37ea90966593e29dfc330720713a7913a2c21&version=2&tipsonly=1
[task 2020-04-28T04:48:03.102Z] + https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.revision.cdd14a1df784642f8741b0f76041e92ba99ec9df.source.manifest-upload
[task 2020-04-28T04:48:03.103Z] + Downloading manifest from https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.revision.cdd14a1df784642f8741b0f76041e92ba99ec9df.source.manifest-upload/artifacts/public/manifests.tar.gz
[task 2020-04-28T04:48:03.103Z] + https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.revision.cdd14a1df784642f8741b0f76041e92ba99ec9df.source.manifest-upload/artifacts/public/manifests.tar.gz
[task 2020-04-28T04:48:03.103Z] + Unpacking meta/MANIFEST.json to /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_tests/web-platform/meta/MANIFEST.json
[task 2020-04-28T04:48:03.103Z] + Unpacking mozilla/meta/MANIFEST.json to /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_tests/web-platform/mozilla/meta/MANIFEST.json
[task 2020-04-28T04:48:03.103Z] Commit message:
[task 2020-04-28T04:48:03.104Z] try: -b do -p win32 -u none -t all --tag bar
[task 2020-04-28T04:48:03.104Z]
[task 2020-04-28T04:48:03.104Z]
[task 2020-04-28T04:48:03.104Z] # Ran 5 tests, 0 skipped, 1 failed.
[task 2020-04-28T04:48:03.211Z] Test configuration changed. Regenerating backend.
[task 2020-04-28T04:48:03.211Z] No build detected, test metadata may be incomplete.
[taskcluster 2020-04-28 04:48:03.569Z] === Task Finished ===
[taskcluster 2020-04-28 04:48:03.569Z] Unsuccessful task run with exit code: 1 completed in 132.593 seconds

Comment 1

[BugBot [:suhaib / :marco]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[Edwin Takahashi (:egao | infrequent contributor)]

:aryx - what is this cram(try) task? The changes that I landed in Bug 1608837 isn't turned on.

Comment 3

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

It tests the mach try fuzzy command (bug 1391675) . From looking at the regressor https://hg.mozilla.org/integration/autoland/annotate/ae97a2314ad89d60d89b1f775d21f48798d9de3f/testing/mozbase/moztest/moztest/resolve.py it's not obvious to me what is failing. The failure log references mozilla-central artifacts like https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.revision.cdd14a1df784642f8741b0f76041e92ba99ec9df.source.manifest-upload/artifacts/public/manifests.tar.gz but also running a new instance of cram(try) on central still fails

There was a wpt manifest generation task 21 hours ago.

ahal, do you have pointers for Edwin why cram(try) fails?

Comment 4

[Andrew Halberstadt [:ahal]]

These are tests that run mach commands on the shell. You can run it with ./mach cramtest tools/tryselect and they are defined in .t files under tools/tryselect/test.

In this particular case, it looks like after your changes the test is causing the load_wpt_data function in the test resolver to get called which is changing the output of the command (.t tests fail if the stdout changes for any reason).

You could try modifying setup.sh to find a clever way of avoiding the WPT manifest load. Or maybe we need to modify the test.

Comment 5

[Edwin Takahashi (:egao | infrequent contributor)]

I think the easiest might be to either:

1. redirect the output to sys.stderr (which I think I do already, but not sure now)
2. suppress the output from manifestupdat within add_wpt_manifest_data entirely

Comment 6

[Edwin Takahashi (:egao | infrequent contributor)]

Attachment: Bug 1633782 - disable manifestupdate logger so it does not pollute taskgraph generation output

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/37716ee6bbc1
disable manifestupdate logger so it does not pollute taskgraph generation output r=ahal

Comment 8

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/37716ee6bbc1

Comment 9

[Edwin Takahashi (:egao | infrequent contributor)]

The patch landed on central, this should resolve the issue.

Comment 10

[Intermittent Failures Robot]

10 failures in 5423 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 8
• autoland: 2

Platform breakdown:

• linux1804-64: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633782&startday=2020-04-27&endday=2020-05-03&tree=all