Closed Bug 1419475 Opened 3 years ago Closed 2 years ago

mochitest-webgl tests are retried too often

Categories

(Testing :: Code Coverage, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: marco, Assigned: marco)

References

Details

Attachments

(2 files)

Most mochitest-webgl chunks are retried several times until they fail.
(In reply to Marco Castelluccio [:marco] from comment #0)
> Most mochitest-webgl chunks are retried several times until they fail.

Well, the tests don't actually fail, they never start.
Do you know what might be causing this?
Flags: needinfo?(rthijssen)
most likely bug 1372172
Flags: needinfo?(rthijssen)
Rob, are you still working on bug 1372172?
Any idea why this is happening so often with windows10-64-ccov/debug compared to other builds (e.g. windows10-64/debug)? Take a look at https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-tier=3&filter-tier=2&filter-tier=1&filter-searchStr=win%2010%2064%20debug.
Flags: needinfo?(rthijssen)
we don't know what causes it yet but when the gpu gets used by a test, the instance often hangs (impaired state in ec2). before i went on pto i thought it might be related to the theme change (windows falling back to classic theme in the middle of some gpu intensive task), from bug 1343049 but the patch there seems to have not had any effect. so yes, now that i'm back from pto i am looking at this again.
Flags: needinfo?(rthijssen)
Attached patch PatchSplinter Review
Attachment #8934939 - Flags: review?(jmaher)
Comment on attachment 8934939 [details] [diff] [review]
Patch

Review of attachment 8934939 [details] [diff] [review]:
-----------------------------------------------------------------

thanks for the comment linking to the bug
Attachment #8934939 - Flags: review?(jmaher) → review+
Pushed by mcastelluccio@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/5d29d8150f97
Disable mochitest-webgl tests as they are retried too often, until the infrastructure problem is fixed. r=jmaher
Pushed by mcastelluccio@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/eb74c83f92c2
Add another space before comment in yaml file to avoid linting failure. r=me
:grenade, mochitest-gpu are affected by the same problem (they never start). Do you think it's bug 1372172? Isn't bug 1372172 intermittent? This problem with mochitest-gl and mochitest-gpu is permanent with coverage builds.
i guess it could be the same issue if they're starting, but we lose contact with the instance before the task start is shown in treeherder. if you can link to a treeherder job where the tests should have started, but didn't, i can investigate.
(In reply to Rob Thijssen (:grenade UTC+2) from comment #12)
> i guess it could be the same issue if they're starting, but we lose contact
> with the instance before the task start is shown in treeherder. if you can
> link to a treeherder job where the tests should have started, but didn't, i
> can investigate.

Here it is: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=d75d996016dcf325c2db2ed8a47af512d07ffacd&filter-searchStr=win%20ccov%20gpu.
Duplicate of this bug: 1452507
The suites are passing now that bug 1372172 has been fixed: https://treeherder.mozilla.org/#/jobs?repo=try&revision=d94fa5d9d9e6cbd707bbffb5d960e7d92a72ad4e.
Assignee: nobody → mcastelluccio
Status: NEW → ASSIGNED
Attachment #9027519 - Flags: review?(jmaher)
Comment on attachment 9027519 [details] [diff] [review]
Re-enable test suites

Review of attachment 9027519 [details] [diff] [review]:
-----------------------------------------------------------------

great, I am excited to see all the jobs that failed on win10 getting enabled now:)
Attachment #9027519 - Flags: review?(jmaher) → review+
Pushed by mcastelluccio@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/102c7135d952
Re-enable some mochitest suites using virtual-with-gpu that were failing with Windows ccov builds. r=jmaher
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Keywords: leave-open
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.