1610611 - puppeteer is permafailing on autoland

Status: RESOLVED FIXED

(Remote Protocol :: Agent, enhancement)

Description

[Edwin Takahashi (:egao | infrequent contributor)]

:jmaher and I discussed this topic during 1:1.

Currently we have the puppeteer task running on autoland. This is a permafail tier 3 task that takes ~25 minutes per run, wasting a lot of resources. This task is also scheduled for every autoland push.

Comment 1

[Edwin Takahashi (:egao | infrequent contributor)]

:bc - jmaher asked me to ni you for this task. Let me know if you want me to work on it though, if you're busy.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

It's failing which is right, and that's why it is tier-3. We are constantly working on making more and more tests passing. See https://docs.google.com/spreadsheets/d/1GZ4yO2-NGD6kbsT7aMlUPUpUqTaASpqNxJGKhOQ-_BM/edit#gid=0 for the current status.

If it's too much for autoland we may want to only run this task when anything under /remote changes. If that's doable it would be fine for now.

Maja, any objections from your side?

Comment 3

[Bob Clary [:bc] (inactive)]

pup and pup-fis both are perma-orange. These are also configured to run on all projects.

ato: Does this test provide value in its current state? Is there any reason to block disabling this test until it is greened up? If we do need to keep it running in some state, can we reduce the projects this runs on from all to just mozilla-central?

whimboo, maja_zf: Is there benefit to running this given the number of failures remaining to clean up? Would it be an undue burden to disable this everywhere except try until it is green? Restricting to /remote would achieve almost the same effect as disabling everywhere. Personally, I'd vote to disable until green(ish) and failing that option, restrict to running only when /remote changes.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The Fission job we could clearly restrict to try for now. But the normal job I would love to see running on central with changes to /remote. Lets see what Maja thinks...

Comment 5

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #4)

The Fission job we could clearly restrict to try for now. But the normal job I would love to see running on central with changes to /remote. Lets see what Maja thinks...

I agree with Henrik

Comment 6

[Bob Clary [:bc] (inactive)]

Attachment: Bug 1610611 - Restrict remote puppeteer tests to try, mozilla-central; remote puppeteer fission tests to try only, r=whimboo.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d4c1f5c8d353
Restrict remote puppeteer tests to try, mozilla-central; remote puppeteer fission tests to try only, r=whimboo.

Comment 8

[Bob Clary [:bc] (inactive)]

Please don't land my patches. It messed up the blame in the log and confused me for a bit when I didn't see it after rebasing and when I couldn't find the patch in the log under my name.

Unfortunately, it appears the source-test kind transforms do not support optionally keyed by by-test-platform meaning it would be non-trivial to restrict this to opt and not shippable/opt.

Comment 9

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/d4c1f5c8d353