1646767 - Android opt tests completely missing from taskgraph on try

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, defect, P1)

Description

[(No longer employed by Mozilla) Aaron Klotz]

STR:

• ./mach try fuzzy
• Look for test-android-em-7.0-x86_64/opt jobs
• Note that there aren't any; only debug tests available.

Comment 1

[(No longer employed by Mozilla) Aaron Klotz]

I have been told that opt might have been replaced by shippable, but the issue stands: there are no non-debug tests available to choose.

Comment 2

[Marco Castelluccio [:marco]]

(In reply to Aaron Klotz [:aklotz] from comment #0)

STR:

• ./mach try fuzzy
• Look for test-android-em-7.0-x86_x64/opt jobs
• Note that there aren't any; only debug tests available.

I assume it's a typo and you were searching for test-android-em-7.0-x86_64/opt?

Comment 3

[(No longer employed by Mozilla) Aaron Klotz]

Yes. Fixed.

Comment 4

[(No longer employed by Mozilla) Aaron Klotz]

I think this is the actual component where this belongs.

Comment 5

[Marco Castelluccio [:marco]]

Probably a regression from bug 1641948.

Comment 6

[Marco Castelluccio [:marco]]

Yes, I see test-android-em-7.0-x86_64/opt tasks on 680987aac53030c97ebb49038622ba9a63f3e3ef (parent revision of 61fccfec51e98f51d9865bf1ff5808016a948a71, which fixed bug 1641948).

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1641948

Comment 8

[[e-mail] (:bhearsum)]

I don't see any issue re-enabling opt for try, but to be clear: we're now running tests against shippable builds for central/autoland (because we have to build shippable for perf tests anyways), so try will be slightly different than central/autoland in that we'll usually run tests against opt.

I also think we should do a tweak similar to what we did for OS X in bug 1541131 to minimize cases where we build both shippable and opt for android on try (pick opt if we're not running perf tests, but pick shippable if we are).

Comment 9

[(No longer employed by Mozilla) Aaron Klotz]

To be clear on my needs from a GV development perspective: I'm trying to fix timing-dependent intermittents that we frequently seen on autoland and central that only occur in tests with non-debug builds, so I need to be able to push to try in configuration that has some semblance of what I am seeing in CI.

In my particular use case, opt appears to be good enough, but I'll leave the opt vs shippable decision to others.

Comment 10

[(No longer employed by Mozilla) Aaron Klotz]

Part of me wonders: why not allow both shippable and opt for pushing to try? Do people really use shippable that often if they get what they need with opt? I for one wouldn't want to bother wasting the extra time waiting for a shippable build on try unless it was absolutely necessary to reproduce the issue.

Comment 11

[[e-mail] (:bhearsum)]

(In reply to Aaron Klotz [:aklotz] from comment #9)

In my particular use case, opt appears to be good enough, but I'll leave the opt vs shippable decision to others.

Thanks for the perspective! In general, there's very few distinct failures between opt and shippable, so it should be OK.

(In reply to Aaron Klotz [:aklotz] from comment #10)

Part of me wonders: why not allow both shippable and opt for pushing to try? Do people really use shippable that often if they get what they need with opt? I for one wouldn't want to bother wasting the extra time waiting for a shippable build on try unless it was absolutely necessary to reproduce the issue.

It's really more of an issue of defaults. You can always run whatever ones you want with ./mach try fuzzy --full, but if we allow both opt and shippable in the non-full list, it means that if you search for reftest and select everything that shows up, you end up with opt and shippable builds for no good reason.

Comment 12

[Geoff Brown [:gbrown]]

I think part of the issue here is bug 1645728: android-em tests for shippable builds are accidentally running as tier 2; as such, android-em shippable tests are not visible with 'mach try fuzzy' -- tier 2 tasks are only visible with 'mach try fuzzy --full'. Once bug 1645728 is resolved, android-em shippable tests should be visible to 'mach try fuzzy'. (There remains the issue of opt vs shippable.)

Comment 13

[[e-mail] (:bhearsum)]

(In reply to Geoff Brown [:gbrown] from comment #12)

I think part of the issue here is bug 1645728: android-em tests for shippable builds are accidentally running as tier 2; as such, android-em shippable tests are not visible with 'mach try fuzzy' -- tier 2 tasks are only visible with 'mach try fuzzy --full'. Once bug 1645728 is resolved, android-em shippable tests should be visible to 'mach try fuzzy'. (There remains the issue of opt vs shippable.)

Oh, that's good context to have. The fix for that bug is on autoland already, so it should be on its way to central today. The other part should be relatively simple to address.

Comment 14

[[e-mail] (:bhearsum)]

Attachment: Bug 1646767: Android opt tests completely missing from taskgraph on try

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/780abda87480
Android opt tests completely missing from taskgraph on try r=ahal

Comment 16

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/780abda87480