Closed Bug 1515599 Opened 4 years ago Closed 4 years ago

Prevent android hardware tests from running when using try option syntax


(Firefox for Android Graveyard :: Testing, defect)

Not set


(firefox66 fixed)

Firefox 66
Tracking Status
firefox66 --- fixed


(Reporter: bc, Assigned: bc)




(1 file)

Many people continue to use try: -b o -p all -u all -t none when running tests on try which results in what is most likely unwanted tests running on real android hardware at Bitbar. This is a particular issue with the jit tests but is potentially a problem for any tests.

Unnecessary tests on android hardware consume the capacity of our limited number of devices and reduce their lifetimes.

To make the selection of android hardware tests more conscious, I would like to disable any try testing of android hardware using try option syntax and instead just support mach try fuzzy.

shows 3 runs:

1. try: -b o -p android-api-16,android-aarch64 -u jittests -t none
   without the patch. jit tests are run.

2. try: -b o -p android-api-16,android-aarch64 -u jittests -t none
   with the patch. jit tess are not run.

3. Fuzzy query=android-hw jit
   jit tests are run.
Attachment #9032638 - Flags: review?(ahal)
Comment on attachment 9032638 [details] [diff] [review]

Review of attachment 9032638 [details] [diff] [review]:

Lgtm, though I think this will disable android-hw from all try syntax in general, not just when using -p all. I assume that was your intent.
Attachment #9032638 - Flags: review?(ahal) → review+
Yes, if they aren't using try fuzzy they don't get to run android-hw tests.
Pushed by
Prevent android hardware tests from running when using try option syntax, r=ahal.
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 66
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.