(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #14) > one other thought- we would keep all of these tests in the regular reftest.list, but create reftest-qr.list for -qr variants and only run that on -qr variants, correct? Depends on what exactly you mean by the -qr variants. We already have some -qr test sets, e.g. [here](https://searchfox.org/mozilla-central/rev/f36cb2af46edd2659f446b7acdb2154e230ee445/taskcluster/ci/test/test-sets.yml#360) but in that case (which is run on Linux) we still want to run all the reftests. It's only specific resource-constrained platforms (e.g. Android) where we want to run this reftest-qr.list subset. So the way I would do it is create a new reftest-qr-subset job in [reftest.yml](https://searchfox.org/mozilla-central/source/taskcluster/ci/test/reftest.yml) which uses the reftest-qr.list. And then for example [here](https://searchfox.org/mozilla-central/rev/f36cb2af46edd2659f446b7acdb2154e230ee445/taskcluster/ci/test/test-sets.yml#583) I'd add `reftest-qr-subset`. And then adjust the scheduling so that `reftest` runs very rarely (~once/day) and `reftest-qr-subset` runs a little more frequently. If adjusting the scheduling is not possible then we can just drop `reftest` from that test-set. Does that sound reasonable?
Bug 1621095 Comment 15 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #14) > one other thought- we would keep all of these tests in the regular reftest.list, but create reftest-qr.list for -qr variants and only run that on -qr variants, correct? Depends on what exactly you mean by the -qr variants. We already have some -qr test sets, e.g. [here](https://searchfox.org/mozilla-central/rev/f36cb2af46edd2659f446b7acdb2154e230ee445/taskcluster/ci/test/test-sets.yml#360) but in that case (which is run on Linux) we still want to run all the reftests. It's only specific resource-constrained platforms (e.g. Android) where we want to run this reftest-qr.list subset. So the way I would do it is create a new reftest-qr-subset job in [reftest.yml](https://searchfox.org/mozilla-central/source/taskcluster/ci/test/reftest.yml) which uses the reftest-qr.list. And then for example [here](https://searchfox.org/mozilla-central/rev/f36cb2af46edd2659f446b7acdb2154e230ee445/taskcluster/ci/test/test-sets.yml#583) I'd add `reftest-qr-subset`. And then adjust the scheduling so that `reftest` runs very rarely (~once/day) and `reftest-qr-subset` runs a little more frequently (for test platform `android-hw-aarch64-qr`). If adjusting the scheduling is not possible then we can just drop `reftest` from that test-set. Does that sound reasonable?