bugzilla.mozilla.org will be intermittently unavailable on Saturday, March 24th, from 16:00 until 20:00 UTC.

Run reftests in one chunk on mozilla-b2g18* branches



Release Engineering
General Automation
5 years ago
5 years ago


(Reporter: ahal, Assigned: ahal)


Firefox Tracking Flags

(Not tracked)



(1 attachment)



5 years ago
Created attachment 714419 [details] [diff] [review]
Patch 1.0 - run reftests in 1 chunk on b2g18*

I noticed we are still only running reftest-sanity tests on the b2g18* branches, yet we still have them in 10 chunks. There aren't any plans to land the extended set of reftests there yet, so lets cut them down to 1 chunk to save on the slave load. Easy short term win.

I *think* this patch will work, though please pay special attention as I'm not 100% sure and don't know how to test it.

I figured that we always have 2 use cases:
1) We want to run reftest-sanity in 1 chunk
2) We want to run the full set in multiple chunks

This patch makes a separate REFTEST_SANITY_ONLY suite_config. A nice side effect is it lets us get rid of all those mozilla-b2g18* extra_args for the manifest special case.
Attachment #714419 - Flags: review?(aki)

Comment 1

5 years ago
Comment on attachment 714419 [details] [diff] [review]
Patch 1.0 - run reftests in 1 chunk on b2g18*

I thought about renaming it to reftest-sanity, but I think that would be visible to tbpl and the like.

Passes test-masters, and this looks right.  Thanks for the patch!
Attachment #714419 - Flags: review?(aki) → review+
in production

Comment 4

5 years ago
Crap, this didn't exactly have the desired effect. All the reftest-sanity tests are running under the "R" symbol, but we still have R2 - R10 running the same ~10 tests as they were before.

Comment 5

5 years ago
I wonder if maybe we should just rename it to reftest-sanity. This has the added benefit of letting people know that "hey! we aren't actually running any meaningful tests here!"

Though, I'm not sure how we are still running R2-R10. I guess buildbot is trying to be clever or something.

Comment 6

5 years ago
Hm so looking at https://tbpl.mozilla.org/?tree=Mozilla-B2g18&rev=c2dfc2565ae0 it now seems to have resolved itself? I'm confused.. but oh well?
If you mean https://tbpl.mozilla.org/?tree=Mozilla-B2g18_v1_0_1&rev=2eb017b9ba1b and https://tbpl.mozilla.org/?tree=Mozilla-B2g18&rev=65998655101c they're showing exactly what they should: the on-push builds finished before the reconfig, so they triggered and ran R1-R10 (except one was still pending after the reconfig when it no longer existed, so I killed that zombie), then the 11pm nightly triggered just R. The tip pushes on each tree have just an R from the on-push build, and an R from the 7am nightly.

Comment 8

5 years ago
Ah thanks, that makes sense. I didn't notice the nightly build. All is well with the world.
Last Resolved: 5 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.