Closed Bug 935214 Opened 8 years ago Closed 8 years ago

Consolidate Android x86 test chunks and sets once kvm supported

Categories

(Infrastructure & Operations Graveyard :: CIDuty, task)

x86
macOS
task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

Attachments

(2 files)

Once kvm support is available for Android x86 emulator tests, run times for mochitests and reftests (including crashtests and jsreftests) will be reduced dramatically.

Currently, we run several test suites as multiple chunks to avoid 60 minute timeouts. Post-kvm, we can consolidate many chunks without risking timeouts.

With fewer chunks, we can consolidate the sets too.

With kvm:
 - all mochitests can be run in one chunk, but we sometimes run out of memory, so better to run as 2 chunks;
 - it is still probably better to run mochitest-gl separately, since it is sometimes problematic and we run mochitest-gl separately on Android 4.0;
 - all jsreftests can be run in one chunk;
 - all reftests can be run in one chunk, but we sometimes run out of memory, so better to run as 2 or 3 chunks;
 - all crashtests can be run in one chunk;
 - robocop requires 2 chunks;
 - all xpcshell tests can be run in one chunk (still).

So we need at least 10 tests:
  M1 M2 M-gl JSR R1 R2 C RC1 RC2 X

We may as well split that into 12 tests to give 3 full sets of 4. I suggest:

  S1 { M1 M2 M-gl JSR }    S2 { R1 R2 R3 C }     S3 { RC1 RC2 RC3 X }
I am not very familiar with this code. If I got it all wrong, I'm happy to scrap it -- just trying to be helpful!
Attachment #827629 - Flags: review?(armenzg)
Attachment #827630 - Flags: review?(armenzg)
I put the first set in ANDROID_X86_MOZHARNESS_DICT and the others in ANDROID_X86_NOT_GREEN_DICT arbitrarily. I expect almost all tests are not green today but should be greenable in a couple of days once kvm and the new emulator are deployed.
Comment on attachment 827629 [details] [diff] [review]
(1) consolidate test chunks

Review of attachment 827629 [details] [diff] [review]:
-----------------------------------------------------------------

r=+ pending deploying the kvm support first.
Attachment #827629 - Flags: review?(armenzg) → review+
Comment on attachment 827630 [details] [diff] [review]
(2) consolidate test sets

Review of attachment 827630 [details] [diff] [review]:
-----------------------------------------------------------------

r=+ pending deploying the kvm support first.
Attachment #827630 - Flags: review?(armenzg) → review+
Attachment #827629 - Flags: checked-in+
Attachment #827630 - Flags: checked-in+
in production
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Component: Platform Support → Buildduty
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.