Closed Bug 1548106 Opened 1 year ago Closed 1 year ago

revisit chunking of web-platform-tests-reftests


(Testing :: General, enhancement, P2)



(firefox68 fixed)

Tracking Status
firefox68 --- fixed


(Reporter: egao, Assigned: egao)


(Blocks 1 open bug)



(1 file)


I noticed that for most of the platforms, web-platform-tests-reftests currently running on mozilla-central, the chunks are set to 6.

This produces tests that run for a relatively short duration in multiple chunks, as exemplified in this try push:


Using Wr1 as standardized example, in the mozilla-central revisio e8aebe488b2f2e567940577de25013d00e818f7c (linked above):

linux64-shippable/opt: 13 minutes, 17:58:49 - 17:54:45 = 00:04:04 spent on setup
windows10-64-shippable/opt: 11 minutes, 18:10:29 - 18:06:32 = 00:03:57 spent on setup
macosx64-shippable/opt: 13 minutes, 17:34:48 - 17:32:15 = 00:02:33 spent on setup


Perhaps investigate if it is feasible to reduce chunks for common platforms that have runtimes of around 20 minutes or less, such that each chunk takes approximately 30 minutes to execute (which is an informal line set in discussions, that chunks ideally should not exceed 30 minutes)

Assignee: nobody → egao

I have two try runs with some changes:

4 chunks across the board

Only a small handful of tests were run in this push:

  • windows10-64
  • windows10-aarch64

windows10-aarch64 has a failure, not sure if this is caused by the chunk resizing.

Overall, this led me to believe I could further reduce the chunks for faster, more stable platforms (eg. windows10-64, linux64)

3 chunks across the board

All platforms were run, and some failures were noted.

It was also noted that ccov builds do not like being put into small chunks since their execution time is long.

Chunk numbers adjusted per platform

Chunks were adjusted as follows:

windows10-aarch64/.*: 4
windows7-32/.*: 4
linux64-ccov/debug: 6
macosx64.*/.*: 6
android.*: 6
default: 3
Type: defect → enhancement
Summary: reduce chunks for web-platform-tests-reftests → revisit chunking of web-platform-tests-reftests
Blocks: 1548160
Priority: -- → P2
No longer blocks: 1548160
Blocks: 1548924
No longer blocks: 1548924
Attachment #9062570 - Attachment description: Bug 1548106 - revisit chunking of web-platform-tests-reftests → Bug 1548106 - task efficiency: review chunking of web-platform-tests-reftests
Blocks: 1548965
No longer blocks: 1548965
Pushed by
task efficiency: review chunking of web-platform-tests-reftests r=jmaher,gbrown
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
You need to log in before you can comment on or make changes to this bug.