845472 - Run reftests in parallel or otherwise reduce wall time

Status: RESOLVED DUPLICATE of bug 813742

(Testing :: Reftest, defect)

Description

[Gregory Szorc [:gps]]

This is a purposefully high-level bug requesting that reftests be run in parallel (on the same machine) or achieve the traditional net reduction in wall time that executing in parallel typically achieves. Ideally, the wall time of reftests decreases linearly with the number of CPU cores in a machine. The trend in CPUs is towards more cores: we should make use of them in our automation!

From a quick glance at a random m-c job on TBPL, it appears we spend 30+ hours in wall time running reftests and variants (like crashtests). This is a substantial percentage of the total wall time spent for checkins (>20%). Reduction in the total wall time spent running reftests would allow the build infrastructure to process more jobs and this would benefit everybody. I imagine reduction in wall time would also allow developers running reftests locally to be more productive!

I just talked with dbaron and he seemed to think that this goal is loosely attainable and there are a number of avenues worth pursuing. However, they require some discussions between layout, graphics, and possibly tools/build for the automation components.

I don't expect immediate movement on this bug. I just figured we should have something on file.