1324047 - Update the in-tree test timings

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

It's been over a year since they were last updated and some platforms/flavors are seriously out of whack.

Comment 1

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1324047 - Update testing/runtimes/writeruntimes.py for new e10s schema in ActiveData,

At some point ActiveData stopped considering 'e10s' as a separate suite and instead stored this information
in the "run.type" field. This updates the writeruntimes.py script and mochitest runtime file resolving accordingly.

Review commit: https://reviewboard.mozilla.org/r/99784/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/99784/

Comment 2

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1324047 - Update test runtimes data files,

Review commit: https://reviewboard.mozilla.org/r/99786/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/99786/

Comment 3

[Andrew Halberstadt [:ahal]]

Here's a try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4821d5bbb1a5d29f0d0c37a2a936ced7da16396c

It looks very slightly better from some unscientific comparing. However, all the chunks I compared to looked very similar in runtime even before this patch. Ryan, which platforms/flavors specifically should I be looking at to see how much of a difference this makes?

I'll land this anyway in the meantime, as it shouldn't hurt.. You can treat this needinfo as a request to verify whether this makes the platforms/flavors you care about better.

Comment 4

[Jonathan Griffin (:jgriffin)]

Comment on attachment 8820248 [details]
Bug 1324047 - Update testing/runtimes/writeruntimes.py for new e10s schema in ActiveData,

https://reviewboard.mozilla.org/r/99784/#review100384

Comment 5

[Jonathan Griffin (:jgriffin)]

Comment on attachment 8820249 [details]
Bug 1324047 - Update test runtimes data files,

https://reviewboard.mozilla.org/r/99786/#review100386

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fe3aa973cfdd
Update testing/runtimes/writeruntimes.py for new e10s schema in ActiveData, r=jgriffin
https://hg.mozilla.org/integration/autoland/rev/ee94d1be41a3
Update test runtimes data files, r=jgriffin

Comment 7

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/fe3aa973cfdd
https://hg.mozilla.org/mozilla-central/rev/ee94d1be41a3

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Linux debug dt was the one I was noticing in particular. Looks better now on trunk (ranging in the 55-65min range roughly per chunk).

Comment 9

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/70f7c3097f86
https://hg.mozilla.org/releases/mozilla-aurora/rev/cbcb03d56b92