1333860 - some tp5o talos tests either not running in e10s mode or not setting e10s property

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[William Lachance (:wlach)]

It looks like around Dec 1st, we stopped reporting talos e10s memory scores seperately from the main scores. You can see the e10s series disappearing (and the non-e10s version of the same test) going bimodal here:

https://treeherder.mozilla.org/perf.html#/graphs?timerange=5184000&series=%5Bmozilla-inbound,2fb451a876e0332a94d719611e0e27631cdd5620,1,1%5D&series=%5Bmozilla-inbound,600ab158ecfcae0f4c8bd031d1d8bf5eb440b34e,1,1%5D

I'm not seeing anything in the commits around that time that would cause this to happen:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=e777cf9e040f&tochange=331ca1a40ca24f3f8dfeece611f05754e2735a5e

This event generated an alert which we marked as "wontfix" with no bug annotation or anything:

https://treeherder.mozilla.org/perf.html#/alerts?id=4453

I only noticed this because bdahl pointed out the weirdness on irc: http://logs.glob.uno/?c=mozilla%23perfherder#c61

jmaher, do you have any clue what happened here?

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

I see all of the counters are showing this behaviour, it is as if we are recording proper tp5 counters, but when running in e10s, we post the data for the counters (not the test) as non-e10s :(

Looking at the commit range, this seemed to happen abruptly, and the perfherder_data blob appears the same for the old working ones vs the new failing ones.

per irc conversation, bug 1318474 seems to be what changed in perfherder- the solution is to either revert that change in perfherder, or adjust talos to meet the more updated futuristic data model.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

we should add the extraOptions field to all tests/counters/data, not just a field in the header.  This would be a change in talos, in this general area:
http://searchfox.org/mozilla-central/rev/bf98cd4315b5efa1b28831001ad27d54df7bbb68/testing/talos/talos/output.py#66

Comment 3

[William Lachance (:wlach)]

I'll take this, per jmaher's request

Comment 4

[William Lachance (:wlach)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c5da91e6e13e

Comment 5

[William Lachance (:wlach)]

Attachment: Bug 1333860 - Fix setting e10s property on counter subtests

Review commit: https://reviewboard.mozilla.org/r/107528/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/107528/

Comment 6

[William Lachance (:wlach)]

(In reply to William Lachance (:wlach) from comment #4)
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=c5da91e6e13e

Try run worked out nicely, publishing review request.

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8830817 [details]
Bug 1333860 - Fix setting e10s property on counter subtests

https://reviewboard.mozilla.org/r/107528/#review108658

thanks for fixing this!

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0702ce2e636a
Fix setting e10s property on counter subtests r=jmaher

Comment 9

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/0702ce2e636a