1383637 - tp5o summary pgo e10s - 4.0% regression windows10-64 (Tue July 18 2017)

Status: RESOLVED WONTFIX

(Firefox :: General, defect)

Description

[Kevin Brosnan [Ex-Mozilla]]

https://treeherder.mozilla.org/perf.html#/graphs?series=%5Bmozilla-central,c00763b23b39207671b795a12ba29d38ddc17f06,1,1%5D&series=%5Bmozilla-central,ac46ba40f08bbbf209a6c34b8c054393bf222e67,0,1%5D&series=%5Bautoland,c00763b23b39207671b795a12ba29d38ddc17f06,1,1%5D&selected=%5Bautoland,c00763b23b39207671b795a12ba29d38ddc17f06,228271,298877279,1%5D

My week over week comparison of tp5o summary pgo e10s on Windows 10-64 shows around a 4% regression in the benchmark. The last good push is 18aeda1e84fb on Autoland Mon Jul 17, 17:44:53. Looks like something on the 18th shifted the low values from 233 to 238. Looks similar to 1381804.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

we appear to run tp5 very irregularly on win10, so I did retriggers to get the investigation started:
https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&filter-searchStr=tp5%20win%20x64&tochange=a1ea69a3a0a760036e047f6ff89897aa7ffd4e53&fromchange=b9031e50fd764e92d10055bc97a916a0e8e7deab&selectedJob=115241714

I assume this will take a few round to retrigger more, backfill, etc.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

there is a clear bump here:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=2fb0f93b7b5bc9880f57534673f0e27c6b5a789e&tochange=e8e5ca57b442fd63555e5e17ddcf96a339a0a6d1

it is <1% though, not really that easy to track, :kbrosnan, do you see other areas on the graphs that we should be investigating:
https://treeherder.mozilla.org/perf.html#/graphs?timerange=2592000&series=%5Bautoland,c00763b23b39207671b795a12ba29d38ddc17f06,1,1%5D&series=%5Bautoland,c02f26c0202fbfa009a40190f610bf4257c557a5,1,1%5D

Comment 3

[Panos Astithas (he/him) [:past] (please ni?)]

Hey Dão, any thoughts on how bug 1367630 might be affecting tp5o?

Comment 4

[Dão Gottwald [:dao]]

(In reply to Panos Astithas [:past] (56 Regression Engineering Owner) (please ni?) from comment #3)
> Hey Dão, any thoughts on how bug 1367630 might be affecting tp5o?

We're doing slightly more work, see bug 1383873 comment 2. A <1% bump seems reasonable. Can we wontfix this?

Comment 5

[Ryan VanderMeulen [:RyanVM]]

Agreed that this looks pretty much unactionable.