Bugzilla

(In reply to Julien Cristau [:jcristau] from comment #6)
> I'm trying to understand the status here, looking for help on the following:
> - is this code riding the trains or is it preffed off somehow?
The regressing "ParserAtoms" patch is currently riding trains and there isn't a pref for it.
> - did bug 1664312 and/or bug 1664189 make a difference here, and if yes what's the remaining gap?  I see 1664312 itself triggered a new alert and that is planned to be addressed in bug 1666274
Some of the regressions such as netflix load are down to < 2%.
Android data is too noise to tell one way or another. I can try to simulate changes with artificial stacks that only have relevant changes and ignore anything else in 82 (which seem to have affected the numbers but not enough to trigger alerts). This is probably about a 1000 raptor jobs on the android hardware to get through the noise and look at the various test pages so it would likely take a few to get results.
> - what is the plan for 82 now that is on beta?
I'm not sure what the least worst thing is here. It isn't really feasible to revert at this point (12 bugs would need to go, and another 5-10 would need merge conflicts resolved).
Regressions:
 (A) Hard to tell from data what current impact is. The ebay one is most concerning but I can't tell from the graphs if Bug 1664312 helped or it is just noise.
 (B) This seems like an unlucky test case hitting bad luck. It only appears on a single platform and the underlying results between retries have 4x variation. I'm not sure there is anything to do for 82. In 83 the results will bounce back (and early data suggests be even better).
 (C) This is helped a lot by Bug 1664312 and the graph shows we are now better then before.
 (D,E) These are noisy and may have been helped a little by Bug 1664312. I don't think there is much to do in 82. Further tuning in 83 will happen.
 (F) There is a possible uplift of medium-complexity that will get us back the 150kB base content overhead. Since fission is not shipping by default yet, I'm leaning towards not uplifting at this point. We can re-evaluate at end of this week after it has baked on nightly though.

(In reply to Julien Cristau [:jcristau] from comment #6)
> I'm trying to understand the status here, looking for help on the following:
> - is this code riding the trains or is it preffed off somehow?

The regressing "ParserAtoms" patch is currently riding trains and there isn't a pref for it.

> - did bug 1664312 and/or bug 1664189 make a difference here, and if yes what's the remaining gap?  I see 1664312 itself triggered a new alert and that is planned to be addressed in bug 1666274

Some of the regressions such as netflix load are down to < 2%.
Android data is too noise to tell one way or another. I can try to simulate changes with artificial stacks that only have relevant changes and ignore anything else in 82 (which seem to have affected the numbers but not enough to trigger alerts). This is probably about a 1000 raptor jobs on the android hardware to get through the noise and look at the various test pages so it would likely take a few to get results.

> - what is the plan for 82 now that is on beta?

I'm not sure what the least worst thing is here. It isn't really feasible to revert at this point (12 bugs would need to go, and another 5-10 would need merge conflicts resolved).
Regressions:
 (A) Hard to tell from data what current impact is. The ebay one is most concerning but I can't tell from the graphs if Bug 1664312 helped or it is just noise.
 (B) This seems like an unlucky test case hitting bad luck. It only appears on a single platform and the underlying results between retries have 4x variation. I'm not sure there is anything to do for 82. In 83 the results will bounce back (and early data suggests be even better).
 (C) This is helped a lot by Bug 1664312 and the graph shows we are now better then before.
 (D,E) These are noisy and may have been helped a little by Bug 1664312. I don't think there is much to do in 82. Further tuning in 83 will happen.
 (F) There is a possible uplift of medium-complexity that will get us back the 150kB base content overhead. Since fission is not shipping by default yet, I'm leaning towards not uplifting at this point. We can re-evaluate at end of this week after it has baked on nightly though.