1268216 - tab paint on osx is so noisy it is pointless to run

Status: NEW

(Testing :: Talos, defect, P3)

Description

[Joel Maher ( :jmaher ) (UTC -0800)]

holy random number generator:
https://treeherder.mozilla.org/perf.html#/graphs?timerange=5184000&series=%5Bmozilla-aurora,cda01fa619e132de1147856a0b8ffd0e0891a4de,1%5D&series=%5Bmozilla-inbound,cda01fa619e132de1147856a0b8ffd0e0891a4de,1%5D&series=%5Bmozilla-central,cda01fa619e132de1147856a0b8ffd0e0891a4de,1%5D&series=%5Bfx-team,cda01fa619e132de1147856a0b8ffd0e0891a4de,1%5D

we should fix this test or turn it off on osx.

Comment 1

[Joel Maher ( :jmaher ) (UTC -0800)]

:mconley, I have you as the point of contact for this test, can you help us understand why this is a random number generator, or could you give me the ok to turn this off for osx, possibly just non-e10s.

Comment 2

[Mike Conley (:mconley) (:⚙️)]

I'll hopefully have time to look at this next week. Keeping ni?.

Comment 3

[Mike Conley (:mconley) (:⚙️)]

And by next week, I meant next month. Whoops.

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚GMT]

Attachment: Screenshot_2019-08-09 tabpaint opt e10s stylo macosx1014-64-shippable (mozilla-central) and others.png

The link in comment 0 is no longer valid, however looking at the recent tab paint results for macOS the data does appear to be bimodal. I don't know if this is the 'random number generator' issue that was originally reported, but perhaps there is still some investigation we could do here?

Comment 5

[Mike Conley (:mconley) (:⚙️)]

perhaps there is still some investigation we could do here?

Anything we can do to reduce the noise of these tests is useful. I apologize for having been assigned to this bug for so long without any movement. I'll remove myself for now.

For future spelunkers who might want to investigate this - one thing we might want to consider is if the 16ms vsync is a "cliff" that we're hitting here (where tabs paint one frame later, and so show up 16ms later), and that's what's causing the bi-modality here. We might want to check if ASAP modes helps reduce the noise here.