1490064 - Investigate similar variability groupings to search for timers

Status: NEW

(Testing :: Code Coverage, enhancement)

Description

[Greg Mierzwinski [:sparky]]

Attachment: similarity_groupings_1.zip

I found a way to divide all files with similar variability into groups, and these can be found in the attachment.

Each JSON file is it's own group and within each JSON you can find the grouped source files, and their coverage across all runs. The ordering of these lists is constant across all files. I thought it might be easier for us to find any remaining timers if we divide up the variability based on similarity over time. 

The data was obtained from this push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ef71da0a70daeb8fef3382c1d63de6bb0e5a3713
It unions (instead of differences) the test coverage with the baseline coverage so we can look at both simulatenously.

From this zip file, I found the following so far from this JSON: similarity_similar-group-35_1536530873.json

This file: [1] https://dxr.mozilla.org/mozilla-central/source/obj-x86_64-pc-linux-gnu/dom/bindings/RegisterWorkerBindings.cpp#99
is called by: https://dxr.mozilla.org/mozilla-central/source/dom/workers/WorkerPrivate.cpp#5283 (a get or create function)
which is called by: https://dxr.mozilla.org/mozilla-central/source/dom/workers/WorkerRunnable.cpp#268
which could be called by either of these: https://dxr.mozilla.org/mozilla-central/search?q=%2Bcallers%3Amozilla%3A%3Adom%3A%3AWorkerRunnable%3A%3ARun%28%29
Now, I can't find which it is so it may be something that DXR can't find, or we lost coverage data during generation (most likely).
There is a lot of coverage related to this [1] file, and most of it is in '*Bindings*' files.

In this JSON, I think there are few sources of variability, with `gfx/layers/client/ClientImageLayer.cpp` being part of the second.
It is, after a long chain, likely used by: [2] https://dxr.mozilla.org/mozilla-central/source/gfx/layers/ImageContainer.cpp#824
But this is always called by (in all runs): [3] https://dxr.mozilla.org/mozilla-central/source/gfx/layers/client/ImageClient.cpp#202
This might be because of an intermittent bug! And it could be caused by backend being intermittently `None` at [2].

The third source of variability has something to do with `ImageClient.cpp` [3].
It's new coverage contains a call to remove something: https://dxr.mozilla.org/mozilla-central/source/gfx/layers/client/ImageClient.cpp#68
which is called by: https://dxr.mozilla.org/mozilla-central/source/gfx/layers/client/ImageClient.cpp#256
and this gets called by (after going back through `ClientImageLayer.cpp`) a render: https://dxr.mozilla.org/mozilla-central/source/gfx/layers/composite/ImageLayerComposite.cpp#85 

I haven't yet determined how these are related, and I have yet to find a timer for them. Currently, it seems like this variability is unavoidable. But there are about 38 groupings to look through so one of them may contain something interesting for us.

Comment 1

[Greg Mierzwinski [:sparky]]

Attachment: similarity_similar-group-35_1536530873.json

This is the file I analyzed in the bug description.

Also, this analysis can be seen here: https://github.com/gmierz/coco-tools/blob/master/utils/cocoanalyze/analysis_types.py#L145-L219

In summary, I normalize the data to the range [0, 1], then I group all files which have minimized differences with the maximum difference allowed being user defined through `correlation_thresold`.

Comment 2

[Greg Mierzwinski [:sparky]]

(From the zip file from the description)

In file: similarity_similar-group-291_1536530873.json
I see that this file is used intermittently, and this line is also being hit: https://dxr.mozilla.org/mozilla-central/source/js/xpconnect/loader/URLPreloader.cpp#199
So, it seems like the variability is also coming from caching, and caching issues based on this.

In this file: similarity_similar-group-2441_1536530873.json
A telemetry file is used intermittently: https://dxr.mozilla.org/mozilla-central/source/storage/TelemetryVFS.cpp