1397751 - (damp) [meta] Fix and improve devtools performance regression CI (aka DAMP)

Status: NEW

(DevTools :: General, defect, P3)

Description

[Alexandre Poirot [:ochameau]]

Over the last year, I kept working on various performance improvements and I've seen performance regress as I fixed things.
It became pretty clear that DAMP, in its current state, miss many and significant regressions.

A good example was bug 1359855. In order to prevent loading DevTools codebase at Firefox startup, it has been delayed to the first time we interact with the tools.
The piece of code that has been moved is significant.
So significant that sessionrestore talos test reported a win for firefox startup when we moved it to later.
But DAMP should have been reporting the same amount as a regression for when we open the first tool. But it did not.

This precise scenario is most likely related to bug 1394804.

But that one bug can't explain everything.

The fact that DAMP only tracks the summary value is another significant issue (bug 1397745).

Also, we have to review each panel test to ensure each OPEN/RELOAD/CLOSE steps are correctly tracked and that there is no pending computation that overlap between each test. (like bug 1396539, specific to the inspector).

I imagine there is some other improvements to be made to DAMP. Let's use this bug as a meta bug for improving DAMP.