Bugzilla

I will be continuing to look at this but, from what I can tell, there _is_ a regression from my commit on the pretty-print tests on macOS, but it is not nearly as significant as shown in the alert above.

I used _./mach perf --alert 41942 --rebuild 20_ on both my commit, and the commit right before it, to run these tests again in CI. Because I don't know what I'm doing, that gave me two comparison reports - one [comparing my commit against itself](https://treeherder.mozilla.org/perfherder/comparesubtest?originalProject=try&newProject=try&newRevision=f79001ef195c9928c25a9d9d9d03e775c0d9fa72&originalSignature=4765722&newSignature=4765722&framework=12&application=firefox&originalRevision=28146d83527d22e112a2621357d18107bcfe65e3&page=1&filter=pretty&replicates=1) and another [doing the same for the previous commit](https://treeherder.mozilla.org/perfherder/comparesubtest?originalProject=try&newProject=try&newRevision=9df7a5191e7bda0e013274f6a40cbebaf47c44c0&originalSignature=4765722&newSignature=4765722&framework=12&application=firefox&originalRevision=42f1905f7cd297928f5d240ac9d27190f2cbfb37&page=1&filter=pretty&replicates=1).

If you look _between_ those two comparison reports, it looks like both of those tests went from ~8500ms to ~8800ms, which is about a 3.5% regression.

I will be continuing to look at this but, from what I can tell, there _is_ a regression from my commit on the pretty-print tests on macOS, but it is not nearly as significant as shown in the alert above.

I used _./mach perf --alert 41942 --rebuild 20_ on both my commit, and the commit right before it, to run these tests again in CI. Because I don't know what I'm doing, that gave me two comparison reports - one [comparing my commit against itself](https://treeherder.mozilla.org/perfherder/comparesubtest?originalProject=try&newProject=try&newRevision=f79001ef195c9928c25a9d9d9d03e775c0d9fa72&originalSignature=4765722&newSignature=4765722&framework=12&application=firefox&originalRevision=28146d83527d22e112a2621357d18107bcfe65e3&page=1&filter=pretty&replicates=1) and another [doing the same for the previous commit](https://treeherder.mozilla.org/perfherder/comparesubtest?originalProject=try&newProject=try&newRevision=9df7a5191e7bda0e013274f6a40cbebaf47c44c0&originalSignature=4765722&newSignature=4765722&framework=12&application=firefox&originalRevision=42f1905f7cd297928f5d240ac9d27190f2cbfb37&page=1&filter=pretty&replicates=1).

If you look _between_ those two comparison reports, it looks like both of those tests went from ~8500ms to ~8800ms, which is about a 3.5% regression.

The graphs from all of those runs are still tri-modal and visually they look almost identical. I'm not sure that that difference is even really related to the change or just to the randomness of the modality.