Closed Bug 1929248 Opened 4 months ago Closed 3 months ago

Add a way to monitor performance regressions that were recently changed to FIXED/WONTFIX

Categories

(Tree Management :: Perfherder, task, P2)

Product:
Tree Management
Component:
Perfherder
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: sparky, Unassigned)

References

(Blocks 1 open bug)

Details

(Whiteboard: [fxp])

Currently, we have no standard way of monitoring performance regressions which have recently been resolved as WONTFIX/FIXED. We should add a bugbot rule to send emails that summarize recent changes to perf-alert bugs.

Blocks: 1929249

I've made a PR here for a new bugbot rule: https://github.com/mozilla/bugbot/pull/2535

You can see what the emails will look like with the following HTML code. Note that it also contains the resolution comment that was made with the resolution change (if available):

<!DOCTYPE html>
<html lang="en-us">
    <head>
        <meta charset="utf-8">
    </head>
    <body>
        <p>Hi there,</p>
        
        <p>The following bug list contains performance alerts whose resolution has changed in the last week:</p>
<table style="border:1px solid black;border-collapse:collapse;" border="1">
    <thead>
        <tr>
            <th>Bug</th>
            <th>Summary</th>
            <th>Resolution</th>
        </tr>
    </thead>
    <tbody>
        <tr bgcolor="#E0E0E0"
            >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1929201">1929201</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>25.51 - 3.21% speedometer3 TodoMVC-Preact-Complex-DOM/CompletingAllItems/Async / speedometer3 wallclock-for-tracking-only + 9 more (Linux, OSX) regression on Wed October 23 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> I think given the regression/progression magnitude, and that the patch is a known performance trade-off (not strict improvement per se), we should not worry too much about those two regressions.
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by emilio@crisal.io</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
        </tr>
    <tr >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1928943">1928943</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>34316.95 - -3932% browser-console:parent-process memory / browser-console:parent-process memory (Linux) regression on Tue October 29 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> https://hg.mozilla.org/mozilla-central/rev/89128774a49c
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by smolnar@mozilla.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
        </tr>
    <tr bgcolor="#E0E0E0"
            >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1928089">1928089</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>1.22% Base Content Heap Unclassified (Linux) regression on Wed October 23 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> No resolution comment provided.
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by rjesup@jesup.org</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
        </tr>
    <tr >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1927104">1927104</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>627.86 - 2.16% pdfpaint tiling-pattern-box.pdf / pdfpaint issue14990.pdf + 24 more (Windows) regression on Thu October 17 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> (In reply to Donal Meehan [:dmeehan] from comment #8)
> :calixte/:lsalman any updates on this?
> We are tracking this for Fx133 but next week is the final week of beta. 
> What's the next step here?

The issue was resolved between both bug 1926668, and further in bug 1928106 for 134, but it's up to Calixte whether bug 1928106 is worth uplifting to 133.

Otherwise, I am okay with this simply being wontfix/fix-optional for 133, as it's expected to be a mixed bag of improvements and regressions on a testsuite as wide as pdfpaint.
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by lsalzman@mozilla.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
        </tr>
    <tr bgcolor="#E0E0E0"
            >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1925081">1925081</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>4.31% pdfpaint issue9534_reduced.pdf (Linux) regression on Mon October 7 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> Spoke with :rebeca after looking at the regressing android patch and the linux platform where the regression was noted I am closing this as an invalid bug, apologies for all the confusion!
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by aglavic@mozilla.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
        </tr>
    </tbody>
</table>
        <p>
            Sincerely,
            <br>
            BugBot
        </p>
        <p>
                See the search query on <a href="https://mzl.la/3CdrbDk">Bugzilla</a>.
            </p>
        
        <p>
            See the source / report issues on the <a href="https://github.com/mozilla/bugbot/">GitHub repo</a>
        </p>
    </body>
</html>

This has been resolved now, and we're receiving the emails in perfalerts-activity google group.

Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.