Open Bug 1936814 Opened 2 months ago Updated 1 month ago

15.12 - 12.54% browser_ml_engine_perf.js EXAMPLE-cold-start-pipeline-ready-latency / browser_ml_engine_multi_perf.js intent-initialization-latency + 9 more (Linux) regression on Wed December 4 2024

Categories

(Core :: Permission Manager, defect, P2)

defect

Tracking

()

ASSIGNED
Tracking Status
firefox-esr128 --- unaffected
firefox133 --- unaffected
firefox134 --- unaffected
firefox135 --- fix-optional

People

(Reporter: intermittent-bug-filer, Assigned: maltejur)

References

(Regression)

Details

(Keywords: perf, perf-alert, regression)

Perfherder has detected a mozperftest performance regression from push e1c5bea6ace463fe543feffeaa26d3d51ceff0bb. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio Test Platform Options Absolute values (old vs new)
15% browser_ml_engine_perf.js EXAMPLE-cold-start-pipeline-ready-latency linux1804-64-shippable 7,189.12 -> 8,275.92
15% browser_ml_engine_perf.js EXAMPLE-cold-start-initialization-latency linux1804-64-shippable 7,206.12 -> 8,293.25
13% browser_ml_engine_multi_perf.js intent-pipeline-ready-latency linux1804-64-shippable 18,970.42 -> 21,519.08
13% browser_ml_engine_multi_perf.js intent-initialization-latency linux1804-64-shippable 18,991.75 -> 21,538.67
13% browser_ml_engine_multi_perf.js engine4-initialization-latency linux1804-64-shippable 17,551.17 -> 19,867.33
13% browser_ml_engine_multi_perf.js engine4-pipeline-ready-latency linux1804-64-shippable 17,530.00 -> 19,842.25
13% browser_ml_engine_multi_perf.js suggest-initialization-latency linux1804-64-shippable 21,416.75 -> 24,200.00
13% browser_ml_engine_multi_perf.js suggest-pipeline-ready-latency linux1804-64-shippable 21,396.88 -> 24,179.17
13% browser_ml_engine_multi_perf.js engine3-pipeline-ready-latency linux1804-64-shippable 17,528.25 -> 19,759.83
13% browser_ml_engine_multi_perf.js engine3-initialization-latency linux1804-64-shippable 17,549.83 -> 19,781.83
13% browser_ml_engine_multi_perf.js intent-initialization-latency linux1804-64-shippable 19,024.79 -> 21,409.58

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run all of these tests on try with ./mach try perf --alert 43070

The following documentation link provides more information about this command.

For more information on performance sheriffing please see our FAQ.

If you have any questions, please do not hesitate to reach out to aglavic@mozilla.com.

Flags: needinfo?(maltejur)

Set release status flags based on info from the regressing bug 1919558

I am still looking into what is causing this, I suspect it has something to do with the remote settings client. So far, I can only confirm though that backing out my changes would improve the performance values in browser_ml_engine_* again:

https://perf.compare/compare-results?baseRev=952e56d73f931e54e5d0e884bb489f1f48bc4810&newRev=b6c575f570c348d6479b199599c985ac7eb26178&baseRepo=try&newRepo=try&framework=15

As my stack of regressing patches is somewhat larger, and this only seems to affect the performance of the still experimental ML engine, I would prefer if my patches don't get backed out until the end of the week. I will try to figure out if there is any way to mitigate this performance impact until then.

Assignee: nobody → maltejur
Status: NEW → ASSIGNED
Flags: needinfo?(maltejur)

The severity field is not set for this bug.
:timhuang, could you have a look please?

For more information, please visit BugBot documentation.

Flags: needinfo?(tihuang)
Severity: -- → S3
Flags: needinfo?(tihuang)
Priority: -- → P2
You need to log in before you can comment on or make changes to this bug.