1978731 - 5% Regression on Jetstream3-8bitbench (Windows) and 5% on gcc-loops (Andoid) on 21jul2025

Status: RESOLVED WONTFIX

(Core :: JavaScript: WebAssembly, defect, P2)

Description

[Mayank Bansal]

See this for Windows:
https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&replicates=0&selected=5314873,2078170340&series=autoland,5314873,1,13&timerange=5184000&zoom=1753109887954,1753112383528,139.31269370153822,150.7785942728531

Andoid GCC-loops: https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&replicates=0&selected=5441253,2077878265&series=autoland,5441253,1,13&timerange=5184000&zoom=1753017567434,1753224930844,32.64797607720272,36.108639180534965

Changelog points to Bug 1970035

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1970035

:jseward, since you are the author of the regressor, bug 1970035, could you take a look? Also, could you set the severity field?

For more information, please visit BugBot documentation.

Comment 2

[Mayank Bansal]

The regression on Android is weird... maybe the perf-tsting setup is reporting incorrect values, or classifying the higher numbers as a regression ?

Comment 3

[Mayank Bansal]

FWIW, the Windows numbers returned to baseline.
What i dont know yet is f the regression (and the return to baseline) was some sort of glitch, or whethe the improvements are separate and due to bug 1962563 / Bug 1978600.

Comment 4

[Ben Visness [:bvisness]]

8bitbench wouldn't be impacted by bug 1962563, but could be impacted by bug 1978600.

Comment 5

[Yury Delendik (:yury)]

(In reply to Mayank Bansal from comment #3)

FWIW, the Windows numbers returned to baseline.
What i dont know yet is f the regression (and the return to baseline) was some sort of glitch, or whethe the improvements are separate and due to bug 1962563 / Bug 1978600.

Is there more specific commit? The bug 1978600 affects only X86 architectures -- it is strange to see "5% on gcc-loops (Andoid)". Is there a regression on Linux?

Comment 6

[Mayank Bansal]

(In reply to Yury Delendik (:yury) from comment #5)

Is there more specific commit? The bug 1978600 affects only X86 architectures -- it is strange to see "5% on gcc-loops (Andoid)". Is there a regression on Linux?

This is the more refined changelog : https://hg-edge.mozilla.org/integration/autoland/pushloghtml?fromchange=ce0ee8e87702d0c470573542f420bc5b5c97878d&tochange=9c318e31a4ce16547b6bd1e2b1edc49c42d92ab7 , in which bug 1978600 seems the most likely.

Comment 7

[Yury Delendik (:yury)]

(In reply to Mayank Bansal from comment #3)

FWIW, the Windows numbers returned to baseline.
What i dont know yet is f the regression (and the return to baseline) was some sort of glitch, or whethe the improvements are separate and due to bug 1962563 / Bug 1978600.

I don't see anything obvious with 8bitbench that points to regression/improvement from the backout of my patch, see comparison per request from bug 1978998 comment 8 -- too much noise (base is with the 1978600 change, new is without this change).

Comment 8

[Julian Seward [:jseward]]

The gcc-loops regression on Android is real and is handled in bug 1979829.

However I cannot reproduce the 8bitbench regressions (on x86_64-Linux) locally,
running in the shell. I suspect this is some artefact of the measurement
system. I have the following numbers for 8bitbench-wasm on Intel Tiger Lake
(best of 10 runs):

before EAA patch (Bug 1970035 - part 2 ..)
    Average:      23.933
    Total Score:  23.596 
    51,991,822,903   instructions:u    #    3.92  insn per cycle
    13,284,481,298   cycles:u          #    4.166 GHz 

after EAA patch
    Average:      25.434
    Total Score:  25.009 
    52,061,732,831   instructions:u    #    4.16  insn per cycle
    12,498,497,798   cycles:u          #    4.164 GHz

The patch gives a 0.13% instruction count regression, however it runs in 9.6%
fewer cycles and so the Average and Total Score numbers improve accordingly.

Subsequent changes to m-c (not sure which changes) push the instruction count
below 51 billion:

m-c of 29 July 2025:
    Average:      25.957
    Total Score:  25.543 
    50,892,312,838   instructions:u    #    4.15  insn per cycle
    12,245,606,250   cycles:u          #    4.166 GHz                       

I propose not to chase this any further.