1551122 - Performance regression when running JS benchmark

Status: RESOLVED INACTIVE

(Core :: Performance, defect, P3)

Description

[can cao]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36

Steps to reproduce:

Run JS benchmark(Speedmeter2.0, RoboHornet) on Firefox 66.0.5 and Firefox 65.0.2

Actual results:

Performance regression, test result please check the below:

            Speedometer 2.0 (score)               Speedometer 2.0 (score)

hardware i7_8700K_HD630 i7_8700K_HD630
Browser Firefox 65.0.2 Firefox 66.0.5
Linux OS 101.8 88.1
Windows OS 102 96.33

Expected results:

Test result should be no obvious change.

Comment 1

[Randell Jesup [:jesup] (needinfo me)]

I've tried reproducing with 65.0.2 and 66.0.5 for linux x86_64 from http://archive.mozilla.org/pub/firefox/releases/

Fedora 29, core i9, lots of ram. In multiple runs, using the same profile, I see a small advantage to 66 over 65:

66.0.5: 65.8 +- 0.89 65.0.2: 63.1 +- 1.3
after stopping something:
66.0.5: 69.9 +- 1.6 65.0.2: 66.7 +- 1.0

This is with https://browserbench.org/Speedometer2.0/

Can you try fresh profiles, and let them "bake" for a few minutes after browsing to example.com, before loading speedometer?
No extensions added?

Thanks!

Comment 2

[can cao]

Attachment: firefox-65.png

Comment 3

[can cao]

Attachment: firefox-66.png

Comment 4

[can cao]

Attachment: Device-info.png

Comment 5

[can cao]

I got three screenshots for test result of Firefox and Device info. Please take a look.

Note:
Clear all the history,uninstall the firefox-66, reboot, install the firefox-65, wait a few minutes then launch browser and run speedometer2.0.

Comment 6

[Randell Jesup [:jesup] (needinfo me)]

Where are these 66 and 65's coming from? are they installed from official mozilla builds (which you can get at archive.mozilla.org/pub/firefox/releases? Can you attach about:support to this bug, or the data in about:buildconfig for each? Many distros on linux do their own builds, with options they choose, so that can vary a lot from what we ship.

Also: Note that clearing history isn't the same as a fresh profile. See about:profiles

Thanks again!

Comment 7

[can cao]

Attachment: firefox-65-support.txt

Comment 8

[can cao]

Attachment: firefox-66-support.txt

Comment 9

[can cao]

Attachment: firefox-65-buildconfig.txt

Comment 10

[can cao]

Attachment: firefox-66-buildconfig.txt

Comment 11

[can cao]

Download Link: https://archive.mozilla.org/pub/firefox/releases/65.0.2/linux-x86_64/en-US/
https://archive.mozilla.org/pub/firefox/releases/66.0.5/linux-x86_64/en-US/

About:support: please refer to the Firefox-65-support.txt and Firefox-66-support.txt (comment 7,comment 8)

About:buildconfig Please refer to the Firefox-65-buildconfig.txt and Firefox-66-buildconfig.txt (comment 9, comment 10)

refresh about:profiles

Issue always exist.

Comment 12

[Randell Jesup [:jesup] (needinfo me)]

Can we get profiles of 65 and 66, using profiler.firefox.com? Install the extension, then add "JS" to the list of threads (see Advanced in the dropdown). Record a profile (just of a section of the run; a full run is too long - maybe 30 seconds). ctrl-shift-2 to capture, then publish and put the link here. Thanks again!

Comment 13

[Dave Hunt [:davehunt] [he/him] ⌚BST]

With no answer from the reporter, we don’t have enough data to reproduce and/or fix this issue. Please reopen or file a new bug with more information if you see it again.