Closed Bug 1558489 Opened 8 months ago Closed 2 months ago

Performance regression in selenium benchmark

Categories

(Firefox :: Untriaged, defect)

67 Branch
defect
Not set

Tracking

()

RESOLVED INCOMPLETE

People

(Reporter: guangli.li, Unassigned, NeedInfo)

Details

(Keywords: regression, regressionwindow-wanted)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36

Steps to reproduce:

  1. install clear linux version: 29550
  2. install phoronix benchmark: sudo swupd bundle-add os-testsuite-phoronix
  3. setup phoronix config: using command "phoronix-test-suite network-setup" to setup network
  4. using command "DISPLAY=:0 phoronix-test-suite run selenium-1.0.6" to select Octance and firefox to get the performance data.

Actual results:

actual result:36116.0

Expected results:

about 38146.0

bisect of this problem:
index 1 9 12 13 16
version 29430 29510 29540 29550 29590
value 38537.0 38181.0 38146.0 36116.0 36244.0
flag good good good bad bad

from version 29540 to 29550, firefox version updated from 66.0.5 to 67. If I use firefox 66.0.5 to instead of 67 on 29550, the benchmark result is 38100. It is good as 29540. So please look at this issue.

If has any questions, please feel be free to let me know.

guangli.li, did you bisection found any regression? aka, did mozregression find the culpable bug responsible for this regression?

Flags: needinfo?(guangli.li)

(In reply to Adrian Florinescu [:adrian_sv] from comment #2)

guangli.li, did you bisection found any regression? aka, did mozregression find the culpable bug responsible for this regression?

Hi Adrian,
Sorry, I didn't do the further bisect.

Flags: needinfo?(guangli.li)

guangli.li, could you perhaps assist into further isolating an actionable regression range? (https://mozilla.github.io/mozregression/) It would be a great help since the environment to run this bisection is rather complicated to set up.

Flags: needinfo?(guangli.li)

guangli.li, can you please check comment 4? Please reopen this issue with a regression range, most helpful with latest Fx channels.

Status: UNCONFIRMED → RESOLVED
Closed: 2 months ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.