Closed Bug 1827415 Opened 2 years ago Closed 2 years ago

Re enable failing benchmark tests on Beta/Release branches on Linux platform

Categories

(Testing :: Raptor, defect, P1)

Product:
Testing
Component:
Raptor
Version:
Default
Type:
defect

Tracking

(firefox114 fixed, firefox115 fixed)

Status:
RESOLVED FIXED
Milestone:
115 Branch
Tracking Flags:
Tracking Status
firefox114 --- fixed
firefox115 --- fixed

People

(Reporter: kshampur, Assigned: kshampur)

References

Details

(Whiteboard: [fxp])

Attachments

(1 file, 1 obsolete file)

Bug 1827415 - Re-enable raptor benchmark tests on Beta branch for Linux and MacOS. r?#perftest
48 bytes, patch
dmeehan
: approval-mozilla-beta+
Details | Diff | Splinter Review
Whiteboard: [fxp]

If there is a problem with geckodriver to launch Firefox you want enable the trace log level in CI eg. using the -vv argument for geckodriver or setting the Firefox preference remote.log.level=Trace. This should give helpful information for the investigation.

worth making note we were unable to reproduce this locally.
Sparky attempted it on his Linux machine using the Beta branch artifacts from the failing job

I am unable to reproduce this with beta simulations (tried late, early, and beta as release)

Maybe I did something wrong? I checked out the top revision in the original failing push i.e. 67a8f58a9d483b8073ee3acc587195c7040905e5, and set the version to 113.0 when doing any of the sims above.

:dmeehan I am wondering, could it be possible to re enable the tests and then uplift to beta to see if the issue still exists?

Flags: needinfo?(dmeehan)

Switching the needinfo to RyanVM (back from PTO and relman for 113)

Flags: needinfo?(dmeehan) → needinfo?(ryanvm)

Aryx, did we see these failures in the Beta simulations for 113 prior to the merge? Not entirely sure how informative comment 3 is for telling us that the root problem went away.

Flags: needinfo?(ryanvm) → needinfo?(aryx.bugmail)

We don't run performance tasks on central-as-beta simulations by default. The Try repository where these run let me add the task to the simulation of March 24th as the oldest and the task also failed for that.

I will bisect this with new central-as-beta simulations which use the old code.

Flags: needinfo?(aryx.bugmail)
See Also: → 1829939

Hi :aerickson,
(with regards to Bug 1827236)

Would you know if there was any linux infra changes (in beta) between April 3rd and april 10th?

Flags: needinfo?(aerickson)

:kshampur, there weren't any changes to the linux hardware infrastructure (releng-hardware/gecko-t-linux-talos-1804 workers) during that period.

Flags: needinfo?(aerickson)

thanks for confirming!

I am seeing passes so I'll put up a patch shortly

Assignee: nobody → kshampur
Status: NEW → ASSIGNED
Priority: P2 → P1

It seems that the beta branch failures for the raptor browsertime benchmark tests on the Linux and MacOS platforms are working again.
It is currently unknown why these tests were perma failing at the time. Bug 1832290 has been filed to look into such related issues.

Pushed by kshampur@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/ed3a487deaf2 Re-enable raptor benchmark tests on Beta branch for Linux and MacOS. r=perftest-reviewers,afinder

https://hg.mozilla.org/mozilla-central/rev/ed3a487deaf2

Status: ASSIGNED → RESOLVED
Closed: 2 years ago
status-firefox115: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 115 Branch
Attached file beta uplift (obsolete) —

Approval Request Comment
[Feature/Bug causing the regression]:
[User impact if declined]: none
[Is this code covered by automated tests?]:yes
[Has the fix been verified in Nightly?]: nightly was not affected just beta
[Needs manual test from QE? If yes, steps to reproduce]: no
[List of other uplifts needed for the feature/fix]: n/a
[Is the change risky?]: no/low
[Why is the change risky/not risky?]: only affects perf tests, does not affect user
[String changes made/needed]: none

Attachment #9333224 - Flags: approval-mozilla-beta?

Kash, the patch you asked to uplift to beta does not contain a patch.

Flags: needinfo?(kshampur)

Comment on attachment 9332929 [details] [diff] [review]
Bug 1827415 - Re-enable raptor benchmark tests on Beta branch for Linux and MacOS. r?#perftest

Beta/Release Uplift Approval Request

  • User impact if declined: none
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): only affects perf tests, does not affect user
  • String changes made/needed: none
  • Is Android affected?: No
Flags: needinfo?(kshampur)
Attachment #9332929 - Flags: approval-mozilla-beta?

hi Pascal, sorry been a while since i've requested uplift. I am not sure if this reattempt is correct. I just want to uplift the only patch landed for this e.g. https://phabricator.services.mozilla.com/D177674

Flags: needinfo?(pascalc)

Comment on attachment 9332929 [details] [diff] [review]
Bug 1827415 - Re-enable raptor benchmark tests on Beta branch for Linux and MacOS. r?#perftest

https://phabricator.services.mozilla.com/D177674

Comment on attachment 9332929 [details] [diff] [review]
Bug 1827415 - Re-enable raptor benchmark tests on Beta branch for Linux and MacOS. r?#perftest

Beta/Release Uplift Approval Request

  • User impact if declined: none
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): only affects perf tests, does not affect user
  • String changes made/needed:
  • Is Android affected?: No
Attachment #9332929 - Attachment is patch: true
Attachment #9332929 - Attachment mime type: text/x-phabricator-request → text/plain

sorry for the spam, think i figured it out

Attachment #9333224 - Attachment is obsolete: true
Attachment #9333224 - Flags: approval-mozilla-beta?

Comment on attachment 9332929 [details] [diff] [review]
Bug 1827415 - Re-enable raptor benchmark tests on Beta branch for Linux and MacOS. r?#perftest

Approved for 114.0b6.

Attachment #9332929 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

Thanks dmeehan!

Flags: needinfo?(pascalc)
Blocks: 1835299
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: