1870979 - Permafailing imgur [tier 2] [mozilla-release] raptor-browsertime Critical: Browsertime process timed out after waiting 120 seconds for output

Status: RESOLVED FIXED

(Testing :: Raptor, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: ncsoregi [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=440809246&repo=mozilla-release
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/MAatYzbVRJemdjkzk7k0hA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2023-12-20T03:31:12.822Z] 03:31:12     INFO -  raptor-browsertime Info: Testing url https://imgur.com/gallery/m5tYJL6 iteration 1
[task 2023-12-20T03:31:12.823Z] 03:31:12     INFO -  raptor-browsertime Info: Start firefox window recorder.
[task 2023-12-20T03:32:20.068Z] 03:32:20     INFO -  raptor-browsertime Info: Stop firefox window recorder.
[task 2023-12-20T03:34:20.202Z] 03:34:20 CRITICAL -  raptor-browsertime Critical: Browsertime process timed out after waiting 120 seconds for output
[task 2023-12-20T03:34:20.202Z] 03:34:20     INFO -  raptor-mitmproxy Info: MitmproxyDesktop stop!!
[task 2023-12-20T03:34:20.202Z] 03:34:20     INFO -  raptor-mitmproxy Info: Mitmproxy stop!!
[task 2023-12-20T03:34:20.202Z] 03:34:20     INFO -  raptor-mitmproxy Info: Stopping mitmproxy playback, killing process 1378
[task 2023-12-20T03:34:20.539Z] 03:34:20     INFO -  raptor-mitmproxy Info: Successfully killed the mitmproxy playback process
[task 2023-12-20T03:34:20.540Z] 03:34:20     INFO -  raptor-mitmproxy Info: Turning off the browser proxy
[task 2023-12-20T03:34:20.540Z] 03:34:20     INFO -  raptor-mitmproxy Info: writing: /opt/worker/tasks/task_170304282856711/build/application/Firefox.app/Contents/Resources/distribution/policies.json
[task 2023-12-20T03:34:20.540Z] 03:34:20     INFO -  raptor-perftest Info: Removing temporary directory: /var/folders/w1/j59dbgks1jsfshgv0w6ybx34000014/T/tmpbuyjxgma
[task 2023-12-20T03:34:20.565Z] 03:34:20    ERROR -  Traceback (most recent call last):
[task 2023-12-20T03:34:20.565Z] 03:34:20     INFO -    File "/opt/worker/tasks/task_170304282856711/build/tests/raptor/raptor/raptor.py", line 191, in <module>
[task 2023-12-20T03:34:20.565Z] 03:34:20     INFO -      main()
[task 2023-12-20T03:34:20.565Z] 03:34:20     INFO -    File "/opt/worker/tasks/task_170304282856711/build/tests/raptor/raptor/raptor.py", line 137, in main
[task 2023-12-20T03:34:20.566Z] 03:34:20     INFO -      success = raptor.run_tests(raptor_test_list, raptor_test_names)
[task 2023-12-20T03:34:20.566Z] 03:34:20     INFO -    File "/opt/worker/tasks/task_170304282856711/build/tests/raptor/raptor/perftest.py", line 467, in run_tests
[task 2023-12-20T03:34:20.566Z] 03:34:20     INFO -      self.run_test(test, timeout=int(test.get("page_timeout")))
[task 2023-12-20T03:34:20.566Z] 03:34:20     INFO -    File "/opt/worker/tasks/task_170304282856711/build/tests/raptor/raptor/browsertime/base.py", line 987, in run_test
[task 2023-12-20T03:34:20.566Z] 03:34:20     INFO -      f"Browsertime process timed out after waiting {output_timeout} seconds "
[task 2023-12-20T03:34:20.566Z] 03:34:20     INFO -  Exception: Browsertime process timed out after waiting 120 seconds for output
[task 2023-12-20T03:34:20.605Z] 03:34:20     INFO - Return code: 1
[task 2023-12-20T03:34:20.605Z] 03:34:20  WARNING - setting return code to 1
[task 2023-12-20T03:34:20.605Z] 03:34:20     INFO - Copying Raptor results to upload dir:
[task 2023-12-20T03:34:20.605Z] 03:34:20     INFO - /opt/worker/tasks/task_170304282856711/build/blobber_upload_dir/perfherder-data.json
[task 2023-12-20T03:34:20.605Z] 03:34:20     INFO - Copying raptor results from /opt/worker/tasks/task_170304282856711/build/raptor.json to /opt/worker/tasks/task_170304282856711/build/blobber_upload_dir/perfherder-data.json
[task 2023-12-20T03:34:20.605Z] 03:34:20 CRITICAL - Error copying results /opt/worker/tasks/task_170304282856711/build/raptor.json to upload dir /opt/worker/tasks/task_170304282856711/build/blobber_upload_dir/perfherder-data.json
[task 2023-12-20T03:34:20.605Z] 03:34:20     INFO - [Errno 2] No such file or directory: '/opt/worker/tasks/task_170304282856711/build/raptor.json'

Comment 1

[Natalia Csoregi [:nataliaCs]]

This is permafailing starting with: https://treeherder.mozilla.org/jobs?repo=mozilla-release&group_state=expanded&selectedTaskRun=MAatYzbVRJemdjkzk7k0hA.0&searchStr=os%2Cx%2C10.15%2Cwebrender%2Cshippable%2Copt%2Cbrowsertime%2Cperformance%2Ctests%2Con%2Cfirefox%2Ctest-macosx1015-64-shippable-qr%2Fopt-browsertime-tp6-firefox-imgur%2Cimgur&revision=88baa785c531cb093208bfc3bb6c55b9b554d5a6

Retriggering passed jobs on earlier pushes makes them fail: https://treeherder.mozilla.org/jobs?repo=mozilla-release&group_state=expanded&selectedTaskRun=MAatYzbVRJemdjkzk7k0hA.0&searchStr=os%2Cx%2C10.15%2Cwebrender%2Cshippable%2Copt%2Cbrowsertime%2Cperformance%2Ctests%2Con%2Cfirefox%2Ctest-macosx1015-64-shippable-qr%2Fopt-browsertime-tp6-firefox-imgur%2Cimgur&revision=a8daee71c97036d89232cdb16881b834448a2286

Greg, any chance you have a few spare minutes to check this?
Thank you!

Comment 2

[Intermittent Failures Robot]

13 failures in 3349 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 8
• autoland: 2
• mozilla-central: 2
• mozilla-beta: 1

Platform and build breakdown:

• macosx1015-64-shippable-qr: 8
  • opt: 8
• linux1804-64-shippable-qr: 5
  • opt: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2023-12-18&endday=2023-12-24&tree=all

Comment 3

[Greg Mierzwinski [:sparky]]

This should get resolved by the re-recording from bug 1863130, but I'll have another look next week.

Comment 4

[Intermittent Failures Robot]

10 failures in 1162 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• autoland: 4
• mozilla-beta: 1
• mozilla-release: 4

Platform and build breakdown:

• linux1804-64-clang-trunk-qr: 1
  • opt: 1
• linux1804-64-shippable-qr: 5
  • opt: 5
• macosx1015-64-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2023-12-25&endday=2023-12-31&tree=all

Comment 5

[Intermittent Failures Robot]

9 failures in 2274 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 9

Platform and build breakdown:

• macosx1015-64-shippable-qr: 5
  • opt: 5
• linux1804-64-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-01-01&endday=2024-01-07&tree=all

Comment 6

[Greg Mierzwinski [:sparky]]

We're still waiting on a merge to see if it gets fixed. Currently, the test is still using the old recording.

Comment 7

[Intermittent Failures Robot]

18 failures in 636 pushes (0.028 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-beta: 3
• autoland: 4
• mozilla-release: 11

Platform and build breakdown:

• android-hw-a51-11-0-aarch64-shippable-qr: 1
  • opt: 1
• linux1804-64-shippable-qr: 16
  • opt: 16
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-01-10&endday=2024-01-10&tree=all

Comment 8

[Intermittent Failures Robot]

28 failures in 3217 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 21
• mozilla-beta: 3
• autoland: 4

Platform and build breakdown:

• macosx1015-64-shippable-qr: 6
  • opt: 6
• linux1804-64-shippable-qr: 21
  • opt: 21
• android-hw-a51-11-0-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-01-08&endday=2024-01-14&tree=all

Comment 9

[Intermittent Failures Robot]

6 failures in 3599 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1
• mozilla-central: 2
• autoland: 3

Platform and build breakdown:

• linux1804-64-shippable-qr: 3
  • opt: 3
• android-hw-a51-11-0-aarch64-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-01-15&endday=2024-01-21&tree=all

Comment 10

[Intermittent Failures Robot]

11 failures in 4034 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 6
• autoland: 5

Platform and build breakdown:

• linux1804-64-shippable-qr: 11
  • opt: 11

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-01-22&endday=2024-01-28&tree=all

Comment 11

[Intermittent Failures Robot]

7 failures in 3952 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3
• mozilla-beta: 1
• mozilla-release: 2
• autoland: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 4
  • opt: 4
• macosx1015-64-shippable-qr: 2
  • opt: 2
• android-hw-a51-11-0-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-01-29&endday=2024-02-04&tree=all

Comment 12

[Intermittent Failures Robot]

1 failures in 4009 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• android-hw-a51-11-0-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-02-05&endday=2024-02-11&tree=all

Comment 13

[Intermittent Failures Robot]

2 failures in 3125 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform and build breakdown:

• macosx1015-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-02-12&endday=2024-02-18&tree=all

Comment 14

[Intermittent Failures Robot]

2 failures in 3382 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform and build breakdown:

• macosx1015-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-02-19&endday=2024-02-25&tree=all

Comment 15

[Intermittent Failures Robot]

4 failures in 4052 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 3

Platform and build breakdown:

• macosx1015-64-shippable-qr: 3
  • opt: 3
• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-02-26&endday=2024-03-03&tree=all

Comment 16

[Intermittent Failures Robot]

5 failures in 4198 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1
• autoland: 3
• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 1
  • opt: 1
• macosx1015-64-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-03-04&endday=2024-03-10&tree=all

Comment 17

[Intermittent Failures Robot]

4 failures in 3849 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2
• autoland: 2

Platform and build breakdown:

• linux1804-64-shippable-qr: 3
  • opt: 3
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-03-11&endday=2024-03-17&tree=all

Comment 18

[Intermittent Failures Robot]

1 failures in 3791 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1870979&startday=2024-03-18&endday=2024-03-24&tree=all

Comment 19

[Cosmin Sabou [:CosminS]]

No longer happening since the 124 merge to mozilla-release: https://treeherder.mozilla.org/jobs?repo=mozilla-release&noTelemetry=1&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&searchStr=imgur&fromchange=163c38ee7a7a13a344e7fe15b75d86e6c5212ae0
Let's deem this as fixed so we avoid misclassifications here.