1642045 - Intermittent raptor-browsertime Critical: Failed waiting on page <random_url> to finished loading, timed out after 300000 ms WebDriverError: Failed to decode response from marionette

Status: RESOLVED INCOMPLETE

(Testing :: Raptor, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: ccoroiu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=304320138&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Gt5ZCSRBQ5-7Ms9EXts1hQ/runs/0/artifacts/public/logs/live_backing.log

---

[task 2020-05-29T21:40:43.393Z] 21:40:34 INFO - raptor-browsertime Info: Testing url https://www.amazon.com/s/ref=nb_sb_noss_2/139-6317191-5622045?url=search-alias%3Daps&field-keywords=mobile+phone iteration 6
[task 2020-05-29T21:40:43.393Z] 21:40:34 INFO - raptor-browsertime Info: Start firefox window recorder.
[task 2020-05-29T21:40:43.393Z] 21:40:34 INFO - raptor-browsertime Info: Fri, 29 May 2020 21:40:34 GMT adb:command Send '0019host:transport:ZY32259TSG'
[task 2020-05-29T21:40:43.393Z] 21:40:34 INFO - raptor-browsertime Info: Fri, 29 May 2020 21:40:34 GMT adb:command Send '003Dshell:rm -rf -- "/sdcard/browsertime-firefox-windowrecording"'
[task 2020-05-29T21:40:43.393Z] 21:40:34 INFO - raptor-browsertime Info: Fri, 29 May 2020 21:40:34 GMT adb:command Send '0019host:transport:ZY32259TSG'
[task 2020-05-29T21:40:43.393Z] 21:40:34 INFO - raptor-browsertime Info: Fri, 29 May 2020 21:40:34 GMT adb:command Send '0037shell:mkdir /sdcard/browsertime-firefox-windowrecording'
[task 2020-05-29T21:40:43.393Z] 21:40:43 INFO - Exception in thread ProcessReader:
[task 2020-05-29T21:40:43.393Z] 21:40:43 ERROR - Traceback (most recent call last):
[task 2020-05-29T21:40:43.393Z] 21:40:43 INFO - File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
[task 2020-05-29T21:40:43.393Z] 21:40:43 INFO - self.run()
[task 2020-05-29T21:40:43.393Z] 21:40:43 INFO - File "/usr/lib/python2.7/threading.py", line 754, in run
[task 2020-05-29T21:40:43.393Z] 21:40:43 INFO - self.__target(*self.__args, **self.__kwargs)
[task 2020-05-29T21:40:43.393Z] 21:40:43 INFO - File "/builds/task_1590788038/workspace/build/venv/lib/python2.7/site-packages/mozprocess/processhandler.py", line 1116, in _read
[task 2020-05-29T21:40:43.393Z] 21:40:43 INFO - callback(line.rstrip())
[task 2020-05-29T21:40:43.393Z] 21:40:43 INFO - File "/builds/task_1590788038/workspace/build/venv/lib/python2.7/site-packages/mozprocess/processhandler.py", line 1026, in call
[task 2020-05-29T21:40:43.393Z] 21:40:43 INFO - e(*args, **kwargs)
[task 2020-05-29T21:40:43.393Z] 21:40:43 INFO - File "/builds/task_1590788038/workspace/build/tests/raptor/raptor/browsertime/base.py", line 348, in _line_handler
[task 2020-05-29T21:40:43.393Z] 21:40:43 INFO - raise Exception("Browsertime failed to run")
[task 2020-05-29T21:41:15.309Z] 21:40:43 INFO - Exception: Browsertime failed to run
[task 2020-05-29T21:41:15.309Z] 21:40:48 CRITICAL - raptor-browsertime Critical: Failed waiting on page to finished loading, timed out after 300000 ms WebDriverError: Failed to decode response from marionette
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - raptor-browsertime-android Info: removing reverse socket connections
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - adb command_output: adb -s ZY32259TSG wait-for-device reverse --remove-all, timeout: None, timedout: None, exitcode: 0, output:
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - raptor-mitmproxy Info: Stopping mitmproxy playback, killing process 845
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - raptor-mitmproxy Info: Successfully killed the mitmproxy playback process
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - raptor-perftest Info: Removing temporary directory: /tmp/tmpSO4Y86
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - raptor-perftest Info: Removing temporary directory: /tmp/tmpVOSxcr
[task 2020-05-29T21:41:15.309Z] 21:40:48 ERROR - Traceback (most recent call last):
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - File "/builds/task_1590788038/workspace/build/tests/raptor/raptor/raptor.py", line 202, in <module>
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - main()
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - File "/builds/task_1590788038/workspace/build/tests/raptor/raptor/raptor.py", line 149, in main
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - success = raptor.run_tests(raptor_test_list, raptor_test_names)
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - File "/builds/task_1590788038/workspace/build/tests/raptor/raptor/browsertime/android.py", line 143, in run_tests
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - return super(BrowsertimeAndroid, self).run_tests(tests, test_names)
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - File "/builds/task_1590788038/workspace/build/tests/raptor/raptor/perftest.py", line 386, in run_tests
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - self.run_test(test, timeout=int(test.get("page_timeout")))
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - File "/builds/task_1590788038/workspace/build/tests/raptor/raptor/browsertime/base.py", line 364, in run_test
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - raise Exception(self.browsertime_failure)
[task 2020-05-29T21:41:15.309Z] 21:40:48 INFO - Exception: Failed waiting on page to finished loading, timed out after 300000 ms WebDriverError: Failed to decode response from marionette
[task 2020-05-29T21:41:15.309Z] 21:40:48 ERROR - Return code: 1

Comment 1

[Intermittent Failures Robot]

3 failures in 5458 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1642045&startday=2020-05-25&endday=2020-05-31&tree=all

Comment 2

[Greg Mierzwinski [:sparky]]

Remove the "exception followed by" since the exception is always thrown because of the way we are killing browsertime on failures.

Comment 3

[Intermittent Failures Robot]

2 failures in 4949 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 2

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1642045&startday=2020-06-01&endday=2020-06-07&tree=all

Comment 4

[Intermittent Failures Robot]

1 failures in 4035 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1642045&startday=2020-06-15&endday=2020-06-21&tree=all

Comment 5

[Intermittent Failures Robot]

3 failures in 5252 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• fenix: 2

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 1
• android-hw-p2-8-0-android-aarch64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1642045&startday=2020-06-22&endday=2020-06-28&tree=all

Comment 6

[Intermittent Failures Robot]

2 failures in 5265 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1642045&startday=2020-07-06&endday=2020-07-12&tree=all

Comment 7

[Intermittent Failures Robot]

1 failures in 4014 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 1

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1642045&startday=2020-07-13&endday=2020-07-19&tree=all

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

New occurrence: https://treeherder.mozilla.org/logviewer.html#?job_id=313121721&repo=fenix

Comment 10

[Intermittent Failures Robot]

1 failures in 4046 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1642045&startday=2020-08-24&endday=2020-08-30&tree=all

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 12

[Cosmin Sabou [:CosminS]]

https://treeherder.mozilla.org/logviewer?job_id=320513927&repo=autoland

Comment 13

[Intermittent Failures Robot]

2 failures in 4128 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 1

Platform breakdown:

• macosx1014-64-shippable: 1
• macosx1014-64-shippable-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1642045&startday=2020-11-02&endday=2020-11-08&tree=all

Comment 14

[Intermittent Failures Robot]

1 failures in 4040 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2020-11-09&endday=2020-11-15&tree=all

Comment 15

[Intermittent Failures Robot]

2 failures in 4810 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• autoland: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2020-11-16&endday=2020-11-22&tree=all

Comment 16

[Florin Strugariu [:Bebe]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 17

[Intermittent Failures Robot]

3 failures in 4327 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 1
• mozilla-central: 2

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable: 1
  • opt: 1
• windows10-64-ref-hw-2017: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-02-22&endday=2021-02-28&tree=all

Comment 18

[Intermittent Failures Robot]

18 failures in 3683 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 17
• autoland: 1

Platform and build breakdown:

• windows10-64-ref-hw-2017: 17
  • opt: 17
• windows10-64-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-03-01&endday=2021-03-07&tree=all

Comment 19

[Intermittent Failures Robot]

8 failures in 4622 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 8

Platform and build breakdown:

• windows10-64-ref-hw-2017: 8
  • opt: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-03-08&endday=2021-03-14&tree=all

Comment 20

[Intermittent Failures Robot]

22 failures in 714 pushes (0.031 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 22

Platform and build breakdown:

• windows10-64-shippable-qr: 4
  • opt: 4
• windows10-64-shippable: 3
  • opt: 3
• macosx1015-64-shippable-qr: 2
  • opt: 2
• macosx1015-64-shippable: 2
  • opt: 2
• linux1804-64-shippable-qr: 3
  • opt: 3
• linux1804-64-shippable: 4
  • opt: 4
• android-hw-p2-8-0-android-aarch64-shippable: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-03-26&endday=2021-03-26&tree=all

Comment 21

[Intermittent Failures Robot]

22 failures in 3967 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 22

Platform and build breakdown:

• windows10-64-shippable-qr: 4
  • opt: 4
• windows10-64-shippable: 3
  • opt: 3
• macosx1015-64-shippable-qr: 2
  • opt: 2
• macosx1015-64-shippable: 2
  • opt: 2
• linux1804-64-shippable-qr: 3
  • opt: 3
• linux1804-64-shippable: 4
  • opt: 4
• android-hw-p2-8-0-android-aarch64-shippable: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-03-22&endday=2021-03-28&tree=all

Comment 22

[Greg Mierzwinski [:sparky]]

Looks like this was a transient failure. I'll watch it over the course of the week.

Comment 23

[Intermittent Failures Robot]

1 failures in 3854 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-04-05&endday=2021-04-11&tree=all

Comment 24

[Intermittent Failures Robot]

3 failures in 3224 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform and build breakdown:

• linux1804-64-shippable: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-04-19&endday=2021-04-25&tree=all

Comment 25

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 26

[Intermittent Failures Robot]

2 failures in 3061 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform and build breakdown:

• windows10-64-shippable: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-05-17&endday=2021-05-23&tree=all

Comment 27

[Sandor Molnar[:smolnar]]

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=343758362&repo=autoland&lineNumber=2188

Comment 28

[Intermittent Failures Robot]

2 failures in 3115 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 1

Platform and build breakdown:

• android-hw-g5-7-0-arm7-shippable-qr: 1
  • opt: 1
• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-06-21&endday=2021-06-27&tree=all

Comment 29

[Intermittent Failures Robot]

3 failures in 723 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• mozilla-central: 1

Platform and build breakdown:

• macosx1014-64-shippable-qr: 1
  • opt: 1
• linux1804-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-06-28&endday=2021-07-04&tree=all

Comment 30

[Intermittent Failures Robot]

4 failures in 2387 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2
• autoland: 2

Platform and build breakdown:

• windows10-64-shippable-qr: 3
  • opt: 3
• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-07-05&endday=2021-07-11&tree=all

Comment 31

[Alexandru Michis [:malexandru]]

Andi, this has permafailed on beta in clang builds:
https://treeherder.mozilla.org/jobs?repo=mozilla-beta&fromchange=d073be56651cf542531709be27cf3e77c12c1242&searchStr=browsertime&duplicate_jobs=visible

Could you please take a look?

Comment 32

[Andi [:andi]]

The job has been removed from Beta, does this also happens on central?

Comment 33

[Cosmin Sabou [:CosminS]]

This happens only on beta, central is all green: https://treeherder.mozilla.org/jobs?repo=mozilla-central&group_state=expanded&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&searchStr=clang-trunk%2C&revision=3880d0d21aa306cfac44ea0d0fa188b59ae4233c
Bug 1720276 was filled for the beta failures.

Comment 34

[Intermittent Failures Robot]

2 failures in 2224 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform and build breakdown:

• macosx1015-64-shippable-qr: 1
  • opt: 1
• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-07-12&endday=2021-07-18&tree=all

Comment 35

[Intermittent Failures Robot]

4 failures in 2940 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• fenix: 3

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 1
  • opt: 1
• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-api-16-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-07-19&endday=2021-07-25&tree=all

Comment 36

[Intermittent Failures Robot]

8 failures in 3131 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 5
• autoland: 2
• mozilla-central: 1

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 4
  • opt: 4
• windows10-64-shippable-qr: 3
  • opt: 3
• android-hw-g5-7-0-arm7-api-16-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-07-26&endday=2021-08-01&tree=all

Comment 37

[Intermittent Failures Robot]

3 failures in 2829 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 3

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1
• android-hw-p2-8-0-android-aarch64-shippable: 1
  • opt: 1
• android-hw-g5-7-0-arm7-api-16-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-08-02&endday=2021-08-08&tree=all

Comment 38

[Intermittent Failures Robot]

12 failures in 3025 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 9
• mozilla-central: 3

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 6
  • opt: 6
• android-hw-g5-7-0-arm7-api-16-shippable: 2
  • opt: 2
• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1
• linux1804-64-clang-trunk-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-08-09&endday=2021-08-15&tree=all

Comment 39

[Intermittent Failures Robot]

12 failures in 2899 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 10
• autoland: 2

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 6
  • opt: 6
• macosx1014-64-shippable-qr: 1
  • opt: 1
• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2
• linux1804-64-shippable-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-api-16-shippable: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-08-16&endday=2021-08-22&tree=all

Comment 40

[Intermittent Failures Robot]

4 failures in 2580 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• fenix: 2
• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 2
  • opt: 2
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-08-23&endday=2021-08-29&tree=all

Comment 42

[Intermittent Failures Robot]

9 failures in 3084 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• mozilla-central: 1
• fenix: 6

Platform and build breakdown:

• windows10-64-shippable-qr: 3
  • opt: 3
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 3
  • opt: 3
• android-hw-g5-7-0-arm7-api-16-shippable: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-08-30&endday=2021-09-05&tree=all

Comment 43

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

The youtube watch (youtube-w) tasks for fenix started to fail frequently, place investigate or redirect the request.

Comment 44

[Alexandru Ionescu (needinfo me) [:alexandrui]]

This might be due to the video recorded (video unavailable) and some policy consent dialog. Might need re-recording.

Comment 45

[Kimberly Sereduck :kimberlythegeek]

This may be caused by crashes in firefox that browsertime fails to report. There is an existing JIRA ticket for providing crash detection in browsertime, increased priority from medium to high

Comment 46

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Can I have that JIRA ticket link?

Comment 47

[Kimberly Sereduck :kimberlythegeek]

https://mozilla-hub.atlassian.net/browse/FXP-1471

Comment 48

[Intermittent Failures Robot]

5 failures in 3266 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 5

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 3
  • opt: 3
• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-api-16-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-09-06&endday=2021-09-12&tree=all

Comment 49

[Intermittent Failures Robot]

6 failures in 3375 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 4
• autoland: 2

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 2
  • opt: 2
• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2
• linux1804-64-shippable-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-09-13&endday=2021-09-19&tree=all

Comment 50

[Intermittent Failures Robot]

18 failures in 2873 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 10
• mozilla-central: 4
• autoland: 4

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable: 2
  • opt: 2
• linux1804-64-shippable-qr: 2
  • opt: 2
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 5
  • opt: 5
• macosx1014-64-shippable-qr: 4
  • opt: 4
• windows10-64-shippable-qr: 2
  • opt: 2
• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2
• android-hw-g5-7-0-arm7-api-16-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-09-20&endday=2021-09-26&tree=all

Comment 51

[Intermittent Failures Robot]

6 failures in 3376 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• fenix: 4
• autoland: 1

Platform and build breakdown:

• macosx1014-64-shippable-qr: 2
  • opt: 2
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 3
  • opt: 3
• android-hw-g5-7-0-arm7-api-16-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-09-27&endday=2021-10-03&tree=all

Comment 52

[Intermittent Failures Robot]

27 failures in 3626 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 12
• autoland: 10
• mozilla-central: 5

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 4
  • opt: 4
• android-hw-p2-8-0-android-aarch64-shippable: 1
  • opt: 1
• android-hw-p2-8-0-android-aarch64-shippable-qr: 3
  • opt: 3
• windows10-64-shippable-qr: 8
  • opt: 8
• android-hw-g5-7-0-arm7-api-16-shippable: 4
  • opt: 4
• macosx1014-64-shippable-qr: 4
  • opt: 4
• macosx1015-64-shippable-qr: 2
  • opt: 2
• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-10-04&endday=2021-10-10&tree=all

Comment 53

[Iulian Moraru]

Sorry for the spam. It was perma failing but the issue got resolved by a backout.

Comment 54

[Andreea Pavel [:apavel]]

There are 43 total failures in the last 7 days on

• windows10-64-shippable-qr opt
• linux1804-64-shippable-qr opt

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=355085749&repo=autoland&lineNumber=1304

[task 2021-10-17T15:10:02.783Z] 15:10:02     INFO -  raptor-browsertime Info: Starting a browsertime pageload
[task 2021-10-17T15:10:02.783Z] 15:10:02     INFO -  raptor-browsertime Info: Waiting for 1000 ms (post_startup_delay)
[task 2021-10-17T15:10:03.782Z] 15:10:03     INFO -  raptor-browsertime Info: Navigating to about:blank, count: 0
[task 2021-10-17T15:10:03.783Z] 15:10:03     INFO -  raptor-browsertime Info: Navigating to url about:blank iteration 3
[task 2021-10-17T15:10:09.814Z] 15:10:09     INFO -  raptor-browsertime Info: Navigating to primary url:https://www.cnn.com/2021/03/22/weather/climate-change-warm-waters-lake-michigan/index.html
[task 2021-10-17T15:10:09.814Z] 15:10:09     INFO -  raptor-browsertime Info: Cycle 0, waiting for 1000 ms
[task 2021-10-17T15:10:10.815Z] 15:10:10     INFO -  raptor-browsertime Info: Cycle 0, starting the measure
[task 2021-10-17T15:10:10.815Z] 15:10:10     INFO -  raptor-browsertime Info: Testing url https://www.cnn.com/2021/03/22/weather/climate-change-warm-waters-lake-michigan/index.html iteration 3
[task 2021-10-17T15:10:17.006Z] 15:10:17    ERROR -  raptor-browsertime Error: Browsertime failed to run
[task 2021-10-17T15:10:17.023Z] 15:10:17     INFO -  raptor-browsertime Info:     at Object.throwDecodedError (C:\Users\task_1634469524\fetches\browsertime\node_modules\selenium-webdriver\lib\error.js:517:15)
[task 2021-10-17T15:10:17.024Z] 15:10:17     INFO -  raptor-browsertime Info:     at parseHttpResponse (C:\Users\task_1634469524\fetches\browsertime\node_modules\selenium-webdriver\lib\http.js:643:13)
[task 2021-10-17T15:10:17.024Z] 15:10:17     INFO -  raptor-browsertime Info:     at Executor.execute (C:\Users\task_1634469524\fetches\browsertime\node_modules\selenium-webdriver\lib\http.js:569:28)
[task 2021-10-17T15:10:17.024Z] 15:10:17     INFO -  raptor-browsertime Info:     at processTicksAndRejections (internal/process/task_queues.js:97:5)
[task 2021-10-17T15:10:17.025Z] 15:10:17     INFO -  raptor-browsertime Info:     at async Driver.execute (C:\Users\task_1634469524\fetches\browsertime\node_modules\selenium-webdriver\lib\webdriver.js:732:17)
[task 2021-10-17T15:10:17.025Z] 15:10:17     INFO -  raptor-browsertime Info: URL failed to load, trying 5 more time(s): Failed waiting on page https://www.cnn.com/2021/03/22/weather/climate-change-warm-waters-lake-michigan/index.html  to finished loading, timed out after 300000 ms
[task 2021-10-17T15:10:17.026Z] 15:10:17 CRITICAL -  raptor-browsertime Critical: Failed waiting on page https://www.cnn.com/2021/03/22/weather/climate-change-warm-waters-lake-michigan/index.html to finished loading, timed out after 300000 ms WebDriverError: Failed to decode response from marionette
[task 2021-10-17T15:10:17.026Z] 15:10:17     INFO -  raptor-mitmproxy Info: MitmproxyDesktop stop!!
[task 2021-10-17T15:10:17.027Z] 15:10:17     INFO -  raptor-mitmproxy Info: Mitmproxy stop!!
[task 2021-10-17T15:10:17.027Z] 15:10:17     INFO -  raptor-mitmproxy Info: Stopping mitmproxy playback, killing process 1120
[task 2021-10-17T15:10:17.027Z] 15:10:17     INFO -  raptor-mitmproxy Info: Sending CTRL_BREAK_EVENT to mitmproxy
[task 2021-10-17T15:10:19.024Z] 15:10:19     INFO -  raptor-mitmproxy Info: Successfully killed the mitmproxy playback process
[task 2021-10-17T15:10:19.025Z] 15:10:19     INFO -  raptor-mitmproxy Info: Turning off the browser proxy
[task 2021-10-17T15:10:19.025Z] 15:10:19     INFO -  raptor-mitmproxy Info: writing: C:\Users\task_1634469524\build\application\firefox\distribution\policies.json
[task 2021-10-17T15:10:19.026Z] 15:10:19     INFO -  raptor-perftest Info: Removing temporary directory: C:\Users\task_1634469524\AppData\Local\Temp\tmpizjlrteh
[task 2021-10-17T15:10:19.064Z] 15:10:19    ERROR -  Traceback (most recent call last):
[task 2021-10-17T15:10:19.065Z] 15:10:19     INFO -    File "C:\Users\task_1634469524\build\tests\raptor\raptor\raptor.py", line 213, in <module>
[task 2021-10-17T15:10:19.065Z] 15:10:19     INFO -      main()
[task 2021-10-17T15:10:19.065Z] 15:10:19     INFO -    File "C:\Users\task_1634469524\build\tests\raptor\raptor\raptor.py", line 159, in main
[task 2021-10-17T15:10:19.066Z] 15:10:19     INFO -      success = raptor.run_tests(raptor_test_list, raptor_test_names)
[task 2021-10-17T15:10:19.066Z] 15:10:19     INFO -    File "C:\Users\task_1634469524\build\tests\raptor\raptor\perftest.py", line 457, in run_tests
[task 2021-10-17T15:10:19.066Z] 15:10:19     INFO -      self.run_test(test, timeout=int(test.get("page_timeout")))
[task 2021-10-17T15:10:19.067Z] 15:10:19     INFO -    File "C:\Users\task_1634469524\build\tests\raptor\raptor\browsertime\base.py", line 567, in run_test
[task 2021-10-17T15:10:19.067Z] 15:10:19     INFO -      raise Exception(self.browsertime_failure)
[task 2021-10-17T15:10:19.068Z] 15:10:19     INFO -  Exception: Failed waiting on page https://www.cnn.com/2021/03/22/weather/climate-change-warm-waters-lake-michigan/index.html to finished loading, timed out after 300000 ms WebDriverError: Failed to decode response from marionette
[task 2021-10-17T15:10:19.123Z] 15:10:19    ERROR - Return code: 1
[task 2021-10-17T15:10:19.123Z] 15:10:19  WARNING - setting return code to 1
[task 2021-10-17T15:10:19.123Z] 15:10:19     INFO - Copying Raptor results to upload dir:
[task 2021-10-17T15:10:19.123Z] 15:10:19     INFO - C:\Users\task_1634469524\build\blobber_upload_dir\perfherder-data.json
[task 2021-10-17T15:10:19.123Z] 15:10:19     INFO - Copying raptor results from C:\Users\task_1634469524\build\raptor.json to C:\Users\task_1634469524\build\blobber_upload_dir\perfherder-data.json
[task 2021-10-17T15:10:19.124Z] 15:10:19 CRITICAL - Error copying results C:\Users\task_1634469524\build\raptor.json to upload dir C:\Users\task_1634469524\build\blobber_upload_dir\perfherder-data.json
[task 2021-10-17T15:10:19.124Z] 15:10:19     INFO - [Errno 2] No such file or directory: 'C:\\Users\\task_1634469524\\build\\raptor.json'
[task 2021-10-17T15:10:19.124Z] 15:10:19     INFO - Running post-action listener: _package_coverage_data
[task 2021-10-17T15:10:19.124Z] 15:10:19     INFO - Running post-action listener: _resource_record_post_action
[task 2021-10-17T15:10:19.124Z] 15:10:19     INFO - Running post-action listener: process_java_coverage_data
[task 2021-10-17T15:10:19.125Z] 15:10:19     INFO - Running post-action listener: stop_device

Greg, can you please take a look?

Comment 55

[Intermittent Failures Robot]

46 failures in 3315 pushes (0.014 failures/push) were associated with this bug in the last 7 days.

This is the #15 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 30
• mozilla-central: 13
• fenix: 1
• mozilla-beta: 2

Platform and build breakdown:

• windows10-64-shippable-qr: 28
  • opt: 28
• macosx1015-64-shippable-qr: 7
  • opt: 7
• macosx1014-64-shippable-qr: 4
  • opt: 4
• linux1804-64-shippable-qr: 4
  • opt: 4
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 1
  • opt: 1
• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-10-11&endday=2021-10-17&tree=all

Comment 56

[Greg Mierzwinski [:sparky]]

Yup, thanks for the ni?.

:kimberlythegeek, could you re-record the cnn pages and see if that helps with these failures?

Comment 57

[Kimberly Sereduck :kimberlythegeek]

(In reply to Greg Mierzwinski [:sparky] from comment #56)

:kimberlythegeek, could you re-record the cnn pages and see if that helps with these failures?

will do--although we may need to disable autoplay for these tests as well. I was getting vismet failures at first, presumably because 1) the cookie prompt was not displaying, so 2) the video was automatically playing. This seemed to affect the detection of whether the page was fully loaded or not.

I got around this by increasing wait time while recording

Comment 58

[Greg Mierzwinski [:sparky]]

Ah good point! Can you try disabling the auto-play for only the cnn page and see if that helps instead of re-recording?

Comment 59

[Kimberly Sereduck :kimberlythegeek]

:sparky on it!

Comment 60

[Kimberly Sereduck :kimberlythegeek]

Attachment: Bug 1642045: Re-record cnn to resolve timeout errors

Comment 61

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/329de07e7a96
Re-record cnn to resolve timeout errors r=perftest-reviewers,AlexandruIonescu

Comment 62

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/329de07e7a96

Comment 63

[Greg Mierzwinski [:sparky]]

That patch won't resolve all of the errors we get with this line.

Comment 64

[Alexandru Ionescu (needinfo me) [:alexandrui]]

== Change summary for alert #31998 (as of Thu, 21 Oct 2021 06:47:36 GMT) ==

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
40%	cnn LastVisualChange	macosx1014-64-shippable-qr	cold webrender	12,440.00 -> 7,476.67

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=31998

Comment 65

[Intermittent Failures Robot]

37 failures in 3186 pushes (0.012 failures/push) were associated with this bug in the last 7 days.

This is the #18 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• fenix: 11
• autoland: 13
• mozilla-central: 13

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2
• windows10-64-shippable-qr: 15
  • opt: 15
• macosx1014-64-shippable-qr: 5
  • opt: 5
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 7
  • opt: 7
• linux1804-64-shippable-qr: 4
  • opt: 4
• android-hw-g5-7-0-arm7-api-16-shippable: 2
  • opt: 2
• macosx1015-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-10-18&endday=2021-10-24&tree=all

Comment 66

[Intermittent Failures Robot]

17 failures in 621 pushes (0.027 failures/push) were associated with this bug yesterday.

Repository breakdown:

• fenix: 4
• mozilla-central: 12
• autoland: 1

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 1
  • opt: 1
• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2
• windows10-64-shippable-qr: 11
  • opt: 11
• android-hw-p2-8-0-android-aarch64-shippable: 1
  • opt: 1
• macosx1015-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-10-27&endday=2021-10-27&tree=all

Comment 67

[Iulian Moraru]

Update:
There have been 40 failures within the last 7 days:
• 1 failures on Android 7.0 MotoG5 Shippable WebRender opt
• 3 failures on Linux 18.04 x64 WebRender Shippable opt
• 4 failures on OS X 10.14 WebRender Shippable opt
• 5 failures on OS X 10.15 WebRender Shippable opt
• 27 failures on Windows 10 x64 WebRender Shippable opt
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=356520554&repo=mozilla-central&lineNumber=1277

[task 2021-10-30T22:58:51.717Z] 22:58:51     INFO -  raptor-browsertime Info: Starting a browsertime pageload
[task 2021-10-30T22:58:51.718Z] 22:58:51     INFO -  raptor-browsertime Info: Waiting for 1000 ms (post_startup_delay)
[task 2021-10-30T22:58:52.717Z] 22:58:52     INFO -  raptor-browsertime Info: Navigating to about:blank, count: 0
[task 2021-10-30T22:58:52.718Z] 22:58:52     INFO -  raptor-browsertime Info: Navigating to url about:blank iteration 1
[task 2021-10-30T22:59:00.764Z] 22:59:00     INFO -  raptor-browsertime Info: Navigating to primary url:https://docs.google.com/document/d/1US-07msg12slQtI_xchzYxcKlTs6Fp7WqIc6W5GK5M8/edit?usp=sharing
[task 2021-10-30T22:59:00.764Z] 22:59:00     INFO -  raptor-browsertime Info: Cycle 0, waiting for 1000 ms
[task 2021-10-30T22:59:01.763Z] 22:59:01     INFO -  raptor-browsertime Info: Cycle 0, starting the measure
[task 2021-10-30T22:59:01.764Z] 22:59:01     INFO -  raptor-browsertime Info: Testing url https://docs.google.com/document/d/1US-07msg12slQtI_xchzYxcKlTs6Fp7WqIc6W5GK5M8/edit?usp=sharing iteration 1
[task 2021-10-30T22:59:15.172Z] 22:59:15     INFO -  raptor-browsertime Info: Navigating to secondary url:https://docs.google.com/document/d/1vUnn0ePU-ynArE1OdxyEHXR2G0sl74ja_st_4OOzlgE/preview
[task 2021-10-30T22:59:15.173Z] 22:59:15     INFO -  raptor-browsertime Info: Navigating to url https://docs.google.com/document/d/1vUnn0ePU-ynArE1OdxyEHXR2G0sl74ja_st_4OOzlgE/preview iteration 1
[task 2021-10-30T22:59:29.227Z] 22:59:29     INFO -  raptor-browsertime Info: Navigating to primary url:https://docs.google.com/document/d/1US-07msg12slQtI_xchzYxcKlTs6Fp7WqIc6W5GK5M8/edit?usp=sharing
[task 2021-10-30T22:59:29.227Z] 22:59:29     INFO -  raptor-browsertime Info: Cycle 1, waiting for 1000 ms
[task 2021-10-30T22:59:30.227Z] 22:59:30     INFO -  raptor-browsertime Info: Cycle 1, starting the measure
[task 2021-10-30T22:59:30.227Z] 22:59:30     INFO -  raptor-browsertime Info: Testing url https://docs.google.com/document/d/1US-07msg12slQtI_xchzYxcKlTs6Fp7WqIc6W5GK5M8/edit?usp=sharing iteration 1
[task 2021-10-30T22:59:36.447Z] 22:59:36    ERROR -  raptor-browsertime Error: Browsertime failed to run
[task 2021-10-30T22:59:36.473Z] 22:59:36     INFO -  raptor-browsertime Info:     at Object.throwDecodedError (C:\Users\task_1635598598\fetches\browsertime\node_modules\selenium-webdriver\lib\error.js:517:15)
[task 2021-10-30T22:59:36.474Z] 22:59:36     INFO -  raptor-browsertime Info:     at parseHttpResponse (C:\Users\task_1635598598\fetches\browsertime\node_modules\selenium-webdriver\lib\http.js:643:13)
[task 2021-10-30T22:59:36.475Z] 22:59:36     INFO -  raptor-browsertime Info:     at Executor.execute (C:\Users\task_1635598598\fetches\browsertime\node_modules\selenium-webdriver\lib\http.js:569:28)
[task 2021-10-30T22:59:36.475Z] 22:59:36     INFO -  raptor-browsertime Info:     at processTicksAndRejections (internal/process/task_queues.js:97:5)
[task 2021-10-30T22:59:36.475Z] 22:59:36     INFO -  raptor-browsertime Info:     at async Driver.execute (C:\Users\task_1635598598\fetches\browsertime\node_modules\selenium-webdriver\lib\webdriver.js:732:17)
[task 2021-10-30T22:59:36.476Z] 22:59:36     INFO -  raptor-browsertime Info: URL failed to load, trying 5 more time(s): Failed waiting on page https://docs.google.com/document/d/1US-07msg12slQtI_xchzYxcKlTs6Fp7WqIc6W5GK5M8/edit?usp=sharing  to finished loading, timed out after 300000 ms
[task 2021-10-30T22:59:36.477Z] 22:59:36 CRITICAL -  raptor-browsertime Critical: Failed waiting on page https://docs.google.com/document/d/1US-07msg12slQtI_xchzYxcKlTs6Fp7WqIc6W5GK5M8/edit?usp=sharing to finished loading, timed out after 300000 ms WebDriverError: Failed to decode response from marionette
[task 2021-10-30T22:59:36.477Z] 22:59:36     INFO -  raptor-mitmproxy Info: MitmproxyDesktop stop!!
[task 2021-10-30T22:59:36.477Z] 22:59:36     INFO -  raptor-mitmproxy Info: Mitmproxy stop!!
[task 2021-10-30T22:59:36.478Z] 22:59:36     INFO -  raptor-mitmproxy Info: Stopping mitmproxy playback, killing process 2036
[task 2021-10-30T22:59:36.478Z] 22:59:36     INFO -  raptor-mitmproxy Info: Sending CTRL_BREAK_EVENT to mitmproxy
[task 2021-10-30T22:59:38.474Z] 22:59:38     INFO -  raptor-mitmproxy Info: Successfully killed the mitmproxy playback process
[task 2021-10-30T22:59:38.475Z] 22:59:38     INFO -  raptor-mitmproxy Info: Turning off the browser proxy
[task 2021-10-30T22:59:38.475Z] 22:59:38     INFO -  raptor-mitmproxy Info: writing: C:\Users\task_1635598598\build\application\firefox\distribution\policies.json
[task 2021-10-30T22:59:38.476Z] 22:59:38     INFO -  raptor-perftest Info: Removing temporary directory: C:\Users\task_1635598598\AppData\Local\Temp\tmpzu8ps7xc
[task 2021-10-30T22:59:38.516Z] 22:59:38    ERROR -  Traceback (most recent call last):
[task 2021-10-30T22:59:38.517Z] 22:59:38     INFO -    File "C:\Users\task_1635598598\build\tests\raptor\raptor\raptor.py", line 213, in <module>
[task 2021-10-30T22:59:38.517Z] 22:59:38     INFO -      main()
[task 2021-10-30T22:59:38.517Z] 22:59:38     INFO -    File "C:\Users\task_1635598598\build\tests\raptor\raptor\raptor.py", line 159, in main
[task 2021-10-30T22:59:38.518Z] 22:59:38     INFO -      success = raptor.run_tests(raptor_test_list, raptor_test_names)
[task 2021-10-30T22:59:38.518Z] 22:59:38     INFO -    File "C:\Users\task_1635598598\build\tests\raptor\raptor\perftest.py", line 457, in run_tests
[task 2021-10-30T22:59:38.518Z] 22:59:38     INFO -      self.run_test(test, timeout=int(test.get("page_timeout")))
[task 2021-10-30T22:59:38.519Z] 22:59:38     INFO -    File "C:\Users\task_1635598598\build\tests\raptor\raptor\browsertime\base.py", line 567, in run_test
[task 2021-10-30T22:59:38.519Z] 22:59:38     INFO -      raise Exception(self.browsertime_failure)

Comment 68

[Iulian Moraru]

Hi Kimberly, are you working on this? Any updates?

Comment 69

[Intermittent Failures Robot]

47 failures in 2845 pushes (0.017 failures/push) were associated with this bug in the last 7 days.

This is the #13 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 18
• mozilla-central: 21
• fenix: 8

Platform and build breakdown:

• windows10-64-shippable-qr: 26
  • opt: 26
• macosx1015-64-shippable-qr: 5
  • opt: 5
• android-hw-g5-7-0-arm7-api-16-shippable: 2
  • opt: 2
• macosx1014-64-shippable-qr: 4
  • opt: 4
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 3
  • opt: 3
• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2
• android-hw-p2-8-0-android-aarch64-shippable: 1
  • opt: 1
• linux1804-64-shippable-qr: 3
  • opt: 3
• android-hw-g5-7-0-arm7-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-10-25&endday=2021-10-31&tree=all

Comment 70

[Kimberly Sereduck :kimberlythegeek]

There have been a high number of intermittent failures for cnn page load test since re-recording in mitm6. I ran multiple (~20) test runs using the mitm6 recording:

• without secondary url, autoplay disabled
• without secondary url, autoplay enabled
• with secondary url, autoplay disabled (current configuration)
• with secondary url, autoplay enabled

there continues to be a high number of failures with each of these methods. These failures arose after re-recording in mitm6, and this site has already been re-recorded multiple times.

I suspect that either this is an issue with mitm6 or with changes to cnn live site since the last recording. Since we are currently working on upgrading to mitmproxy 7 and will need to record again soon anyways, I propose restoring cnn to mitm5 until the upgrade is complete.

Comment 71

[Intermittent Failures Robot]

29 failures in 3030 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

This is the #41 most frequent failure this week.

Repository breakdown:

• mozilla-central: 8
• fenix: 13
• autoland: 8

Platform and build breakdown:

• windows10-64-shippable-qr: 8
  • opt: 8
• android-hw-p2-8-0-android-aarch64-shippable-qr: 6
  • opt: 6
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 5
  • opt: 5
• macosx1014-64-shippable-qr: 2
  • opt: 2
• android-hw-g5-7-0-arm7-api-16-shippable: 2
  • opt: 2
• macosx1015-64-shippable-qr: 2
  • opt: 2
• linux1804-64-tsan-qr: 1
  • opt: 1
• linux1804-64-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-11-01&endday=2021-11-07&tree=all

Comment 72

[Intermittent Failures Robot]

12 failures in 2723 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 11
• autoland: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1
• android-hw-p2-8-0-android-aarch64-shippable: 1
  • opt: 1
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 5
  • opt: 5
• android-hw-g5-7-0-arm7-api-16-shippable: 4
  • opt: 4
• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-11-08&endday=2021-11-14&tree=all

Comment 73

[Intermittent Failures Robot]

10 failures in 3259 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 7
• autoland: 1
• mozilla-central: 2

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 5
  • opt: 5
• macosx1014-64-shippable-qr: 1
  • opt: 1
• windows10-64-shippable-qr: 2
  • opt: 2
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-11-15&endday=2021-11-21&tree=all

Comment 74

[Intermittent Failures Robot]

14 failures in 2819 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 8
• mozilla-central: 3
• autoland: 3

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 5
  • opt: 5
• macosx1014-64-shippable-qr: 4
  • opt: 4
• linux1804-64-shippable-qr: 1
  • opt: 1
• android-hw-p2-8-0-android-aarch64-shippable-qr: 3
  • opt: 3
• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-11-22&endday=2021-11-28&tree=all

Comment 75

[Intermittent Failures Robot]

8 failures in 3170 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3
• fenix: 5

Platform and build breakdown:

• macosx1015-64-shippable-qr: 1
  • opt: 1
• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 3
  • opt: 3
• macosx1014-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-11-29&endday=2021-12-05&tree=all

Comment 76

[Intermittent Failures Robot]

13 failures in 3319 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 12
• autoland: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 5
  • opt: 5
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 7
  • opt: 7
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-12-06&endday=2021-12-12&tree=all

Comment 77

[Intermittent Failures Robot]

8 failures in 3263 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• fenix: 4
• mozilla-central: 2

Platform and build breakdown:

• windows10-64-shippable-qr: 2
  • opt: 2
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 3
  • opt: 3
• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1
• linux1804-64-clang-trunk-qr: 1
  • opt: 1
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-12-13&endday=2021-12-19&tree=all

Comment 78

[Intermittent Failures Robot]

10 failures in 2286 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2
• fenix: 8

Platform and build breakdown:

• linux1804-64-clang-trunk-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 6
  • opt: 6
• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2
• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-12-20&endday=2021-12-26&tree=all

Comment 79

[Intermittent Failures Robot]

3 failures in 1011 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 2
• mozilla-central: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1
• windows10-64-shippable-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2021-12-27&endday=2022-01-02&tree=all

Comment 80

[Intermittent Failures Robot]

6 failures in 2801 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 6

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 6
  • opt: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-01-03&endday=2022-01-09&tree=all

Comment 81

[Intermittent Failures Robot]

2 failures in 3060 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 1
• autoland: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-01-10&endday=2022-01-16&tree=all

Comment 82

[Intermittent Failures Robot]

5 failures in 3444 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 5

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 5
  • opt: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-01-17&endday=2022-01-23&tree=all

Comment 83

[Intermittent Failures Robot]

7 failures in 3445 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 6
• mozilla-central: 1

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 6
  • opt: 6
• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-01-24&endday=2022-01-30&tree=all

Comment 84

[Intermittent Failures Robot]

11 failures in 3165 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 11

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 8
  • opt: 8
• android-hw-p2-8-0-android-aarch64-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-01-31&endday=2022-02-06&tree=all

Comment 85

[Intermittent Failures Robot]

4 failures in 3711 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 4

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 3
  • opt: 3
• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-02-07&endday=2022-02-13&tree=all

Comment 86

[Intermittent Failures Robot]

10 failures in 3281 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 10

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 7
  • opt: 7
• android-hw-p2-8-0-android-aarch64-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-02-14&endday=2022-02-20&tree=all

Comment 87

[Intermittent Failures Robot]

5 failures in 2992 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 4
• autoland: 1

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 4
  • opt: 4
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-02-21&endday=2022-02-27&tree=all

Comment 88

[Intermittent Failures Robot]

1 failures in 3063 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 1

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-02-28&endday=2022-03-06&tree=all

Comment 89

[Intermittent Failures Robot]

5 failures in 2837 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 3
• mozilla-central: 2

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 2
  • opt: 2
• linux1804-64-clang-trunk-qr: 2
  • opt: 2
• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-03-07&endday=2022-03-13&tree=all

Comment 90

[Intermittent Failures Robot]

3 failures in 3304 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 3

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-03-14&endday=2022-03-20&tree=all

Comment 91

[Intermittent Failures Robot]

1 failures in 3341 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 1

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-03-21&endday=2022-03-27&tree=all

Comment 92

[Intermittent Failures Robot]

3 failures in 3323 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 3

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-03-28&endday=2022-04-03&tree=all

Comment 93

[Intermittent Failures Robot]

5 failures in 2922 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 3
• mozilla-beta: 2

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 2
  • opt: 2
• android-hw-g5-7-0-arm7-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-04-04&endday=2022-04-10&tree=all

Comment 94

[Intermittent Failures Robot]

6 failures in 2948 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 6

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 6
  • opt: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-04-11&endday=2022-04-17&tree=all

Comment 95

[Intermittent Failures Robot]

3 failures in 3159 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 3

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-04-18&endday=2022-04-24&tree=all

Comment 96

[Intermittent Failures Robot]

8 failures in 3289 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 7
• mozilla-central: 1

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 7
  • opt: 7
• linux1804-64-clang-trunk-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-04-25&endday=2022-05-01&tree=all

Comment 97

[Intermittent Failures Robot]

27 failures in 3540 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

This is the #47 most frequent failure this week.

Repository breakdown:

• autoland: 13
• fenix: 13
• mozilla-central: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 9
  • opt: 9
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 10
  • opt: 10
• linux1804-64-shippable-qr: 5
  • opt: 5
• macosx1015-64-shippable-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-05-02&endday=2022-05-08&tree=all

Comment 98

[Intermittent Failures Robot]

10 failures in 3317 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 9
• mozilla-central: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 7
  • opt: 7
• linux1804-64-clang-trunk-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-05-09&endday=2022-05-15&tree=all

Comment 99

[Intermittent Failures Robot]

11 failures in 2694 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 9
• mozilla-central: 2

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 3
  • opt: 3
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 6
  • opt: 6
• linux1804-64-clang-trunk-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-05-16&endday=2022-05-22&tree=all

Comment 100

[Intermittent Failures Robot]

3 failures in 2624 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 3

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-05-23&endday=2022-05-29&tree=all

Comment 102

[Intermittent Failures Robot]

10 failures in 2917 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 8
• autoland: 1
• mozilla-central: 1

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 8
  • opt: 8
• linux1804-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-05-30&endday=2022-06-05&tree=all

Comment 103

[Intermittent Failures Robot]

11 failures in 3336 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 8
• mozilla-central: 1
• autoland: 2

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 4
  • opt: 4
• linux1804-64-shippable-qr: 3
  • opt: 3
• android-hw-p2-8-0-android-aarch64-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-06-06&endday=2022-06-12&tree=all

Comment 104

[Intermittent Failures Robot]

9 failures in 3170 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 9

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 7
  • opt: 7
• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-06-13&endday=2022-06-19&tree=all

Comment 105

[Intermittent Failures Robot]

2 failures in 3075 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 2

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-06-20&endday=2022-06-26&tree=all

Comment 106

[Intermittent Failures Robot]

5 failures in 3312 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 5

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 4
  • opt: 4
• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-06-27&endday=2022-07-03&tree=all

Comment 107

[Intermittent Failures Robot]

6 failures in 3072 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• fenix: 5

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-07-04&endday=2022-07-10&tree=all

Comment 108

[Intermittent Failures Robot]

5 failures in 3341 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 4
• mozilla-beta: 1

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 3
  • opt: 3
• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-07-11&endday=2022-07-17&tree=all

Comment 109

[Intermittent Failures Robot]

5 failures in 3497 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 5

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 5
  • opt: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-07-18&endday=2022-07-24&tree=all

Comment 110

[Intermittent Failures Robot]

5 failures in 3245 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 4
• mozilla-beta: 1

Platform and build breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable-qr: 4
  • opt: 4
• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-07-25&endday=2022-07-31&tree=all

Comment 111

[Intermittent Failures Robot]

4 failures in 3057 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• fenix: 3

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1
• android-hw-g5-7-0-arm7-api-16-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-08-01&endday=2022-08-07&tree=all

Comment 112

[Intermittent Failures Robot]

2 failures in 3293 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• mozilla-central: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-08-22&endday=2022-08-28&tree=all

Comment 113

[Intermittent Failures Robot]

2 failures in 3023 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-release: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 1
  • opt: 1
• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-09-12&endday=2022-09-18&tree=all

Comment 114

[Intermittent Failures Robot]

2 failures in 2909 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• autoland: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-09-19&endday=2022-09-25&tree=all

Comment 115

[Intermittent Failures Robot]

1 failures in 3034 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2022-10-10&endday=2022-10-16&tree=all

Comment 116

[Intermittent Failures Robot]

1 failures in 3341 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2023-08-07&endday=2023-08-13&tree=all

Comment 117

[Intermittent Failures Robot]

1 failures in 3267 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2023-09-04&endday=2023-09-10&tree=all

Comment 118

[Intermittent Failures Robot]

10 failures in 4018 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3
• autoland: 6
• mozilla-beta: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 3
  • opt: 3
• macosx1015-64-shippable-qr: 7
  • opt: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2023-10-02&endday=2023-10-08&tree=all

Comment 119

[Intermittent Failures Robot]

2 failures in 3764 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform and build breakdown:

• macosx1015-64-shippable-qr: 1
  • opt: 1
• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2023-10-09&endday=2023-10-15&tree=all

Comment 120

[Intermittent Failures Robot]

1 failures in 3484 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2023-10-16&endday=2023-10-22&tree=all

Comment 121

[Intermittent Failures Robot]

2 failures in 3356 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• mozilla-release: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2023-10-30&endday=2023-11-05&tree=all

Comment 122

[Intermittent Failures Robot]

2 failures in 4108 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2023-11-27&endday=2023-12-03&tree=all

Comment 123

[Intermittent Failures Robot]

6 failures in 3671 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 2
• mozilla-beta: 4

Platform and build breakdown:

• linux1804-64-shippable-qr: 6
  • opt: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2023-12-04&endday=2023-12-10&tree=all

Comment 124

[Intermittent Failures Robot]

2 failures in 4224 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 2

Platform and build breakdown:

• linux1804-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2023-12-11&endday=2023-12-17&tree=all

Comment 125

[Intermittent Failures Robot]

2 failures in 1162 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 2

Platform and build breakdown:

• linux1804-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2023-12-25&endday=2023-12-31&tree=all

Comment 126

[Intermittent Failures Robot]

2 failures in 2274 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 2

Platform and build breakdown:

• linux1804-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2024-01-01&endday=2024-01-07&tree=all

Comment 127

[Intermittent Failures Robot]

1 failures in 3217 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2024-01-08&endday=2024-01-14&tree=all

Comment 128

[Intermittent Failures Robot]

1 failures in 3599 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2024-01-15&endday=2024-01-21&tree=all

Comment 129

[Intermittent Failures Robot]

1 failures in 3235 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2024-04-08&endday=2024-04-14&tree=all

Comment 130

[Intermittent Failures Robot]

2 failures in 3649 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform and build breakdown:

• android-hw-a51-11-0-aarch64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2024-06-03&endday=2024-06-09&tree=all

Comment 131

[Intermittent Failures Robot]

1 failures in 3780 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2024-07-08&endday=2024-07-14&tree=all

Comment 132

[Intermittent Failures Robot]

1 failures in 3218 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• android-hw-a51-11-0-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2024-08-19&endday=2024-08-25&tree=all

Comment 133

[Intermittent Failures Robot]

3 failures in 3668 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2
• autoland: 1

Platform and build breakdown:

• android-hw-a55-14-0-android-aarch64-shippable-qr: 1
  • opt: 1
• windows11-64-shippable-qr: 1
  • opt: 1
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2024-09-02&endday=2024-09-08&tree=all

Comment 134

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Comment 135

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Table

	no_variant
windows11-64-24h2--/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2025-04-14&endday=2025-04-20&tree=all

Comment 136

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Table

	nofis
android-aarch64/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2025-06-30&endday=2025-07-06&tree=all

Comment 137

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Table

	nofis
android-aarch64/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2025-08-18&endday=2025-08-24&tree=all

Comment 138

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Table

	no_variant
windows11-x86_64/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2025-09-08&endday=2025-09-14&tree=all

Comment 139

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Table

	no_variant
windows11-x86_64/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2025-09-15&endday=2025-09-21&tree=all

Comment 140

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=531120207&repo=mozilla-central

Comment 141

[Intermittent Failures Robot]

7 failures were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 7

Table

	no_variant
windows11-x86_64/opt	7

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2025-10-13&endday=2025-10-19&tree=all

Comment 142

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Table

	no_variant
windows11-x86_64/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2025-11-03&endday=2025-11-09&tree=all

Comment 143

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Table

	no_variant
android-aarch64/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2025-11-10&endday=2025-11-16&tree=all

Comment 144

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Table

	no_variant
android-aarch64/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2025-11-17&endday=2025-11-23&tree=all

Comment 145

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Table

	no_variant
android-aarch64/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2025-12-08&endday=2025-12-14&tree=all

Comment 146

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Comment 147

[Intermittent Failures Robot]

1 failures were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Table

	no_variant
android-aarch64/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1642045&startday=2026-03-02&endday=2026-03-08&tree=all