1776394 - Intermittent TEST-UNEXPECTED-TIMEOUT | toolkit/components/telemetry/tests/unit/test_HealthPing.js | Test timed out

Status: RESOLVED FIXED

(Toolkit :: Telemetry, defect, P1)

Description

[Treeherder Bug Filer]

Filed by: nfay [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=382409398&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/XvOrBhAzR1-y-JRGL3NRuA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2022-06-24T09:43:10.275Z] 09:43:10     INFO -  TEST-START | toolkit/components/telemetry/tests/unit/test_HealthPing.js
[task 2022-06-24T09:48:10.278Z] 09:48:10  WARNING -  TEST-UNEXPECTED-TIMEOUT | toolkit/components/telemetry/tests/unit/test_HealthPing.js | Test timed out
[task 2022-06-24T09:48:10.279Z] 09:48:10     INFO -  TEST-INFO took 300003ms
[task 2022-06-24T09:48:10.279Z] 09:48:10     INFO -  >>>>>>>
[task 2022-06-24T09:48:10.279Z] 09:48:10     INFO -  (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2022-06-24T09:48:10.279Z] 09:48:10     INFO -  (xpcshell/head.js) | test run_next_test 0 pending (2)
[task 2022-06-24T09:48:10.280Z] 09:48:10     INFO -  (xpcshell/head.js) | test MAIN run_test finished (2)
[task 2022-06-24T09:48:10.280Z] 09:48:10     INFO -  running event loop
[task 2022-06-24T09:48:10.280Z] 09:48:10     INFO -  toolkit/components/telemetry/tests/unit/test_HealthPing.js | Starting setup
[task 2022-06-24T09:48:10.280Z] 09:48:10     INFO -  (xpcshell/head.js) | test setup pending (2)
[task 2022-06-24T09:48:10.281Z] 09:48:10     INFO -  (xpcshell/head.js) | test run_next_test 0 finished (2)
[task 2022-06-24T09:48:10.281Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790409	Toolkit.Telemetry	TRACE	TelemetryEnvironment::constructor"
[task 2022-06-24T09:48:10.281Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790440	Toolkit.Telemetry	TRACE	TelemetryEnvironment::_getGFXData - Only one display adapter detected."
[task 2022-06-24T09:48:10.282Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790444	Toolkit.Telemetry	TRACE	TelemetryEnvironment::_updateSearchEngine - ignoring early call"
[task 2022-06-24T09:48:10.283Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (error) [JavaScript Error: "readUpdateConfig: Unable to read app update configuration file. Exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIProperties.get]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: resource://gre/modules/UpdateUtils.jsm :: getConfigFilePath :: line 154"  data: no]" {file: "resource://gre/modules/UpdateUtils.jsm" line: 865}]
[task 2022-06-24T09:48:10.283Z] 09:48:10     INFO -  readUpdateConfig@resource://gre/modules/UpdateUtils.jsm:865:10
[task 2022-06-24T09:48:10.283Z] 09:48:10     INFO -  _do_main@Z:\\task_165606228933289\\build\\tests\\xpcshell\\head.js:240:6
[task 2022-06-24T09:48:10.283Z] 09:48:10     INFO -  _execute_test@Z:\\task_165606228933289\\build\\tests\\xpcshell\\head.js:597:5
[task 2022-06-24T09:48:10.284Z] 09:48:10     INFO -  @-e:1:1
[task 2022-06-24T09:48:10.284Z] 09:48:10     INFO -  "
[task 2022-06-24T09:48:10.284Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (error) [JavaScript Error: "1656063790447	Toolkit.Telemetry	ERROR	TelemetryEnvironment::EnvironmentCache - error while initializing: TypeError: can't access property "client", gBeforeShutdownBarrier is null(resource://gre/modules/AddonManager.jsm:4300:5) JS Stack trace: get [e-mail]:4300:5
[task 2022-06-24T09:48:10.285Z] 09:48:10     INFO -  [e-mail]:605:5
[task 2022-06-24T09:48:10.285Z] 09:48:10     INFO -  [e-mail]:976:29
[task 2022-06-24T09:48:10.285Z] 09:48:10     INFO -  [e-mail]:102:26
[task 2022-06-24T09:48:10.285Z] 09:48:10     INFO -  [e-mail]:113:12
[task 2022-06-24T09:48:10.286Z] 09:48:10     INFO -  [e-mail]:483:31
[task 2022-06-24T09:48:10.286Z] 09:48:10     INFO -  setup@test_HealthPing.js:69:9
[task 2022-06-24T09:48:10.286Z] 09:48:10     INFO -  _run_next_test/<@Z:\\task_165606228933289\\build\\tests\\xpcshell\\head.js:1766:22
[task 2022-06-24T09:48:10.286Z] 09:48:10     INFO -  _run_next_test@Z:\\task_165606228933289\\build\\tests\\xpcshell\\head.js:1766:38
[task 2022-06-24T09:48:10.287Z] 09:48:10     INFO -  run@Z:\\task_165606228933289\\build\\tests\\xpcshell\\head.js:812:9
[task 2022-06-24T09:48:10.287Z] 09:48:10     INFO -  _do_main@Z:\\task_165606228933289\\build\\tests\\xpcshell\\head.js:240:6
[task 2022-06-24T09:48:10.287Z] 09:48:10     INFO -  _execute_test@Z:\\task_165606228933289\\build\\tests\\xpcshell\\head.js:597:5
[task 2022-06-24T09:48:10.287Z] 09:48:10     INFO -  @-e:1:1" {file: "resource://gre/modules/Log.jsm" line: 725}]
[task 2022-06-24T09:48:10.288Z] 09:48:10     INFO -  append@resource://gre/modules/Log.jsm:725:12
[task 2022-06-24T09:48:10.288Z] 09:48:10     INFO -  log@resource://gre/modules/Log.jsm:381:16
[task 2022-06-24T09:48:10.288Z] 09:48:10     INFO -  getLoggerWithMessagePrefix/proxy.log@resource://gre/modules/Log.jsm:510:18
[task 2022-06-24T09:48:10.288Z] 09:48:10     INFO -  error@resource://gre/modules/Log.jsm:389:10
[task 2022-06-24T09:48:10.289Z] 09:48:10     INFO -  EnvironmentCache/this._initTask<@resource://gre/modules/TelemetryEnvironment.jsm:1006:17
[task 2022-06-24T09:48:10.289Z] 09:48:10     INFO -  _do_main@Z:\\task_165606228933289\\build\\tests\\xpcshell\\head.js:240:6
[task 2022-06-24T09:48:10.289Z] 09:48:10     INFO -  _execute_test@Z:\\task_165606228933289\\build\\tests\\xpcshell\\head.js:597:5
[task 2022-06-24T09:48:10.289Z] 09:48:10     INFO -  @-e:1:1
[task 2022-06-24T09:48:10.289Z] 09:48:10     INFO -  "
[task 2022-06-24T09:48:10.290Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790448	Toolkit.Telemetry	TRACE	TelemetryEnvironment::_startWatchingPrefs - [object Map]"
[task 2022-06-24T09:48:10.290Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790448	Toolkit.Telemetry	TRACE	TelemetryEnvironment::_stopWatchingPrefs"
[task 2022-06-24T09:48:10.290Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790448	Toolkit.Telemetry	TRACE	TelemetryEnvironment::_updateSearchEngine - ignoring early call"
[task 2022-06-24T09:48:10.290Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790448	Toolkit.Telemetry	TRACE	TelemetryEnvironment::_startWatchingPrefs - [object Map]"
[task 2022-06-24T09:48:10.291Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790448	Toolkit.Telemetry	TRACE	TelemetryEnvironment::observe - aTopic: nsPref:changed, aData: toolkit.telemetry.healthping.enabled"
[task 2022-06-24T09:48:10.291Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790448	Toolkit.Telemetry	TRACE	TelemetryController::setupTelemetry"
[task 2022-06-24T09:48:10.291Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790449	Toolkit.Telemetry	TRACE	TelemetryController::registerJsProbes - registering builtin JS probes"
[task 2022-06-24T09:48:10.292Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790449	Toolkit.Telemetry	TRACE	TelemetryController::registerScalarProbes - registering scalar builtin JS probes"
[task 2022-06-24T09:48:10.292Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790449	Toolkit.Telemetry	TRACE	TelemetryController::loadProbeDefinitions - no builtin JS probe file ScalarArtifactDefinitions.json"
[task 2022-06-24T09:48:10.292Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790450	Toolkit.Telemetry	TRACE	TelemetryReportingPolicy::setup"
[task 2022-06-24T09:48:10.293Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790450	Toolkit.Telemetry	CONFIG	TelemetryController::enableTelemetryRecording - canRecordBase:true, canRecordExtended: true"
[task 2022-06-24T09:48:10.293Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790450	Toolkit.Telemetry	TRACE	TelemetrySession::earlyInit"
[task 2022-06-24T09:48:10.293Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790450	Toolkit.Telemetry	INFO	TelemetryReportingPolicy::get dataSubmissionPolicyNotifiedDate - No date stored yet."
[task 2022-06-24T09:48:10.294Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790451	Toolkit.Telemetry	TRACE	UpdatePing::init - enabled: true"
[task 2022-06-24T09:48:10.294Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790451	Toolkit.Telemetry	TRACE	TelemetryController::registerEventProbes - registering builtin JS Event probes"
[task 2022-06-24T09:48:10.294Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790451	Toolkit.Telemetry	TRACE	TelemetryController::loadProbeDefinitions - no builtin JS probe file EventArtifactDefinitions.json"
[task 2022-06-24T09:48:10.295Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (error) [JavaScript Error: "readUpdateConfig: Unable to read app update configuration file. Exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIProperties.get]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: resource://gre/modules/UpdateUtils.jsm :: getConfigFilePath :: line 154"  data: no]" {file: "resource://gre/modules/UpdateUtils.jsm" line: 865}]
[task 2022-06-24T09:48:10.295Z] 09:48:10     INFO -  readUpdateConfig@resource://gre/modules/UpdateUtils.jsm:865:10
[task 2022-06-24T09:48:10.295Z] 09:48:10     INFO -  _do_main@Z:\\task_165606228933289\\build\\tests\\xpcshell\\head.js:240:6
[task 2022-06-24T09:48:10.296Z] 09:48:10     INFO -  _execute_test@Z:\\task_165606228933289\\build\\tests\\xpcshell\\head.js:597:5
[task 2022-06-24T09:48:10.296Z] 09:48:10     INFO -  @-e:1:1
[task 2022-06-24T09:48:10.296Z] 09:48:10     INFO -  "
[task 2022-06-24T09:48:10.296Z] 09:48:10     INFO -  "CONSOLE_MESSAGE: (info) 1656063790469	Toolkit.Telemetry	TRACE	TelemetryEnvironment::_onEnvironmentChange for system-info"

Comment 1

[Norisz Fay [:noriszfay]]

Hello, this bug seems to be permafailing now. Could you take a look at it please? Thank you!

Comment 2

[Chris H-C :chutten]

Weird. It was green when I checked: https://treeherder.mozilla.org/jobs?repo=try&tier=1%2C2%2C3&revision=a58b56af0ed03e09ed092aff82d12855d5d6f61b

Is it a clue that the configuration that's failing xpcshell-spi-nw-1 wasn't the specific one I ran? I've gone looking to see if I could add jobs of that config, but they aren't in the UI.

Comment 3

[Chris H-C :chutten]

The logs do suggest it's failing in the same place and way (it's testing a send timeout, and for whatever reason the send succeeds despite the timeout value being set to a very low value (one millisecond) meaning that the task waiting for the timeout never sees it.)

Comment 4

[Intermittent Failures Robot]

42 failures in 534 pushes (0.079 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 27
• mozilla-central: 13
• try: 2

Platform and build breakdown:

• windows10-64-2004-qr: 26
  • opt: 20
  • debug: 6
• windows10-64-2004-shippable-qr: 9
  • opt: 9
• macosx1015-64-shippable-qr: 1
  • opt: 1
• windows10-32-2004-qr: 1
  • debug: 1
• windows10-64-2004-devedition-qr: 1
  • opt: 1
• macosx1100-64-shippable-qr: 3
  • opt: 3
• windows10-64-2004-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-06-24&endday=2022-06-24&tree=all

Comment 5

[Cosmin Sabou [:CosminS]]

:barret would you like to have a go at a fix here or could we disable the test on windows until a fix is in place maybe after Chris is back from PTO?

Comment 6

[Cosmin Sabou [:CosminS]]

Attachment: Bug 1776394 - Disable test_HealthPing.js on win10_2004 for frequent failures. r=#intermittent-reviewers

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c6cf4533dfad
Disable test_HealthPing.js on win10_2004 for frequent failures. r=intermittent-reviewers,MasterWayZ DONTBUILD

Comment 8

[Intermittent Failures Robot]

20 failures in 119 pushes (0.168 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 12
• mozilla-central: 5
• try: 3

Platform and build breakdown:

• windows10-64-2004-qr: 10
  • opt: 8
  • debug: 2
• windows10-64-2004-shippable-qr: 8
  • opt: 8
• windows10-64-2004-devedition-qr: 1
  • opt: 1
• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-06-25&endday=2022-06-25&tree=all

Comment 9

[Intermittent Failures Robot]

71 failures in 3075 pushes (0.023 failures/push) were associated with this bug in the last 7 days.

This is the #13 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 39
• mozilla-central: 26
• try: 6

Platform and build breakdown:

• windows10-64-2004-qr: 37
  • opt: 28
  • debug: 9
• windows10-64-2004-shippable-qr: 21
  • opt: 21
• windows10-32-2004-qr: 1
  • debug: 1
• macosx1015-64-shippable-qr: 2
  • opt: 2
• windows10-64-2004-devedition-qr: 3
  • opt: 3
• macosx1100-64-shippable-qr: 6
  • opt: 6
• windows10-64-2004-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-06-20&endday=2022-06-26&tree=all

Comment 10

[bszekely]

https://hg.mozilla.org/mozilla-central/rev/c6cf4533dfad

Comment 11

[Intermittent Failures Robot]

42 failures in 3312 pushes (0.013 failures/push) were associated with this bug in the last 7 days.

This is the #19 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 11
• try: 6
• mozilla-beta: 8
• mozilla-central: 17

Platform and build breakdown:

• macosx1100-64-shippable-qr: 34
  • opt: 34
• macosx1015-64-devedition-qr: 3
  • opt: 3
• macosx1015-64-shippable-qr: 3
  • opt: 3
• macosx1015-64-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-06-27&endday=2022-07-03&tree=all

Comment 12

[Chris H-C :chutten]

ni?me to (try to) figure out what the heck is going on here.

Comment 13

[Chris H-C :chutten]

Interesting that this isn't win10_2004-only, and is still happening across a variety of mac platforms (just at an acceptably-infrequent intermittent rate).

Giving this another go over here. We'll see what comes up with my try run, and how it's different from the ones that used to cause the errors.

One possibility is to drop this particular case from the "heatlh" ping tests. Testing network timeouts like this is prone to failure and we could look into more unit-test-y ways to simulate that without playing with unreliable timeouts. That'll allow us to turn the rest of the test file back on across a wider base of platforms. I'd rather figure it out from base principles and get it fixed, but this is sitting in the back of my mind as I do.

Comment 14

[Intermittent Failures Robot]

34 failures in 3072 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #33 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-release: 1
• mozilla-beta: 10
• mozilla-central: 10
• try: 7
• autoland: 6

Platform and build breakdown:

• macosx1100-64-shippable-qr: 28
  • opt: 28
• macosx1015-64-devedition-qr: 1
  • opt: 1
• macosx1015-64-shippable-qr: 5
  • opt: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-07-04&endday=2022-07-10&tree=all

Comment 15

[Chris H-C :chutten]

Attachment: Bug 1776394 - Remove "health" ping submission timeout test r?TravisLong!

This test causes failures intermittently when left alone, and permanently if
we try to fix it.

It is decidedly an integration/systems test instead of a unit test, and our
framework isn't set up to do reliable and reproducible network errors.

The unit behaviour is covered by the other tests (it just ends up calling
recordSendFailure with a different string), so we're losing more coverage
by skipping the test by platform than we will by removing the test entirely.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d7a1dcec29c3
Remove "health" ping submission timeout test r=TravisLong

Comment 17

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/d7a1dcec29c3

Comment 18

[Chris H-C :chutten]

With volumes of failures at 0 on the 14th and 15th, I think we can call this FIXED

(there may be one last intermittent summary comment coming on Monday, though)

Comment 19

[Intermittent Failures Robot]

30 failures in 3341 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #47 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 11
• try: 4
• mozilla-beta: 6
• autoland: 9

Platform and build breakdown:

• macosx1100-64-shippable-qr: 24
  • opt: 24
• macosx1015-64-shippable-qr: 4
  • opt: 4
• macosx1015-64-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-07-11&endday=2022-07-17&tree=all

Comment 20

[Intermittent Failures Robot]

8 failures in 3497 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 7
• mozilla-esr102: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 8
  • opt: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-07-18&endday=2022-07-24&tree=all

Comment 21

[Intermittent Failures Robot]

2 failures in 3245 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 2

Platform and build breakdown:

• macosx1100-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-07-25&endday=2022-07-31&tree=all

Comment 22

[Intermittent Failures Robot]

4 failures in 3057 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 4

Platform and build breakdown:

• macosx1100-64-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-08-01&endday=2022-08-07&tree=all

Comment 23

[Intermittent Failures Robot]

2 failures in 3339 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 2

Platform and build breakdown:

• macosx1100-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-08-08&endday=2022-08-14&tree=all

Comment 24

[Intermittent Failures Robot]

1 failures in 2916 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-08-15&endday=2022-08-21&tree=all

Comment 25

[Intermittent Failures Robot]

1 failures in 3293 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-08-22&endday=2022-08-28&tree=all

Comment 26

[Intermittent Failures Robot]

2 failures in 3537 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 2

Platform and build breakdown:

• macosx1100-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-08-29&endday=2022-09-04&tree=all

Comment 27

[Intermittent Failures Robot]

1 failures in 3616 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-09-05&endday=2022-09-11&tree=all

Comment 28

[Intermittent Failures Robot]

2 failures in 2793 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 2

Platform and build breakdown:

• macosx1100-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-10-17&endday=2022-10-23&tree=all

Comment 29

[Intermittent Failures Robot]

1 failures in 3743 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-10-24&endday=2022-10-30&tree=all

Comment 30

[Intermittent Failures Robot]

1 failures in 3626 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-10-31&endday=2022-11-06&tree=all

Comment 31

[Intermittent Failures Robot]

2 failures in 3938 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 2

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1
• windows10-64-2004-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-11-28&endday=2022-12-04&tree=all

Comment 32

[Intermittent Failures Robot]

3 failures in 2999 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 3

Platform and build breakdown:

• macosx1100-64-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2022-12-19&endday=2022-12-25&tree=all

Comment 33

[Intermittent Failures Robot]

1 failures in 3606 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2023-01-30&endday=2023-02-05&tree=all

Comment 34

[Intermittent Failures Robot]

3 failures in 3863 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 3

Platform and build breakdown:

• macosx1100-64-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2023-03-27&endday=2023-04-02&tree=all

Comment 35

[Intermittent Failures Robot]

1 failures in 3206 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2023-04-10&endday=2023-04-16&tree=all

Comment 36

[Intermittent Failures Robot]

1 failures in 3594 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2023-06-26&endday=2023-07-02&tree=all

Comment 37

[Intermittent Failures Robot]

1 failures in 3309 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2023-07-10&endday=2023-07-16&tree=all

Comment 38

[Intermittent Failures Robot]

1 failures in 2218 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 1

Platform and build breakdown:

• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776394&startday=2023-08-21&endday=2023-08-27&tree=all