1776767 - Permafailing GeckoMediaPlugins.CDMStorageBasic | Value of: false | gtest | timed out after 300 seconds without output

Status: RESOLVED FIXED

(Core :: Audio/Video: GMP, defect)

Description

[Treeherder Bug Filer]

Filed by: imoraru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=382664966&repo=mozilla-beta
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Cw88jNALRXabk_Lww1nVDg/runs/0/artifacts/public/logs/live_backing.log

---

[task 2022-06-27T20:05:41.044Z] 20:05:41     INFO -  TEST-START | GeckoMediaPlugins.CDMStorageBasic
[task 2022-06-27T20:06:11.315Z] 20:06:11  WARNING -  TEST-UNEXPECTED-FAIL | GeckoMediaPlugins.CDMStorageBasic | Value of: false
[task 2022-06-27T20:06:11.320Z] 20:06:11     INFO -    Actual: false
[task 2022-06-27T20:06:11.321Z] 20:06:11     INFO -  Expected: true @ /builds/worker/checkouts/gecko/dom/media/gtest/TestCDMStorage.cpp:418
[task 2022-06-27T20:11:12.389Z] 20:11:12     INFO -  gtest INFO | gtest | process wait complete, returncode=572
[task 2022-06-27T20:11:12.392Z] 20:11:12  WARNING -  gtest TEST-UNEXPECTED-FAIL | gtest | timed out after 300 seconds without output
[task 2022-06-27T20:11:12.392Z] 20:11:12     INFO -  gtest INFO | rungtests.py exits with code 1
[task 2022-06-27T20:11:12.544Z] 20:11:12    ERROR - Return code: 1
[task 2022-06-27T20:11:12.548Z] 20:11:12    ERROR - No tests run or test summary not found
[task 2022-06-27T20:11:12.548Z] 20:11:12     INFO - TinderboxPrint: gtest-gtest<br/><em class="testfail">T-FAIL</em>
[task 2022-06-27T20:11:12.548Z] 20:11:12  WARNING - # TBPL FAILURE #
[task 2022-06-27T20:11:12.548Z] 20:11:12  WARNING - setting return code to 2
[task 2022-06-27T20:11:12.548Z] 20:11:12  WARNING - The gtest suite: gtest ran with return status: FAILURE
[task 2022-06-27T20:11:12.550Z] 20:11:12     INFO - Running post-action listener: _package_coverage_data
[task 2022-06-27T20:11:12.550Z] 20:11:12     INFO - Running post-action listener: _resource_record_post_action
[task 2022-06-27T20:11:12.551Z] 20:11:12     INFO - Running post-action listener: process_java_coverage_data
[task 2022-06-27T20:11:12.551Z] 20:11:12     INFO - [mozharness: 2022-06-27 20:11:12.551917Z] Finished run-tests step (success)
[task 2022-06-27T20:11:12.551Z] 20:11:12     INFO - Running post-run listener: _resource_record_post_run
[task 2022-06-27T20:11:14.081Z] 20:11:14     INFO - instance_metadata.json not found; unable to determine instance type
[task 2022-06-27T20:11:14.086Z] 20:11:14     INFO - Validating Perfherder data against Z:\task_165635762731293\mozharness\external_tools\performance-artifact-schema.json
[task 2022-06-27T20:11:14.098Z] 20:11:14     INFO - PERFHERDER_DATA: {"framework": {"name": "job_resource_usage"}, "suites": [{"name": "gtest.gtest.overall", "extraOptions": ["buildbot-unknown"], "subtests": [{"name": "cpu_percent", "value": 7.433168058455118}, {"name": "io_write_bytes", "value": 2922785792}, {"name": "io.read_bytes", "value": 579670016}, {"name": "io_write_time", "value": 202}, {"name": "io_read_time", "value": 134}]}, {"name": "gtest.gtest.start-pulseaudio", "subtests": [{"name": "time", "value": 0.002039670944213867}]}, {"name": "gtest.gtest.install", "subtests": [{"name": "time", "value": 10.099332094192505}, {"name": "cpu_percent", "value": 12.95138888888889}]}, {"name": "gtest.gtest.stage-files", "subtests": [{"name": "time", "value": 0.8117291927337646}]}, {"name": "gtest.gtest.run-tests", "subtests": [{"name": "time", "value": 949.8705515861511}, {"name": "cpu_percent", "value": 7.378471488912357}]}]}
[task 2022-06-27T20:11:14.098Z] 20:11:14     INFO - Total resource usage - Wall time: 960s; CPU: 7%; Read bytes: 579670016; Write bytes: 2922785792; Read time: 134; Write time: 202
[task 2022-06-27T20:11:14.098Z] 20:11:14     INFO - TinderboxPrint: CPU usage<br/>7.4%
[task 2022-06-27T20:11:14.098Z] 20:11:14     INFO - TinderboxPrint: I/O read bytes / time<br/>579,670,016 / 134
[task 2022-06-27T20:11:14.098Z] 20:11:14     INFO - TinderboxPrint: I/O write bytes / time<br/>2,922,785,792 / 202
[task 2022-06-27T20:11:14.098Z] 20:11:14     INFO - TinderboxPrint: CPU idle<br/>7,112.7 (92.4%)
[task 2022-06-27T20:11:14.098Z] 20:11:14     INFO - TinderboxPrint: CPU system<br/>227.6 (3.0%)
[task 2022-06-27T20:11:14.098Z] 20:11:14     INFO - TinderboxPrint: CPU user<br/>340.6 (4.4%)
[task 2022-06-27T20:11:14.099Z] 20:11:14     INFO - start-pulseaudio - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2022-06-27T20:11:14.099Z] 20:11:14     INFO - install - Wall time: 10s; CPU: 13%; Read bytes: 188416; Write bytes: 269877248; Read time: 0; Write time: 21
[task 2022-06-27T20:11:14.099Z] 20:11:14     INFO - stage-files - Wall time: 1s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2022-06-27T20:11:14.106Z] 20:11:14     INFO - run-tests - Wall time: 950s; CPU: 7%; Read bytes: 578692096; Write bytes: 2571033600; Read time: 134; Write time: 176
[task 2022-06-27T20:11:14.307Z] 20:11:14  WARNING - returning nonzero exit status 2
[taskcluster 2022-06-27T20:11:14.399Z]    Exit Code: 2
[taskcluster 2022-06-27T20:11:14.399Z]    User Time: 15.625ms
[taskcluster 2022-06-27T20:11:14.399Z]  Kernel Time: 31.25ms
[taskcluster 2022-06-27T20:11:14.399Z]    Wall Time: 19m47.4203963s
[taskcluster 2022-06-27T20:11:14.399Z]       Result: FAILED
[taskcluster 2022-06-27T20:11:14.399Z] === Task Finished ===
[taskcluster 2022-06-27T20:11:14.399Z] Task Duration: 19m47.4243962s
[taskcluster 2022-06-27T20:11:14.501Z] Uploading artifact public/logs/localconfig.json from file logs\localconfig.json with content encoding "gzip", mime type "application/json" and expiry 2023-06-27T18:59:21.419Z
[taskcluster 2022-06-27T20:11:14.929Z] Uploading artifact public/test_info/resource-usage.json from file build\blobber_upload_dir\resource-usage.json with content encoding "gzip", mime type "application/json" and expiry 2023-06-27T18:59:21.419Z
[taskcluster 2022-06-27T20:11:15.131Z] Uploading artifact public/test_info/system-info.log from file build\blobber_upload_dir\system-info.log with content encoding "gzip", mime type "text/plain" and expiry 2023-06-27T18:59:21.419Z
[taskcluster 2022-06-27T20:11:15.305Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2023-06-27T18:59:21.419Z
[taskcluster:error] exit status 2

Comment 1

[Intermittent Failures Robot]

1 failures in 3312 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• windows10-64-2004-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-06-27&endday=2022-07-03&tree=all

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 3

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=388738554&repo=autoland

Comment 4

[Intermittent Failures Robot]

2 failures in 3537 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-08-29&endday=2022-09-04&tree=all

Comment 5

[Intermittent Failures Robot]

1 failures in 3616 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-09-05&endday=2022-09-11&tree=all

Comment 6

[Intermittent Failures Robot]

4 failures in 3023 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• mozilla-release: 2

Platform and build breakdown:

• macosx1015-64-qr: 2
  • opt: 2
• macosx1015-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-09-12&endday=2022-09-18&tree=all

Comment 7

[Intermittent Failures Robot]

1 failures in 2041 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-09-26&endday=2022-10-02&tree=all

Comment 8

[Intermittent Failures Robot]

1 failures in 2449 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-2004-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-10-03&endday=2022-10-09&tree=all

Comment 9

[Intermittent Failures Robot]

1 failures in 3034 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-2004-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-10-10&endday=2022-10-16&tree=all

Comment 10

[Intermittent Failures Robot]

2 failures in 3626 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• try: 1

Platform and build breakdown:

• windows10-64-2004-asan-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-10-31&endday=2022-11-06&tree=all

Comment 11

[Intermittent Failures Robot]

2 failures in 3445 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 1

Platform and build breakdown:

• windows10-64-2004-asan-qr: 1
  • opt: 1
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-11-07&endday=2022-11-13&tree=all

Comment 12

[Intermittent Failures Robot]

1 failures in 3589 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-11-14&endday=2022-11-20&tree=all

Comment 13

[Intermittent Failures Robot]

2 failures in 3938 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform and build breakdown:

• macosx1015-64-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-11-28&endday=2022-12-04&tree=all

Comment 14

[Intermittent Failures Robot]

2 failures in 3558 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• mozilla-central: 1

Platform and build breakdown:

• macosx1015-64-shippable-qr: 1
  • opt: 1
• windows10-64-2004-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-12-12&endday=2022-12-18&tree=all

Comment 15

[Intermittent Failures Robot]

2 failures in 2999 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• try: 1

Platform and build breakdown:

• windows10-64-2004-asan-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2022-12-19&endday=2022-12-25&tree=all

Comment 16

[Intermittent Failures Robot]

3 failures in 2607 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform and build breakdown:

• windows10-64-2004-asan-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-01-02&endday=2023-01-08&tree=all

Comment 17

[Intermittent Failures Robot]

1 failures in 3530 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-01-09&endday=2023-01-15&tree=all

Comment 18

[Intermittent Failures Robot]

2 failures in 3717 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-01-16&endday=2023-01-22&tree=all

Comment 19

[Intermittent Failures Robot]

1 failures in 3606 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr102: 1

Platform and build breakdown:

• windows10-64-2004-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-01-30&endday=2023-02-05&tree=all

Comment 20

[Intermittent Failures Robot]

2 failures in 3766 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1
• windows10-64-2004-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-02-06&endday=2023-02-12&tree=all

Comment 21

[Intermittent Failures Robot]

2 failures in 3856 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform and build breakdown:

• windows10-64-2004-asan-qr: 1
  • opt: 1
• macosx1015-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-02-13&endday=2023-02-19&tree=all

Comment 22

[Intermittent Failures Robot]

2 failures in 3752 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform and build breakdown:

• windows10-64-2004-asan-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-02-27&endday=2023-03-05&tree=all

Comment 23

[Intermittent Failures Robot]

3 failures in 3966 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1
• autoland: 2

Platform and build breakdown:

• windows10-64-2004-asan-qr: 1
  • opt: 1
• macosx1015-64-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-03-06&endday=2023-03-12&tree=all

Comment 24

[Intermittent Failures Robot]

1 failures in 4161 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-03-20&endday=2023-03-26&tree=all

Comment 25

[Intermittent Failures Robot]

1 failures in 3386 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-04-03&endday=2023-04-09&tree=all

Comment 26

[Intermittent Failures Robot]

2 failures in 3206 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform and build breakdown:

• macosx1015-64-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-04-10&endday=2023-04-16&tree=all

Comment 27

[Intermittent Failures Robot]

4 failures in 4004 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1
• autoland: 2
• mozilla-beta: 1

Platform and build breakdown:

• windows10-64-2004-qr: 2
  • debug: 2
• macosx1015-64-qr: 1
  • opt: 1
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-04-17&endday=2023-04-23&tree=all

Comment 28

[Intermittent Failures Robot]

1 failures in 3411 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-05-01&endday=2023-05-07&tree=all

Comment 29

[Intermittent Failures Robot]

18 failures in 4077 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2
• autoland: 6
• mozilla-beta: 8
• try: 2

Platform and build breakdown:

• windows11-64-2009-ccov-qr: 2
  • opt: 2
• windows11-64-2009-asan-qr: 4
  • opt: 4
• windows11-64-2009-qr: 9
  • debug: 8
  • opt: 1
• windows11-64-2009-shippable-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-05-08&endday=2023-05-14&tree=all

Comment 30

[Intermittent Failures Robot]

2 failures in 3578 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1
• autoland: 1

Platform and build breakdown:

• windows11-64-2009-qr: 2
  • debug: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-05-15&endday=2023-05-21&tree=all

Comment 31

[Intermittent Failures Robot]

3 failures in 3885 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2
• autoland: 1

Platform and build breakdown:

• windows11-64-2009-qr: 1
  • opt: 1
• macosx1015-64-qr: 1
  • opt: 1
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-06-05&endday=2023-06-11&tree=all

Comment 32

[Intermittent Failures Robot]

2 failures in 4079 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2

Platform and build breakdown:

• windows11-64-2009-qr: 1
  • opt: 1
• windows11-64-2009-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-06-12&endday=2023-06-18&tree=all

Comment 33

[:gerard-majax]

Do you think we can close this ?

Comment 34

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

It's still a frequent failure. Example. Bug 1832020 should be reopened.

Comment 35

[:gerard-majax]

(In reply to Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout) from comment #34)

It's still a frequent failure. Example. Bug 1832020 should be reopened.

It makes no sense, I pushed retries in sets of 200-300 and it was constantly green.

Comment 36

[:gerard-majax]

(In reply to Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout) from comment #34)

It's still a frequent failure. Example. Bug 1832020 should be reopened.

This one is on ASAN, maybe it's too slow? There's no indication of IPC timeout there though.

Comment 37

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Should the test be excluded on ASan or will the failure be handled by a code change.

Comment 38

[:gerard-majax]

I have no idea, I dont know this part.

Comment 39

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

(In reply to Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout) from comment #37)

Should the test be excluded on ASan or will the failure be handled by a code change.

Jim, please discuss this in the next triage meeting.

Comment 40

[Intermittent Failures Robot]

51 failures in 3880 pushes (0.013 failures/push) were associated with this bug in the last 7 days.

This is the #21 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 8
• mozilla-beta: 20
• autoland: 18
• try: 4
• mozilla-release: 1

Platform and build breakdown:

• windows11-64-2009-ccov-qr: 3
  • opt: 3
• windows11-64-2009-asan-qr: 33
  • opt: 33
• windows11-64-2009-qr: 8
  • debug: 8
• windows11-64-2009-shippable-qr: 6
  • opt: 6
• macosx1015-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-06-19&endday=2023-06-25&tree=all

Comment 41

[Natalia Csoregi [:nataliaCs]]

Update

There have been 44 failures within the last 7 days:

• 1 failure on OS X 10.15 WebRender opt
• 37 failures on Windows 11 x64 22H2 asan WebRender opt
• 6 failures on Windows 11 x64 22H2 CCov WebRender opt

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=421173454&repo=mozilla-central&lineNumber=12758

[task 2023-06-30T05:38:16.161Z] 05:38:16     INFO -  TEST-START | GeckoMediaPlugins.CDMStorageBasic
[task 2023-06-30T05:38:46.156Z] 05:38:46  WARNING -  TEST-UNEXPECTED-FAIL | GeckoMediaPlugins.CDMStorageBasic | Value of: false
[task 2023-06-30T05:38:46.159Z] 05:38:46     INFO -    Actual: false
[task 2023-06-30T05:38:46.159Z] 05:38:46     INFO -  Expected: true @ /builds/worker/checkouts/gecko/dom/media/gtest/TestCDMStorage.cpp:420
[task 2023-06-30T05:43:46.163Z] 05:43:46  WARNING -  gtest TEST-UNEXPECTED-FAIL | gtest | timed out after 300 seconds without output
[task 2023-06-30T05:43:46.174Z] 05:43:46     INFO -  mozcrash INFO | Writing a dump to Z:\task_168810209882326\build\tests\gtest\d8060beb-f2c1-4480-b2b9-606c6afe2e20.dmp for [3284]
[task 2023-06-30T05:43:46.257Z] 05:43:46     INFO -  gtest INFO | gtest | process wait complete, returncode=1
[task 2023-06-30T05:43:46.257Z] 05:43:46     INFO -  mozcrash checking Z:\task_168810209882326\build\tests\gtest for minidumps...
[task 2023-06-30T05:43:46.276Z] 05:43:46     INFO -  mozcrash INFO | Downloading symbols from: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/AeeRTk1UTqyycfHrRjUHxQ/artifacts/public/build/target.crashreporter-symbols.zip
[task 2023-06-30T05:43:51.571Z] 05:43:51     INFO -  mozcrash INFO | Downloading symbols from: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/AeeRTk1UTqyycfHrRjUHxQ/artifacts/public/build/target.crashreporter-symbols.zip
[task 2023-06-30T05:44:01.839Z] 05:44:01     INFO -  mozcrash INFO | Downloading symbols from: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/AeeRTk1UTqyycfHrRjUHxQ/artifacts/public/build/target.crashreporter-symbols.zip
[task 2023-06-30T05:44:25.786Z] 05:44:25     INFO -  mozcrash INFO | Downloading symbols from: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/AeeRTk1UTqyycfHrRjUHxQ/artifacts/public/build/target.crashreporter-symbols.zip
[task 2023-06-30T05:45:06.928Z] 05:45:06     INFO -  mozcrash INFO | Downloading symbols from: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/AeeRTk1UTqyycfHrRjUHxQ/artifacts/public/build/target.crashreporter-symbols.zip
[task 2023-06-30T05:45:07.273Z] 05:45:07     INFO -  gtest ERROR | HTTP Error 404: Not Found
[task 2023-06-30T05:45:07.279Z] 05:45:07     INFO -  gtest INFO  | rungtests.py exits with code 1
[task 2023-06-30T05:45:07.350Z] 05:45:07     INFO - Return code: 1
[task 2023-06-30T05:45:07.363Z] 05:45:07    ERROR - No tests run or test summary not found
[task 2023-06-30T05:45:07.363Z] 05:45:07     INFO - TinderboxPrint: gtest-gtest<br/><em class="testfail">T-FAIL</em>

Comment 42

[Intermittent Failures Robot]

55 failures in 3594 pushes (0.015 failures/push) were associated with this bug in the last 7 days.

This is the #13 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 22
• mozilla-central: 19
• try: 9
• mozilla-beta: 1
• mozilla-esr115: 3
• mozilla-release: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 49
  • opt: 49
• windows11-64-2009-ccov-qr: 6
  • opt: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-06-26&endday=2023-07-02&tree=all

Comment 43

[Justin Link]

I made a change to how TimeStamp comparisons are implemented (making them much simpler and faster) and I'm seeing a noticeable uptick in the rate at which GTest-1proc is failing for the reason captured in this bug. (It was failing quite a bit before my change too but there's a definite increase with it.)

Before:
https://treeherder.mozilla.org/jobs?repo=try&revision=bd1f6ae63071271510c592b9600fe53498d39cf3&selectedTaskRun=HG2pnEPkQ9yLbt0PiDkmKQ.0

After:
https://treeherder.mozilla.org/jobs?repo=try&revision=127269ef4c0c78f37f4461e24e754baa0a471563&selectedTaskRun=HX0OwcwWRbGVAITBnam45g.0

Is this test really sensitive to timing? Or does it do some logic based on comparisons of TimeStamp values?

Comment 44

[Intermittent Failures Robot]

15 failures in 129 pushes (0.116 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 5
• mozilla-central: 3
• try: 6
• mozilla-beta: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 13
  • opt: 13
• windows11-64-2009-ccov-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-07-08&endday=2023-07-08&tree=all

Comment 45

[Intermittent Failures Robot]

72 failures in 2956 pushes (0.024 failures/push) were associated with this bug in the last 7 days.

This is the #10 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 28
• mozilla-central: 18
• mozilla-beta: 8
• mozilla-esr115: 6
• try: 11
• mozilla-release: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 65
  • opt: 65
• windows11-64-2009-ccov-qr: 7
  • opt: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-07-03&endday=2023-07-09&tree=all

Comment 46

[Intermittent Failures Robot]

18 failures in 589 pushes (0.031 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 4
• mozilla-central: 2
• try: 3
• mozilla-beta: 1
• mozilla-release: 2
• mozilla-esr115: 6

Platform and build breakdown:

• windows11-64-2009-asan-qr: 16
  • opt: 16
• windows11-64-2009-ccov-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-07-10&endday=2023-07-10&tree=all

Comment 47

[Natalia Csoregi [:nataliaCs]]

Update

There have been 43 failures within the last 7 days:

• 1 failure on OS X 10.15 WebRender opt
• 33 failures on Windows 11 x64 22H2 asan WebRender opt
• 9 failures on Windows 11 x64 22H2 CCov WebRender opt

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=422465494&repo=mozilla-central&lineNumber=34064

Comment 48

[Intermittent Failures Robot]

63 failures in 3309 pushes (0.019 failures/push) were associated with this bug in the last 7 days.

This is the #12 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 22
• mozilla-central: 12
• try: 8
• mozilla-beta: 10
• mozilla-release: 2
• mozilla-esr115: 9

Platform and build breakdown:

• windows11-64-2009-asan-qr: 54
  • opt: 54
• windows11-64-2009-ccov-qr: 7
  • opt: 7
• macosx1015-64-qr: 2
  • opt: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-07-10&endday=2023-07-16&tree=all

Comment 49

[Intermittent Failures Robot]

15 failures in 756 pushes (0.02 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 6
• autoland: 6
• mozilla-beta: 2
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-ccov-qr: 3
  • opt: 3
• windows11-64-2009-asan-qr: 11
  • opt: 11
• windows11-64-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-07-19&endday=2023-07-19&tree=all

Comment 50

[Intermittent Failures Robot]

64 failures in 3520 pushes (0.018 failures/push) were associated with this bug in the last 7 days.

This is the #12 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 23
• try: 10
• mozilla-beta: 10
• mozilla-esr115: 5
• mozilla-central: 16

Platform and build breakdown:

• windows11-64-2009-asan-qr: 59
  • opt: 59
• windows11-64-2009-ccov-qr: 4
  • opt: 4
• windows11-64-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-07-17&endday=2023-07-23&tree=all

Comment 51

[Natalia Csoregi [:nataliaCs]]

Update

There have been 35 total failures within the last 7 days:

• 3 failures on Windows 11 x64 22H2 CCov WebRender opt
• 32 failures on Windows 11 x64 22H2 asan WebRender opt

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=424026900&repo=autoland&lineNumber=12742

Comment 52

[Intermittent Failures Robot]

48 failures in 3403 pushes (0.014 failures/push) were associated with this bug in the last 7 days.

This is the #17 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 21
• mozilla-beta: 2
• mozilla-central: 16
• try: 7
• mozilla-esr115: 1
• mozilla-release: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 44
  • opt: 44
• windows11-64-2009-ccov-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-07-24&endday=2023-07-30&tree=all

Comment 53

[Intermittent Failures Robot]

21 failures in 652 pushes (0.032 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-beta: 4
• mozilla-release: 2
• autoland: 7
• mozilla-central: 5
• try: 1
• mozilla-esr115: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 18
  • opt: 18
• windows11-64-2009-ccov-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-08-03&endday=2023-08-03&tree=all

Comment 54

[Natalia Csoregi [:nataliaCs]]

Update

There have been 54 total failures within the last 7 days:

• 9 failures on windows11-64-2009-ccov-qr
• 44 failures on Windows 11 x64 22H2 asan WebRender opt
• 1 failure on OS X 10.15 WebRender opt

Recent log: https://treeherder.mozilla.org/logviewer?job_id=425065620&repo=mozilla-central&lineNumber=12749

[task 2023-08-05T22:21:51.223Z] 22:21:51     INFO -  TEST-START | GeckoMediaPlugins.CDMStorageBasic
[task 2023-08-05T22:22:21.232Z] 22:22:21  WARNING -  TEST-UNEXPECTED-FAIL | GeckoMediaPlugins.CDMStorageBasic | Value of: false
[task 2023-08-05T22:22:21.235Z] 22:22:21     INFO -    Actual: false
[task 2023-08-05T22:22:21.235Z] 22:22:21     INFO -  Expected: true @ /builds/worker/checkouts/gecko/dom/media/gtest/TestCDMStorage.cpp:420
[task 2023-08-05T22:27:21.247Z] 22:27:21  WARNING -  gtest TEST-UNEXPECTED-FAIL | gtest | timed out after 300 seconds without output
[task 2023-08-05T22:27:21.260Z] 22:27:21     INFO -  mozcrash INFO | Writing a dump to Z:\task_169127273656953\build\tests\gtest\1199ef74-852d-41df-8d61-170b93f65b26.dmp for [7976]
[task 2023-08-05T22:27:21.372Z] 22:27:21     INFO -  gtest INFO | gtest | process wait complete, returncode=None

This has reached our disable-recommended queue.
Justin, are you still looking into this?
Thank you.

Comment 55

[Justin Link]

No concerns either way from me at this point. I'm not sure why my change consistently increased the failure rate before but, after rebasing my change onto newer code, the increase in failures went away and I landed my change.

Comment 56

[Intermittent Failures Robot]

83 failures in 3276 pushes (0.025 failures/push) were associated with this bug in the last 7 days.

This is the #15 most frequent failure this week.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 30
• try: 11
• mozilla-central: 21
• mozilla-beta: 14
• mozilla-esr115: 5
• mozilla-release: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 73
  • opt: 73
• macosx1015-64-qr: 1
  • opt: 1
• windows11-64-2009-ccov-qr: 9
  • opt: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-07-31&endday=2023-08-06&tree=all

Comment 57

[Intermittent Failures Robot]

15 failures in 696 pushes (0.022 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 1
• autoland: 6
• mozilla-central: 3
• mozilla-release: 1
• mozilla-beta: 3
• mozilla-esr115: 1

Platform and build breakdown:

• macosx1015-64-qr: 1
  • debug: 1
• windows11-64-2009-asan-qr: 13
  • opt: 13
• windows11-64-2009-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-08-10&endday=2023-08-10&tree=all

Comment 58

[Intermittent Failures Robot]

16 failures in 493 pushes (0.032 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 6
• mozilla-central: 3
• try: 2
• mozilla-beta: 4
• mozilla-release: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 15
  • opt: 15
• windows11-64-2009-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-08-11&endday=2023-08-11&tree=all

Comment 59

[Intermittent Failures Robot]

86 failures in 3341 pushes (0.026 failures/push) were associated with this bug in the last 7 days.

This is the #8 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 32
• mozilla-central: 22
• try: 9
• mozilla-beta: 19
• mozilla-release: 3
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 78
  • opt: 78
• windows11-64-2009-ccov-qr: 7
  • opt: 7
• macosx1015-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-08-07&endday=2023-08-13&tree=all

Comment 60

[Intermittent Failures Robot]

18 failures in 608 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 5
• mozilla-central: 2
• try: 2
• mozilla-beta: 4
• mozilla-esr115: 3
• mozilla-release: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 18
  • opt: 18

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-08-15&endday=2023-08-15&tree=all

Comment 61

[Intermittent Failures Robot]

15 failures in 732 pushes (0.02 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 8
• mozilla-central: 2
• try: 1
• mozilla-beta: 3
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 15
  • opt: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-08-16&endday=2023-08-16&tree=all

Comment 62

[Intermittent Failures Robot]

78 failures in 3076 pushes (0.025 failures/push) were associated with this bug in the last 7 days.

This is the #7 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 29
• mozilla-central: 17
• try: 9
• mozilla-beta: 13
• mozilla-esr115: 7
• mozilla-release: 3

Platform and build breakdown:

• windows11-64-2009-asan-qr: 74
  • opt: 74
• windows11-64-2009-ccov-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-08-14&endday=2023-08-20&tree=all

Comment 63

[Natalia Csoregi [:nataliaCs]]

Update

There have been 46 failures within the last 7 days:

• 42 failures on Windows 11 x64 22H2 asan WebRender opt
• 4 failures on Windows 11 x64 22H2 CCov WebRender opt

Recent log: https://treeherder.mozilla.org/logviewer?job_id=427131282&repo=autoland&lineNumber=12746

Comment 64

[Intermittent Failures Robot]

60 failures in 2218 pushes (0.027 failures/push) were associated with this bug in the last 7 days.

This is the #9 most frequent failure this week.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 29
• mozilla-central: 19
• try: 8
• mozilla-release: 3
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 54
  • opt: 54
• windows11-64-2009-ccov-qr: 5
  • opt: 5
• windows11-64-2009-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-08-21&endday=2023-08-27&tree=all

Comment 65

[Intermittent Failures Robot]

73 failures in 3019 pushes (0.024 failures/push) were associated with this bug in the last 7 days.

This is the #6 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 18
• try: 8
• autoland: 32
• mozilla-beta: 13
• mozilla-esr115: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 67
  • opt: 67
• windows11-64-2009-ccov-qr: 3
  • opt: 3
• windows11-64-2009-qr: 2
  • debug: 2
• windows11-32-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-08-28&endday=2023-09-03&tree=all

Comment 66

[Intermittent Failures Robot]

18 failures in 634 pushes (0.028 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 10
• mozilla-central: 3
• mozilla-beta: 3
• try: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 15
  • opt: 15
• windows11-64-2009-qr: 3
  • debug: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-05&endday=2023-09-05&tree=all

Comment 67

[Intermittent Failures Robot]

16 failures in 724 pushes (0.022 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 8
• mozilla-central: 2
• mozilla-beta: 5
• try: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 14
  • opt: 14
• windows11-64-2009-qr: 2
  • debug: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-06&endday=2023-09-06&tree=all

Comment 68

[Intermittent Failures Robot]

25 failures in 653 pushes (0.038 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 10
• mozilla-central: 4
• mozilla-beta: 4
• mozilla-esr115: 5
• try: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 19
  • opt: 19
• windows11-64-2009-qr: 5
  • opt: 1
  • debug: 4
• windows11-64-2009-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-07&endday=2023-09-07&tree=all

Comment 69

[Intermittent Failures Robot]

29 failures in 682 pushes (0.043 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 6
• autoland: 10
• mozilla-beta: 4
• mozilla-esr115: 5
• try: 3
• mozilla-release: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 20
  • opt: 20
• windows11-64-2009-ccov-qr: 1
  • opt: 1
• windows11-64-2009-qr: 8
  • debug: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-08&endday=2023-09-08&tree=all

Comment 70

[Intermittent Failures Robot]

119 failures in 3267 pushes (0.036 failures/push) were associated with this bug in the last 7 days.

This is the #4 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 53
• mozilla-central: 21
• try: 11
• mozilla-beta: 22
• mozilla-esr115: 11
• mozilla-release: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 96
  • opt: 96
• windows11-64-2009-qr: 20
  • debug: 19
  • opt: 1
• windows11-64-2009-ccov-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-04&endday=2023-09-10&tree=all

Comment 71

[Intermittent Failures Robot]

18 failures in 545 pushes (0.033 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 9
• mozilla-central: 3
• try: 2
• mozilla-release: 2
• mozilla-beta: 1
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-qr: 2
  • debug: 2
• windows11-64-2009-asan-qr: 16
  • opt: 16

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-11&endday=2023-09-11&tree=all

Comment 72

[Intermittent Failures Robot]

25 failures in 559 pushes (0.045 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 6
• mozilla-beta: 4
• mozilla-release: 1
• mozilla-esr115: 4
• autoland: 8
• try: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 22
  • opt: 22
• windows11-64-2009-qr: 2
  • debug: 2
• windows11-64-2009-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-12&endday=2023-09-12&tree=all

Comment 73

[Intermittent Failures Robot]

21 failures in 697 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 6
• mozilla-central: 5
• try: 3
• mozilla-beta: 5
• mozilla-esr115: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 20
  • opt: 20
• windows11-64-2009-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-14&endday=2023-09-14&tree=all

Comment 74

[Intermittent Failures Robot]

16 failures in 697 pushes (0.023 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 9
• mozilla-central: 3
• try: 1
• mozilla-beta: 2
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 14
  • opt: 14
• windows11-32-2009-qr: 1
  • opt: 1
• windows11-64-2009-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-15&endday=2023-09-15&tree=all

Comment 75

[Intermittent Failures Robot]

137 failures in 3337 pushes (0.041 failures/push) were associated with this bug in the last 7 days.

This is the #4 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 49
• mozilla-central: 25
• try: 31
• mozilla-release: 3
• mozilla-beta: 18
• mozilla-esr115: 11

Platform and build breakdown:

• windows11-64-2009-qr: 4
  • debug: 4
• windows11-64-2009-asan-qr: 128
  • opt: 128
• windows11-64-2009-ccov-qr: 4
  • opt: 4
• windows11-32-2009-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-11&endday=2023-09-17&tree=all

Comment 76

[Intermittent Failures Robot]

15 failures in 653 pushes (0.023 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 5
• mozilla-beta: 3
• mozilla-central: 4
• mozilla-esr115: 1
• try: 1
• mozilla-release: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 13
  • opt: 13
• windows11-64-2009-ccov-qr: 1
  • opt: 1
• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-18&endday=2023-09-18&tree=all

Comment 77

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Attachment: Bug 1776767 - disabled GeckoMediaPlugins.CDMStorageBasic on Windows asan for permanent failures r=jmathies

The test fails permanently in the Windows ASan configuration since bug 1841352
landed. With no fix in sight, this should be disabled until fixed.

Comment 78

[Intermittent Failures Robot]

70 failures in 3850 pushes (0.018 failures/push) were associated with this bug in the last 7 days.

This is the #10 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 34
• mozilla-beta: 3
• mozilla-central: 21
• mozilla-esr115: 1
• try: 10
• mozilla-release: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 65
  • opt: 65
• windows11-64-2009-ccov-qr: 3
  • opt: 3
• windows11-32-2009-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-18&endday=2023-09-24&tree=all

Comment 79

[Intermittent Failures Robot]

16 failures in 753 pushes (0.021 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 6
• mozilla-central: 6
• mozilla-beta: 2
• try: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 14
  • opt: 14
• windows11-64-2009-qr: 1
  • debug: 1
• windows11-64-2009-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-26&endday=2023-09-26&tree=all

Comment 80

[Intermittent Failures Robot]

24 failures in 703 pushes (0.034 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 10
• mozilla-beta: 5
• mozilla-central: 4
• try: 3
• mozilla-release: 1
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 21
  • opt: 21
• windows11-32-2009-qr: 2
  • debug: 2
• windows11-64-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-27&endday=2023-09-27&tree=all

Comment 81

[Intermittent Failures Robot]

25 failures in 720 pushes (0.035 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-esr115: 5
• autoland: 8
• mozilla-central: 4
• mozilla-beta: 5
• try: 2
• mozilla-release: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 23
  • opt: 23
• windows11-64-2009-ccov-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-28&endday=2023-09-28&tree=all

Comment 82

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8068f7fac7d4
disabled GeckoMediaPlugins.CDMStorageBasic on Windows asan for permanent failures r=media-playback-reviewers,alwu

Comment 83

[Intermittent Failures Robot]

16 failures in 537 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-esr115: 3
• autoland: 5
• mozilla-beta: 3
• mozilla-central: 2
• try: 3

Platform and build breakdown:

• windows11-64-2009-asan-qr: 15
  • opt: 15
• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-29&endday=2023-09-29&tree=all

Comment 84

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/8068f7fac7d4

Comment 85

[Cristian Tuns]

This is still happening on windows asan: https://treeherder.mozilla.org/logviewer?job_id=430864782&repo=mozilla-central

Comment 86

[Natalia Csoregi [:nataliaCs]]

Seems to be permafailing: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&fromchange=8068f7fac7d44e5f28fec14686fc33ad260de99d&searchStr=windows%2C11%2Cx64%2C22h2%2Casan%2Cwebrender%2Copt%2Ctest-windows11-64-2009-asan-qr%2Fopt-gtest-1proc%2Cgtest-1proc&tochange=8c9ba927121cd7f1cb5b31ffb86addc4c34b59a8

Aryx, do you have a few minutes to look into this?
Thank you.

Comment 87

[Intermittent Failures Robot]

25 failures in 118 pushes (0.212 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 14
• mozilla-beta: 7
• mozilla-central: 3
• try: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 25
  • opt: 25

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-30&endday=2023-09-30&tree=all

Comment 88

[Intermittent Failures Robot]

154 failures in 3726 pushes (0.041 failures/push) were associated with this bug in the last 7 days.

This is the #3 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 78
• try: 14
• mozilla-central: 24
• mozilla-beta: 25
• mozilla-esr115: 11
• mozilla-release: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 146
  • opt: 146
• windows11-64-2009-qr: 2
  • debug: 2
• windows11-64-2009-ccov-qr: 3
  • opt: 3
• windows11-32-2009-qr: 2
  • debug: 2
• windows11-32-2009-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-09-25&endday=2023-10-01&tree=all

Comment 89

[Intermittent Failures Robot]

15 failures in 700 pushes (0.021 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 5
• mozilla-beta: 2
• mozilla-central: 5
• try: 2
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 13
  • opt: 13
• windows11-64-2009-ccov-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-03&endday=2023-10-03&tree=all

Comment 90

[Intermittent Failures Robot]

19 failures in 836 pushes (0.023 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 7
• mozilla-beta: 6
• mozilla-central: 3
• try: 2
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 19
  • opt: 19

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-04&endday=2023-10-04&tree=all

Comment 91

[Intermittent Failures Robot]

20 failures in 754 pushes (0.027 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 9
• mozilla-central: 4
• mozilla-beta: 4
• try: 2
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 18
  • opt: 18
• windows11-64-2009-ccov-qr: 1
  • opt: 1
• macosx1015-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-05&endday=2023-10-05&tree=all

Comment 92

[Intermittent Failures Robot]

17 failures in 728 pushes (0.023 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 7
• mozilla-beta: 3
• mozilla-central: 2
• try: 4
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 15
  • opt: 15
• windows11-32-2009-qr: 1
  • debug: 1
• windows11-64-2009-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-06&endday=2023-10-06&tree=all

Comment 93

[Intermittent Failures Robot]

109 failures in 4018 pushes (0.027 failures/push) were associated with this bug in the last 7 days.

This is the #9 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 39
• mozilla-beta: 22
• mozilla-central: 23
• try: 16
• mozilla-release: 4
• mozilla-esr115: 5

Platform and build breakdown:

• windows11-64-2009-asan-qr: 101
  • opt: 101
• macosx1015-64-shippable-qr: 1
  • opt: 1
• windows11-64-2009-ccov-qr: 5
  • opt: 5
• macosx1015-64-qr: 1
  • opt: 1
• windows11-32-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-02&endday=2023-10-08&tree=all

Comment 94

[Intermittent Failures Robot]

22 failures in 618 pushes (0.036 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 6
• mozilla-beta: 3
• mozilla-central: 7
• try: 2
• mozilla-release: 2
• mozilla-esr115: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 18
  • opt: 18
• windows11-64-2009-ccov-qr: 3
  • opt: 3
• windows11-64-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-09&endday=2023-10-09&tree=all

Comment 95

[Intermittent Failures Robot]

19 failures in 760 pushes (0.025 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 8
• mozilla-beta: 4
• mozilla-central: 4
• try: 2
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 16
  • opt: 16
• windows11-64-2009-ccov-qr: 2
  • opt: 2
• windows11-64-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-10&endday=2023-10-10&tree=all

Comment 96

[Intermittent Failures Robot]

17 failures in 768 pushes (0.022 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 9
• mozilla-beta: 3
• mozilla-central: 3
• try: 1
• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 16
  • opt: 16
• windows11-64-2009-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-11&endday=2023-10-11&tree=all

Comment 97

[Intermittent Failures Robot]

20 failures in 766 pushes (0.026 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-beta: 4
• autoland: 9
• mozilla-central: 2
• try: 2
• mozilla-release: 1
• mozilla-esr115: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 19
  • opt: 19
• windows11-64-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-12&endday=2023-10-12&tree=all

Comment 98

[Intermittent Failures Robot]

20 failures in 638 pushes (0.031 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 3
• mozilla-central: 5
• mozilla-beta: 3
• autoland: 7
• mozilla-esr115: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 19
  • opt: 19
• windows11-64-2009-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-13&endday=2023-10-13&tree=all

Comment 99

[Intermittent Failures Robot]

116 failures in 3764 pushes (0.031 failures/push) were associated with this bug in the last 7 days.

This is the #8 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 46
• mozilla-beta: 20
• mozilla-central: 26
• try: 12
• mozilla-release: 3
• mozilla-esr115: 9

Platform and build breakdown:

• windows11-64-2009-asan-qr: 105
  • opt: 105
• windows11-64-2009-ccov-qr: 8
  • opt: 8
• windows11-64-2009-qr: 3
  • debug: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-09&endday=2023-10-15&tree=all

Comment 100

[Intermittent Failures Robot]

19 failures in 631 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 6
• mozilla-beta: 4
• mozilla-central: 3
• try: 2
• mozilla-esr115: 2
• mozilla-release: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 19
  • opt: 19

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-16&endday=2023-10-16&tree=all

Comment 101

[Ryan VanderMeulen [:RyanVM]]

The patch should be checking for MOZ_ASAN rather than ASAN. Fixing that makes CDMStorageForgetThisSite start failing, though.

Comment 102

[Ryan VanderMeulen [:RyanVM]]

So far, every test I disable just pushes the failures to the next test. I'm inclined to just skip all CDMStorageTests on Windows ASAN.

Comment 103

[Ryan VanderMeulen [:RyanVM]]

...which in turn is just pushing the failures to GeckoMediaPlugins.GMPTestCodec

Comment 104

[Intermittent Failures Robot]

15 failures in 848 pushes (0.018 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 7
• mozilla-central: 4
• try: 2
• elm: 2

Platform and build breakdown:

• windows11-64-2009-asan-qr: 14
  • opt: 14
• windows11-64-2009-ccov-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-18&endday=2023-10-18&tree=all

Comment 105

[:gerard-majax]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #102)

So far, every test I disable just pushes the failures to the next test. I'm inclined to just skip all CDMStorageTests on Windows ASAN.

TBH, the ASAN behavior would just be coherent with the rest of what I investigated back then where there was a huge latency on doing things early in the startup process and that we could not figure out, leading to the problem. Given how ASAN can slow things, it would likely trigger the same behavior. Disabling tests and failing on the next one just confirms this, since we just postpone the time we need to start things and they get slowish.

Comment 106

[Ryan VanderMeulen [:RyanVM]]

(In reply to :gerard-majax from comment #105)

(In reply to Ryan VanderMeulen [:RyanVM] from comment #102)

So far, every test I disable just pushes the failures to the next test. I'm inclined to just skip all CDMStorageTests on Windows ASAN.

TBH, the ASAN behavior would just be coherent with the rest of what I investigated back then where there was a huge latency on doing things early in the startup process and that we could not figure out, leading to the problem. Given how ASAN can slow things, it would likely trigger the same behavior. Disabling tests and failing on the next one just confirms this, since we just postpone the time we need to start things and they get slowish.

Yep, makes sense. Guess I'll just keep hammering at this on Try until I skip my way to victory.

Comment 107

[Ryan VanderMeulen [:RyanVM]]

Attachment: Bug 1776767 - Skip failing GMP tests on Windows ASAN. r=alwu

Until we can resolve the underlying bug which is causing GMP launch
slowdowns on Windows ASAN builds, just skip the impacted tests.

Comment 108

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ff0c4e0dc579
Skip failing GMP tests on Windows ASAN. r=alwu

Comment 109

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/ff0c4e0dc579

Comment 110

[Intermittent Failures Robot]

16 failures in 800 pushes (0.02 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 3
• autoland: 8
• try: 3
• elm: 1
• mozilla-release: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 15
  • opt: 15
• windows11-64-2009-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-19&endday=2023-10-19&tree=all

Comment 111

[Intermittent Failures Robot]

63 failures in 3484 pushes (0.018 failures/push) were associated with this bug in the last 7 days.

This is the #14 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 29
• mozilla-central: 13
• mozilla-beta: 4
• try: 9
• mozilla-esr115: 2
• mozilla-release: 3
• elm: 3

Platform and build breakdown:

• windows11-64-2009-asan-qr: 61
  • opt: 61
• windows11-64-2009-ccov-qr: 1
  • opt: 1
• windows11-64-2009-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-16&endday=2023-10-22&tree=all

Comment 112

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-esr115/rev/cfc3ea85cd56

Comment 113

[Intermittent Failures Robot]

1 failures in 3657 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr115: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1776767&startday=2023-10-23&endday=2023-10-29&tree=all

Comment 114

[Joel Maher ( :jmaher ) (UTC -8)]

as per bug 1860717, the failures are on win/ccov. I am seeing this as a perma fail while migrating win/ccov tests from 22H2 -> 24H2.

I find that skipping each test until I skipped all the win/asan tests on ccov as well is what was needed to get green.

Comment 115

[Joel Maher ( :jmaher ) (UTC -8)]

:alwu, given that do you have concerns with skipping those tests on ccov?

Comment 116

[Alastor Wu [:alwu]]

I'm good to skip them.

Comment 117

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: Bug 1776767 - Skip some gtests on win11/24h2 ccov. r=alwu

Comment 118

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ab9ac5b46a26
Skip some gtests on win11/24h2 ccov. r=alwu

Comment 119

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/ab9ac5b46a26

Comment 120

[BugBot [:suhaib / :marco/ :calixte]]

A patch has been attached on this bug, which was already closed. Filing a separate bug will ensure better tracking. If this was not by mistake and further action is needed, please alert the appropriate party. (Or: if the patch doesn't change behavior -- e.g. landing a test case, or fixing a typo -- then feel free to disregard this message)