1558966 - Intermittent /encrypted-media/clearkey-mp4-playback-temporary-multikey-sequential-readyState.https.html | org.w3.clearkey, successful playback, temporary, mp4, multiple keys, sequential, readyState - assert_equals: It should require all keys to reach end

Status: RESOLVED INCOMPLETE

(Core :: Audio/Video: Playback, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: apavel [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=251475240&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/OpO7Hl5MSXyykmDaO75Szg/runs/0/artifacts/public/logs/live_backing.log

---

20:26:50 INFO - TEST-START | /encrypted-media/clearkey-mp4-playback-temporary-multikey-sequential-readyState.https.html
20:26:50 INFO - Closing window 34359738369
20:26:50 INFO - PID 10988 | [Child 3272, Chrome_ChildThread] WARNING: pipe error: 109: file z:/task_156[GMP 10800, Chrome_ChildThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:50 INFO - PID 10988 | [Parent 7272, Gecko_IOThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:50 INFO - PID 10988 | [Child 11244, Chrome_ChildThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:52 INFO -
20:26:52 INFO - TEST-UNEXPECTED-FAIL | /encrypted-media/clearkey-mp4-playback-temporary-multikey-sequential-readyState.https.html | org.w3.clearkey, successful playback, temporary, mp4, multiple keys, sequential, readyState - assert_equals: It should require all keys to reach end of content expected 2 but got 1
20:26:52 INFO - onTimeupdate@https://web-platform.test:8443/encrypted-media/scripts/playback-temporary-multikey-sequential.js:95:17
20:26:52 INFO - Test.prototype.step@https://web-platform.test:8443/resources/testharness.js:1594:25
20:26:52 INFO - Test.prototype.step_func/<@https://web-platform.test:8443/resources/testharness.js:1618:35
20:26:52 INFO - EventListener.handleEventrunTest/</<@https://web-platform.test:8443/encrypted-media/scripts/playback-temporary-multikey-sequential.js:110:20
20:26:52 INFO - promise callbackrunTest/<@https://web-platform.test:8443/encrypted-media/scripts/playback-temporary-multikey-sequential.js:107:12
20:26:52 INFO - Test.prototype.step@https://web-platform.test:8443/resources/testharness.js:1594:25
20:26:52 INFO - async_test@https://web-platform.test:8443/resources/testharness.js:576:22
20:26:52 INFO - runTest@https://web-platform.test:8443/encrypted-media/scripts/playback-temporary-multikey-sequential.js:17:5
20:26:52 INFO - @https://web-platform.test:8443/encrypted-media/clearkey-mp4-playback-temporary-multikey-sequential-readyState.https.html:52:9
20:26:52 INFO - TEST-OK | /encrypted-media/clearkey-mp4-playback-temporary-multikey-sequential-readyState.https.html | took 2159ms
20:26:52 INFO - PID 10988 | [Child 11244, Chrome_ChildThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/1560371212653 Marionette INFO Stopped listening on port 2828
20:26:52 INFO - PID 10988 | [GMP 8292, Chrome_ChildThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:52 INFO - PID 10988 | [GMP 8292, Chrome_ChildThread] WARNING: pipe erro[Parent 7272, Gecko_IOThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:52 INFO - PID 10988 | [Child 6104, Chrome_ChildThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:52 INFO - PID 10988 | [Child 6104, Chrome_ChildThread] WARNING: [Parent 7272, Gecko_IOThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:52 INFO - PID 10988 | [Parent 7272, Gecko_IOThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:52 INFO - PID 10988 | [Child 9080, Chrome_ChildThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:52 INFO - PID 10988 | [Child 90[Child 10340, Chrome_ChildThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:52 INFO - PID 10988 | [[Parent 7272, Gecko_IOThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:52 INFO - PID 10988 | [Child 2472, Chrome_ChildThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:52 INFO - PID 10988 | [Child 2472, Chrome_ChildThread] WARNING: pipe erro[Parent 7272, Gecko_IOThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:52 INFO - PID 10988 | [Child 1716, Chrome_ChildThread] WARNING: pipe error: 109: file z:/task_1560362683/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
20:26:53 INFO - PID 10988 | [Child 1716, Chrome_ChildThread] WARNING: pipe error:[GPU 5524, Ch
20:26:53 INFO - Browser exited with return code 0
20:26:53 INFO - PROCESS LEAKS None
20:26:53 INFO - Closing logging queue
20:26:53 INFO - queue closed
20:26:53 INFO - Setting up ssl

Comment 1

[Intermittent Failures Robot]

19 failures in 951 pushes (0.02 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 7
• mozilla-central: 4
• mozilla-inbound: 6
• try: 2

Platform breakdown:

• windows7-32: 2
• windows7-32-shippable: 4
• windows10-64: 8
• windows10-64-shippable: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-06-13&endday=2019-06-13&tree=all

Comment 2

[Dorel Luca [:dluca]]

Retriggers show that this bug was introduced where Bug 1554699 landed here: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=windows%2C10%2Cx64%2Cshippable%2Copt%2Cweb%2Cplatform%2Ctests%2Ctest-windows10-64-shippable%2Fopt-web-platform-tests-e10s-7%2Cw%28wpt7%29&revision=89350b209bcecaa0f0ec46769647cee3f0d8084b

Can you please take a look at this bug?

Comment 3

[Intermittent Failures Robot]

52 failures in 4753 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #33 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 24
• mozilla-central: 9
• mozilla-inbound: 13
• try: 6

Platform breakdown:

• windows10-64-shippable: 24
• windows10-64: 16
• windows7-32: 3
• windows7-32-shippable: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-06-10&endday=2019-06-16&tree=all

Comment 4

[Andreas Pehrson [:pehrsons]]

The only patches in bug 1554699 I see that could cause this are [1] and [2].
They both made HTMLMediaElement get up to spec on its play algorithm, by not firing some "timeupdate" events anymore. Does this sound reasonable? Is there a latent bug here?

Bryce, would you be the right person to look at this?

[1] https://hg.mozilla.org/mozilla-central/rev/7a6d1ef04809
[2] https://hg.mozilla.org/mozilla-central/rev/e0a7360fcc7c

Comment 5

[Intermittent Failures Robot]

16 failures in 614 pushes (0.026 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 5
• mozilla-inbound: 1
• autoland: 8
• try: 2

Platform breakdown:

• windows7-32: 4
• windows7-32-shippable: 4
• windows10-64-shippable: 5
• windows10-64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-06-19&endday=2019-06-19&tree=all

Comment 6

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

EME related stuff sounds like me. Holding NI so I can investigate after the all hands.

Comment 7

[Intermittent Failures Robot]

60 failures in 2581 pushes (0.023 failures/push) were associated with this bug in the last 7 days.

This is the #31 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-inbound: 14
• autoland: 26
• try: 6
• mozilla-central: 14

Platform breakdown:

• windows10-64: 21
• windows7-32-shippable: 10
• windows10-64-shippable: 21
• windows7-32-devedition: 1
• windows7-32: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-06-17&endday=2019-06-23&tree=all

Comment 8

[Intermittent Failures Robot]

15 failures in 844 pushes (0.018 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• mozilla-inbound: 2
• mozilla-central: 6
• autoland: 7

Platform breakdown:

• windows10-64-shippable: 7
• windows10-64: 5
• windows7-32: 2
• windows7-32-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-06-26&endday=2019-06-26&tree=all

Comment 9

[Andrei Ciure[:aciure]]

There are 55 failures associated to this bug in the last 7 days. These are occurring on windows10-64 debug and windows10-64-shippable opt.

:bryce any updates on this?

Comment 10

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Not yet. I have a couple of higher priority issues I'm looking into at the moment, but this is still fairly high up my queue and I hope to get to it in the next week.

Comment 11

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Starting to look at this.

Notes

• Windows only failures (win7 + win10).
• Debug and opt failures.
• No luck reproing locally after an hour of soaking with --reapeat-until-unexpected.

Analysis

The test is failing on this check. The function containing the check is triggered by timeupdate events, which is interesting given comment 4.

The failed check is run if (_video.currentTime > config.duration) {, i.e. when the video is beyond a certain playback time as specified in the config (5 seconds). The failing check is asserting that the test has a session for every init data in the test file. It expects two, but in the cases we're failing we only have 1. Looking at the test it appears this would happen if the onWaitingForKey function is not being called -- this function looks like the only path to add the second session to the session list.

So if we fire timeupdate after the video has reached 5 seconds and we haven't yet got two sessions we fail. If we're firing less timeupdates it seems to me we should have less opportunities for the check we're failing to be run, which makes a new race unlikely. Andreas, do the changes that mean we fire less timeupdate events change the timing of any of the remaining events?

If this isn't a race, perhaps we're not firing waitingforkey as we should be? I'm currently tracing changes in the regressing bug to see if I can find anything like that.

It's not clear to me why this is Windows specific yet.

Comment 12

[Andreas Pehrson [:pehrsons]]

See comment 4, the regression seems to have happened because there's no longer a timeupdate dispatched from PlayInternal. No other events affected.

Comment 13

[Intermittent Failures Robot]

20 failures in 815 pushes (0.025 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 11
• try: 3
• mozilla-central: 4
• mozilla-inbound: 2

Platform breakdown:

• windows10-64-shippable: 6
• windows10-64: 9
• windows7-32-shippable: 4
• windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-06-28&endday=2019-06-28&tree=all

Comment 14

[Intermittent Failures Robot]

83 failures in 4409 pushes (0.019 failures/push) were associated with this bug in the last 7 days.

This is the #26 most frequent failure this week.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• mozilla-central: 20
• try: 12
• mozilla-inbound: 14
• autoland: 37

Platform breakdown:

• windows7-32: 5
• windows7-32-shippable: 17
• windows10-64-devedition: 2
• windows10-64-shippable: 31
• windows10-64: 27
• windows10-64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-06-24&endday=2019-06-30&tree=all

Comment 15

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

I think this is the result of bug 1554699 changing the internal state of HTMLMediaElement and in turn that altering our timing of the remaining timeupdates we fire. I.e. not only are we not firing timeupdate in certain cases following bug 1554699, we have modified the timing of the timeupdates we fire outside of those cases. Why this causes the bug is not yet clear.

I've done a number of try runs to try and track this down:

• Backing out the two changsets from comment 4 appears to fix the bug.
• If I then trim back that code by removing some of the other updates steps, as well as making the check only fire as if it were pereodic, but remove the guarantee the event will fire, we get can repro.
• If I relax the check so we're always firing the event and updating state -- like if the aPeriodic arg is set to false, then we appear to fix the bug.
• If we don't fire timeupdate but update internal state, we appear to fix the bug.

Since the bug is intermittent, it's possible that I've been lucky/unlucky, have not done enough runs to repro, and that the bug is not actually fixed in some of the above cases. Assuming that is not the case, the last run is interesting because it suggests the issue is not us firing events, but changing the internal state.

The only place that the state we're changing above is used is when we call FireTimeUpdate with the aPeriodic flag set to true. This happens in two places I can see:

• HTMLMediaElement::UpdateSrcStreamTime
• MeidaDecoder::FireTimeUpdate

So my hypothesis is that the in state and thus the change in timing around these is what is causing the intermittent.

Comment 16

[Andreas Pehrson [:pehrsons]]

It could be triggering some obscure, latent, race I guess.

Have you managed to repro with logs, e.g., MOZ_LOG=timestamp,nsMediaElement:5,nsMediaElementEvents:5 ?

Comment 17

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Moving the check that the failing test is doing on timeupdate to ended seemingly fixes the issue also and seems like a better way to detect that playback has ended than checking in timeupdate with a duration. However, it would be good to understand why this is happening before we update the test.

Can we set logs in the JS of the wpt such that we would get those logs on try? That would be the easiest way to confirm further.

Comment 18

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Try run where I overload the assert to do some logging. It looks like we're not ever firing waitingForKey which is a real head-scratcher.

Trimmed log from when the assert fires:

video.duration: 5.083333, video.currentTime: 5.077332, everWaitingForKey:false, waitingForKeyTime: 0

In local runs I see waitingForKey fired just before 2 seconds. So the test is not seeing this event, and we're also continuing to play the video to the end.

I'm going to retrigger some more test runs on the cases mentioned in comment 4 to see if we get any more fails with a bigger sample.

Comment 19

[Intermittent Failures Robot]

18 failures in 602 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 10
• mozilla-inbound: 4
• mozilla-central: 3
• try: 1

Platform breakdown:

• windows7-32-shippable: 3
• windows10-64: 7
• windows10-64-shippable: 5
• windows10-64-devedition: 1
• windows7-32: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-07-02&endday=2019-07-02&tree=all

Comment 20

[Andreas Pehrson [:pehrsons]]

(In reply to Bryce Seager van Dyk (:bryce) from comment #17)

Can we set logs in the JS of the wpt such that we would get those logs on try? That would be the easiest way to confirm further.

You should be able to set them by prefs:
logging.config.timestamp:true
logging.config.sync:true
logging.nsMediaElement:5
logging.nsMediaElementEvents:5

Comment 21

[Intermittent Failures Robot]

16 failures in 648 pushes (0.025 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 8
• mozilla-inbound: 5
• try: 1
• mozilla-central: 2

Platform breakdown:

• windows10-64: 8
• windows10-64-shippable: 6
• windows7-32-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-07-04&endday=2019-07-04&tree=all

Comment 22

[Andreea Pavel [:apavel]]

Attachment: Bug 1558966 - disabled test on win for frequent failures r=jmaher

Comment 23

[Intermittent Failures Robot]

74 failures in 3501 pushes (0.021 failures/push) were associated with this bug in the last 7 days.

This is the #14 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 38
• mozilla-beta: 1
• try: 8
• mozilla-central: 15
• mozilla-inbound: 12

Platform breakdown:

• windows10-64-shippable: 28
• windows7-32-shippable: 12
• windows10-64: 26
• windows10-64-devedition: 3
• windows7-32: 4
• windows7-32-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-07-01&endday=2019-07-07&tree=all

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fb8fbe6000f3
disabled test on win for frequent failures r=jmaher

Comment 25

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/fb8fbe6000f3

Comment 26

[Intermittent Failures Robot]

23 failures in 4710 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 5
• autoland: 3
• mozilla-inbound: 3
• mozilla-central: 4
• mozilla-beta: 8

Platform breakdown:

• windows10-64: 10
• windows10-64-shippable: 5
• windows7-32-shippable: 5
• windows7-32-devedition: 2
• windows10-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-07-08&endday=2019-07-14&tree=all

Comment 27

[Intermittent Failures Robot]

14 failures in 4298 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 13
• try: 1

Platform breakdown:

• windows10-64-devedition: 5
• windows10-64: 3
• windows10-64-shippable: 4
• windows7-32: 1
• windows7-32-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-07-15&endday=2019-07-21&tree=all

Comment 28

[Intermittent Failures Robot]

5 failures in 4608 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 5

Platform breakdown:

• windows10-64-devedition: 2
• windows10-64-shippable: 1
• windows7-32-devedition: 1
• windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-07-22&endday=2019-07-28&tree=all

Comment 29

[Intermittent Failures Robot]

9 failures in 4439 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 9

Platform breakdown:

• windows10-64-devedition: 2
• windows10-64-shippable: 3
• windows10-64: 2
• windows7-32-devedition: 1
• windows7-32-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-07-29&endday=2019-08-04&tree=all

Comment 30

[Intermittent Failures Robot]

7 failures in 4666 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 6
• try: 1

Platform breakdown:

• windows10-64: 2
• windows10-64-devedition: 2
• windows10-64-shippable: 2
• windows7-32-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-08-05&endday=2019-08-11&tree=all

Comment 31

[Intermittent Failures Robot]

13 failures in 4839 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 13

Platform breakdown:

• windows10-64-shippable: 4
• windows10-64-devedition: 3
• windows10-64: 2
• windows7-32-devedition: 3
• windows7-32-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-08-12&endday=2019-08-18&tree=all

Comment 32

[Intermittent Failures Robot]

7 failures in 5073 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 6
• try: 1

Platform breakdown:

• windows10-64-devedition: 5
• windows10-64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-08-19&endday=2019-08-25&tree=all

Comment 33

[Intermittent Failures Robot]

2 failures in 5047 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 2

Platform breakdown:

• windows7-32-shippable: 1
• windows10-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-08-26&endday=2019-09-01&tree=all

Comment 34

[Intermittent Failures Robot]

2 failures in 4530 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 2

Platform breakdown:

• windows10-64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-09-23&endday=2019-09-29&tree=all

Comment 35

[Intermittent Failures Robot]

1 failures in 4292 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform breakdown:

• windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-09-30&endday=2019-10-06&tree=all

Comment 36

[Intermittent Failures Robot]

3 failures in 3801 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 3

Platform breakdown:

• windows10-64-shippable: 1
• windows10-64: 1
• windows7-32-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1558966&startday=2019-10-07&endday=2019-10-13&tree=all

Comment 37

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.