1523923 - Intermittent Assertion failure: rv, at z:/build/build/src/nsprpub/pr/src/md/windows/w95thred.c:276

Status: RESOLVED FIXED

(Core :: WebVR, defect, P5)

Description

[Treeherder Bug Filer]

#[markdown(off)]
Filed by: rgurzau [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=224939040&repo=try

https://queue.taskcluster.net/v1/task/aRA7oopBSlyLKcqLLH4UIw/runs/0/artifacts/public/logs/live_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/aRA7oopBSlyLKcqLLH4UIw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

12:18:51 INFO - JavaScript error: resource://reftest/reftest.jsm, line 1558: NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIPropertyBag2.getPropertyAsAString]
12:18:51 INFO - [Parent 5612, Gecko_IOThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
12:18:51 INFO - [GPU 5032, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
12:18:51 INFO - ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
12:18:51 INFO - [GPU 5032, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
12:18:51 INFO - [GPU 5032, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
12:18:51 INFO - [Child 4700, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file z:/build/build/src/xpcom/threads/nsThread.cpp, line 869
12:18:51 INFO - nsStringStats
12:18:51 INFO - => mAllocCount: 6221
12:18:51 INFO - => mReallocCount: 0
12:18:51 INFO - => mFreeCount: 6221
12:18:51 INFO - => mShareCount: 4639
12:18:51 INFO - => mAdoptCount: 71
12:18:51 INFO - => mAdoptFreeCount: 71
12:18:51 INFO - => Process ID: 4700, Thread ID: 5740
12:18:52 INFO - 1548850732210 Marionette TRACE Received observer notification xpcom-will-shutdown
12:18:52 INFO - 1548850732210 Marionette INFO Stopped listening on port 2828
12:18:52 INFO - 1548850732211 Marionette DEBUG Remote service is inactive
12:18:52 INFO - [Parent 5612, Main Thread] WARNING: NS_ENSURE_TRUE(mDB) failed: file z:/build/build/src/netwerk/cache/nsDiskCacheDeviceSQL.cpp, line 1284
12:18:52 INFO - [GPU 5032, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
12:18:52 INFO - [GPU 5032, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
12:18:52 INFO - ###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
12:18:52 INFO - [Parent 5612, Gecko_IOThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
12:18:52 INFO - [Parent 5612, Gecko_IOThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
12:18:52 INFO - [GPU 5032, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
12:18:52 INFO - [GPU 5032, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
12:18:52 INFO - [GPU 5032, Compositor] WARNING: VRThread::Shutdown() may only be called from an XPCOM thread: file z:/build/build/src/gfx/vr/VRThread.cpp, line 57
12:18:52 INFO - Assertion failure: rv, at z:/build/build/src/nsprpub/pr/src/md/windows/w95thred.c:276
12:18:52 INFO - [Parent 5612, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file z:/build/build/src/xpcom/threads/nsThread.cpp, line 869
12:18:52 INFO - nsStringStats
12:18:52 INFO - => mAllocCount: 1238
12:18:52 INFO - => mReallocCount: 0
12:18:52 INFO - => mFreeCount: 1238
12:18:52 INFO - => mShareCount: 296
12:18:52 INFO - => mAdoptCount: 0
12:18:52 INFO - => mAdoptFreeCount: 0
12:18:52 INFO - => Process ID: 5032, Thread ID: 6620

Comment 1

[Intermittent Failures Robot]

5 failures in 2628 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 5

Platform breakdown:

• windows10-64: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-01-28&endday=2019-02-03&tree=all

Comment 2

[Ryan VanderMeulen [:RyanVM]]

Immediately prior to the assertion:
WARNING: VRThread::Shutdown() may only be called from an XPCOM thread: file z:/build/build/src/gfx/vr/VRThread.cpp, line 57

The timing of this bug lines up exactly with bug 1514417 landing (and I found this today after it was uplifted to Beta as another data point).

Kip, can you please take a look since dmu is on PTO?

Comment 3

[Daosheng Mu[:daoshengmu]]

I guess we should shutdown the VRThread (mSubmitThread) early when calling StopPresentation() instead of ~VRDisplayHost(). Sorry for I am away from my device to help verify.

Comment 4

[:kip (Kearwood Gilbert)]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #2)

Immediately prior to the assertion:
WARNING: VRThread::Shutdown() may only be called from an XPCOM thread: file z:/build/build/src/gfx/vr/VRThread.cpp, line 57

The timing of this bug lines up exactly with bug 1514417 landing (and I found this today after it was uplifted to Beta as another data point).

Kip, can you please take a look since dmu is on PTO?

I'll take this one. Thanks for the tip, Daosheng!

Comment 5

[Intermittent Failures Robot]

15 failures in 2658 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 11
• mozilla-beta: 4

Platform breakdown:

• windows10-64: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-02-04&endday=2019-02-10&tree=all

Comment 6

[Daosheng Mu[:daoshengmu]]

Take Bug 1310152 Comment 9 as the reference, I am wondering we might shutdown the VR thread while it is still running submit frame tasks. We can test it with ./mach test dom/vr/test/reftest/change_size.html --repeat=20

Comment 7

[Intermittent Failures Robot]

21 failures in 2832 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 7
• mozilla-beta: 14

Platform breakdown:

• windows10-64: 19
• windows10-64-qr: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-02-11&endday=2019-02-17&tree=all

Comment 8

[Cosmin Sabou [:CosminS]]

This looks to be near perma fails since this uplift to beta: https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&searchStr=windows%2C10%2Cx64%2Cdebug%2Creftests%2Cwith%2Ce10s%2Ctest-windows10-64%2Fdebug-reftest-e10s-1%2Cr-e10s%28r1%29&group_state=expanded&revision=e320050c6f958301665225d0e63d09e6df73eb91

Range: https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&searchStr=windows%2C10%2Cx64%2Cdebug%2Creftests%2Cwith%2Ce10s%2Ctest-windows10-64%2Fdebug-reftest-e10s-1%2Cr-e10s%28r1%29&group_state=expanded&tochange=bf56566e9cdbf447be4dfe4f59504f3d812e9441&fromchange=35e4f7994c010a09d9edeb3889b43508de71177e&selectedJob=228486363

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=228486363&repo=mozilla-beta&lineNumber=39408

Ed, could you take a look over this and see what could have caused this? Thank you.

Comment 9

[Ed Lee :Mardak]

I pushed to try the parent of that push:

No Bug, mozilla-beta repo-update HSTS HPKP remote-settings - a=repo-update r=RyanVM

and the assertion seems to trigger on R2 (instead of R1)

https://treeherder.mozilla.org/#/jobs?repo=try&revision=333f05551296a161307053278d3fdf87bed70fee

In the failure cases, the last test seems to be:

REFTEST TEST-END | file:///Z:/task_1550515971/build/tests/reftest/tests/dom/vr/test/reftest/change_size.html == file:///Z:/task_1550515971/build/tests/reftest/tests/dom/vr/test/reftest/wrapper.html?change_size.png

Comment 10

[Ed Lee :Mardak]

Looks like it was failing in R4 from https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=229138827&revision=1b92df3fb13a3bab6365213a6290c7b8b25dc98d originally pushed on the 16th, but looking back more, it looks like it was failing from at least the 7th:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&searchStr=10-64%2Fdebug-reftest-e10s-4&revision=d7e94ba9dbad1189eeb1fbb4ea1f456668b23381&selectedJob=226885498

That push does contain 9b74b2602df1 DM Bug 1514417 - Using named mutex for VR threads to access Shmem on Windows. r=kip, a=lizzard as mentioned in comment 2.

Comment 11

[Daosheng Mu[:daoshengmu]]

I am trying to use CancelableRunnable as VR submit frame tasks, and it looks better in local. I am waiting for the actual try results after the Treeherder be opened.

Comment 12

[Daosheng Mu[:daoshengmu]]

It looks like it would be resolved in Nightly after applying my WIP patch. :Markdak, could you help me verify it, https://treeherder.mozilla.org/#/jobs?repo=try&revision=8dfe4775de479bdd7732ff21d7b299b7c54c1bc8?

Thanks,

Comment 13

[Daosheng Mu[:daoshengmu]]

Attachment: Bug 1523923 - Cancel VR submit frame tasks when the VRDisplayHost is destroyed.

Comment 14

[Ed Lee :Mardak]

R4 on that try included reftest/tests/dom/vr/test/reftest/change_size.html so I retriggered a bunch and looks like they're all passing.

Comment 15

[:kip (Kearwood Gilbert)]

Assigning to Daosheng, who has a WIP patch for this issue.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/249fb9b3679b
Cancel VR submit frame tasks when the VRDisplayHost is destroyed. r=kip

Comment 17

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/249fb9b3679b

Comment 18

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/249fb9b3679b

Comment 19

[Ryan VanderMeulen [:RyanVM]]

Please request Beta approval on this when you get a chance.

Comment 20

[Intermittent Failures Robot]

37 failures in 2602 pushes (0.014 failures/push) were associated with this bug in the last 7 days.

This is the #34 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-beta: 11
• try: 26

Platform breakdown:

• windows10-64: 33
• windows10-64-qr: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-02-18&endday=2019-02-24&tree=all

Comment 21

[Daosheng Mu[:daoshengmu]]

Comment on attachment 9045699 [details]
Bug 1523923 - Cancel VR submit frame tasks when the VRDisplayHost is destroyed.

Beta/Release Uplift Approval Request

• Feature/Bug causing the regression: Bug 1514417
• User impact if declined: The intermittent assertion failure on Bug 1523923 will continue to happen
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): It is used to fix the intermittent assertion and has been verified in Nightly.
• String changes made/needed:

Comment 22

[Daosheng Mu[:daoshengmu]]

Comment on attachment 9045699 [details]
Bug 1523923 - Cancel VR submit frame tasks when the VRDisplayHost is destroyed.

We are investigating some new bugs that caused by this. I will give a uplift request later.

Comment 23

[Liz Henry (:lizzard) (relman/hg->git project)]

Given that we are getting closer to 66 release, might it be better to back this out in nightly?

Comment 24

[Daosheng Mu[:daoshengmu]]

(In reply to Liz Henry (:lizzard) (use needinfo) from comment #23)

Given that we are getting closer to 66 release, might it be better to back this out in nightly?

This haven't be uplift to beta, and we also have patches on our hands already, we will fix them soon.

Comment 25

[Arthur Iakab [arthur_iakab]]

Recent failure https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&revision=b1090ef7dc8bcaeb3ffdbc866ae074c744018a71&selectedJob=231041746&searchStr=windows%2C10%2Cx64%2Cdebug%2Creftests%2Cwith%2Ce10s%2Ctest-windows10-64%2Fdebug-reftest-e10s-4%2Cr-e10s%28r4%29

Log failure https://treeherder.mozilla.org/logviewer.html#?job_id=231041746&repo=mozilla-beta

Comment 26

[Daosheng Mu[:daoshengmu]]

(In reply to Arthur Iakab [arthur_iakab] from comment #25)

Recent failure https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&revision=b1090ef7dc8bcaeb3ffdbc866ae074c744018a71&selectedJob=231041746&searchStr=windows%2C10%2Cx64%2Cdebug%2Creftests%2Cwith%2Ce10s%2Ctest-windows10-64%2Fdebug-reftest-e10s-4%2Cr-e10s%28r4%29

Log failure https://treeherder.mozilla.org/logviewer.html#?job_id=231041746&repo=mozilla-beta

I am curious why we mark this as reopen? It resolves this issue in Nightly, and it just haven't been uplift to the Beta.

Comment 27

[Liz Henry (:lizzard) (relman/hg->git project)]

Correct, we count a bug as RESOLVED FIXED when it is fixed in the current nightly version. We depend on the status-firefoxXX:affected flags to settle what will happen on beta, release, and esr60 branches.

Looks like there was no request to bring it to beta, so I think comment 25 is just a mistake. I will mark this as fix-optional for 66, and please let me know if you think we need uplift to beta when you are ready.

Comment 28

[Intermittent Failures Robot]

15 failures in 2981 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 15

Platform breakdown:

• windows10-64: 12
• windows10-64-qr: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-02-25&endday=2019-03-03&tree=all

Comment 29

[Ryan VanderMeulen [:RyanVM]]

Given that next week is RC week for 66, we should probably just let this ride the trains at this point given the fallout from it.

Comment 30

[Bogdan Tara[:bogdan_tara | bogdant]]

:daoshengmu & :lizzard this happens on beta:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&fromchange=c4e950fe3fe672c62dd0163f45767ee22188aac1&tochange=99d77e9e30525eabcee95ea999c94a1671776fc0&selectedJob=231926478

Comment 31

[Liz Henry (:lizzard) (relman/hg->git project)]

Let's not uplift to beta :)

Comment 32

[Daosheng Mu[:daoshengmu]]

I am waiting for Bug 1530663's landing. Once Bug 1530663 is landed, I am going to consider to give a uplift for this bug.

Comment 33

[Stefan Hindli [:stefan_hindli]]

New failure in beta: https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&selectedJob=232670311

Log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=232670311&repo=mozilla-beta&lineNumber=67215

15:04:05 INFO - => Process ID: 9480, Thread ID: 11288
15:04:05 INFO - 1552057445227 Marionette TRACE Received observer notification xpcom-will-shutdown
15:04:05 INFO - 1552057445227 Marionette INFO Stopped listening on port 2828
15:04:05 INFO - 1552057445228 Marionette DEBUG Remote service is inactive
15:04:05 INFO - [Parent 864, Main Thread] WARNING: NS_ENSURE_TRUE(mDB) failed: file z:/build/build/src/netwerk/cache/nsDiskCacheDeviceSQL.cpp, line 1284
15:04:05 INFO - [GPU 4632, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
15:04:05 INFO - ###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
15:04:05 INFO - [GPU 4632, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
15:04:05 INFO - [Parent 864, Gecko_IOThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
15:04:05 INFO - [GPU 4632, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
15:04:05 INFO - [GPU 4632, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 332
15:04:05 INFO - [GPU 4632, Compositor] WARNING: VRThread::Shutdown() may only be called from an XPCOM thread: file z:/build/build/src/gfx/vr/VRThread.cpp, line 57
15:04:05 INFO - Assertion failure: rv, at z:/build/build/src/nsprpub/pr/src/md/windows/w95thred.c:276
15:04:05 INFO - [Parent 864, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file z:/build/build/src/xpcom/threads/nsThread.cpp, line 869
15:04:05 INFO - nsStringStats
15:04:05 INFO - => mAllocCount: 1221
15:04:05 INFO - => mReallocCount: 0
15:04:05 INFO - => mFreeCount: 1221
15:04:05 INFO - => mShareCount: 292
15:04:05 INFO - => mAdoptCount: 0
15:04:05 INFO - => mAdoptFreeCount: 0
15:04:05 INFO - => Process ID: 4632, Thread ID: 10104

Comment 34

[Ryan VanderMeulen [:RyanVM]]

Beta is 66, not 67. This patch wasn't backported there.

Comment 35

[Intermittent Failures Robot]

23 failures in 516 pushes (0.045 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-beta: 23

Platform breakdown:

• windows10-64: 22
• windows10-64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-03-08&endday=2019-03-08&tree=all

Comment 36

[Intermittent Failures Robot]

37 failures in 2818 pushes (0.013 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-beta: 37

Platform breakdown:

• windows10-64: 36
• windows10-64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-03-04&endday=2019-03-10&tree=all

Comment 37

[Daosheng Mu[:daoshengmu]]

:lizzard, we are considering to uplift this bug fix to FF 66, and it needs to uplift its dependent bugs,
Bug 1530489, Bug 1530663, and Bug 1534390 too. I know it is a little late to take the train FF 66, but it is probably worth because it will resolve a few try fails and the stability problem at the current mozilla-beta channel. If you are ok, I will give a request soon.

Comment 38

[Liz Henry (:lizzard) (relman/hg->git project)]

It is too late for 66, We built the release candidate yesterday, and while we are doing one more build this looks too complex for the last minute. I think it needs time on 67 (which will be on mozilla-beta next Monday)

Comment 39

[Intermittent Failures Robot]

6 failures in 2771 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 6

Platform breakdown:

• windows10-64: 4
• windows10-64-qr: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-03-11&endday=2019-03-17&tree=all

Comment 40

[Intermittent Failures Robot]

2 failures in 2561 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 2

Platform breakdown:

• windows10-64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-03-18&endday=2019-03-24&tree=all

Comment 41

[Intermittent Failures Robot]

2 failures in 3910 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 2

Platform breakdown:

• windows10-64-qr: 1
• windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-03-25&endday=2019-03-31&tree=all

Comment 42

[Intermittent Failures Robot]

7 failures in 2843 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 7

Platform breakdown:

• windows10-64: 5
• windows10-64-qr: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-04-01&endday=2019-04-07&tree=all

Comment 43

[Intermittent Failures Robot]

3 failures in 3025 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 3

Platform breakdown:

• windows10-64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-04-08&endday=2019-04-14&tree=all

Comment 44

[Intermittent Failures Robot]

4 failures in 3031 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 4

Platform breakdown:

• windows10-64: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-04-22&endday=2019-04-28&tree=all

Comment 45

[Intermittent Failures Robot]

3 failures in 4131 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 3

Platform breakdown:

• windows10-64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-04-29&endday=2019-05-05&tree=all

Comment 46

[Intermittent Failures Robot]

3 failures in 4705 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 3

Platform breakdown:

• windows10-64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-05-06&endday=2019-05-12&tree=all

Comment 47

[Intermittent Failures Robot]

1 failures in 4867 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1523923&startday=2019-12-09&endday=2019-12-15&tree=all