Closed Bug 1434983 Opened 6 years ago Closed 6 years ago

Intermittent dom/media/tests/mochitest/test_peerConnection_basicScreenshare.html | Error in test execution: Error: Timeout checking for stats for track {d052d795-95ae-084a-871c-e234ceec9b38} after at least30000ms waitForRtpFlow@

Categories

(Core :: WebRTC: Audio/Video, defect, P5)

Product:
Core
Component:
WebRTC: Audio/Video
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla62
Tracking Flags:
Tracking Status
firefox-esr60 --- fixed
firefox60 --- wontfix
firefox61 --- fixed
firefox62 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: bwc)

References

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

Bug 1434983: Make screenshare and windowshare test one-way, to work around limitations on OS X.
59 bytes, text/x-review-board-request
pehrsons
: review+
Details 
When this happens, we get a raft of other failures afterward. It seems that something in this test breaks some part of Firefox in a way that is not recoverable. No outbound or inbound RTP for video seems to be happening. It does not seem to matter whether we're using fake video or not. Maybe we're deadlocking somewhere in the video path?

 17843 06:30:19 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_basicScreenshare.html | Error in test execution: Error: Timeout checking for stats for track {17fa857c-b0cc-f44d-8305-633a3ddbf6dc} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest
18256 06:30:50 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_basicVideo.html | Error in test execution: Error: Timeout checking for stats for track {f4633628-ac47-2047-9417-dfd463a62345} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js
18660 06:31:21 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_basicVideoVerifyRtpHeaderExtensions.html | Error in test execution: Error: Timeout checking for stats for track {dd8f1393-21fd-964f-bedd-08a70ae867be} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/me
19058 06:31:53 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_basicWindowshare.html | Error in test execution: Error: Timeout checking for stats for track {15e9a964-64a9-bc47-9d6f-6be63333c08f} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest
20327 06:32:58 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_captureStream_canvas_2d.html | Error in test execution: Error: Timeout for element _remote_{05d6cdaf-422c-7841-96d2-1e4b2f030e3c} waitForMediaElementFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1495:13 ... waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1599:22 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1595:7 ... PC_
20784 06:33:59 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_captureStream_canvas_2d_noSSRC.html | Error in test execution: Error: Timeout for element _remote_{b7aa1e33-41b8-c44b-814d-996aeffe25c4} waitForMediaElementFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1495:13 ... waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1599:22 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1595:7
20888 06:33:59 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_capturedVideo.html | seek.webm failed in playback (code=3). Stream should be OK. Continuing test.
22705 06:34:30 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_capturedVideo.html | Error in test execution: Error: Timeout checking for stats for track {29de5058-1d14-7845-a75a-8bb2f589bd2c} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1601:49 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc
24458 06:35:31 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_capturedVideo.html | Error in test execution: Error: Timeout for element _remote_{86bd4f5c-9bff-f841-a2b1-aeefb57ae0a0} waitForMediaElementFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1495:13 ... waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1599:22 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1595:7 ... PC_REMOTE_WAI
24985 06:35:53 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_checkPacketDumpHook.html | Error in test execution: Error: Timeout waiting for rtp recv packet on level 1 timeout/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/head.js:572:63 ... promise callback*timeout@http://mochi.test:8888/tests/dom/media/tests/mochitest/head.js:572:26 ... waitForRecvPacket@http://mochi.test:8888/tests/dom/media/tests/mochitest/test_peerConnection_checkPacketDumpHook.html:43:1
25805 06:36:26 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_constructedStream.html | Error in test execution: Error: Timeout checking for stats for track {91ffc767-2989-0f49-9007-6f3f6848c804} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1601:49 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochites
26546 06:36:58 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_forwarding_basicAudioVideoCombined.html | Error in test execution: Error: Timeout checking for stats for track {5ae36af0-15d6-1b49-923c-0c3cbdd86652} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1601:49 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/med
27588 06:37:29 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_forwarding_basicAudioVideoCombined.html | Error in test execution: Error: Timeout checking for stats for track {3e0e62c8-6d7c-cd4f-b28b-3f641d176b47} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1601:49 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/med
29358 06:38:44 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_multiple_captureStream_canvas_2d.html | Error in test execution: Error: Timeout for element _remote_{e171895f-20f3-fa43-a159-ee70cd379ac6} waitForMediaElementFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1495:13 ... waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1599:22 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1595:
30521 06:39:19 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_offerRequiresReceiveVideo.html | Error in test execution: Error: Timeout checking for stats for track {0793a9fb-b890-1343-9001-b0559b15c6e5} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/
31077 06:39:51 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_offerRequiresReceiveVideoAudio.html | Error in test execution: Error: Timeout checking for stats for track {58f6759c-8f76-ef4d-b5d9-26a9e869c1a4} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/t
33233 06:40:32 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_removeThenAddVideoTrack.html | Error in test execution: Error: Timeout checking for stats for track {9ca9c563-300c-4d45-ae40-abf342f689bc} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mo
33635 06:41:03 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_removeThenAddVideoTrackNoBundle.html | Error in test execution: Error: Timeout checking for stats for track {922df05f-ca48-2949-8da8-4b9536ec7ab5} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/
34039 06:41:34 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_removeVideoTrack.html | Error in test execution: Error: Timeout checking for stats for track {b5bbd8ba-70a9-7a49-a2f7-2386b4ae6a37} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest
34998 06:46:43 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_renderAfterRenegotiation.html | Test timed out.
35551 06:47:16 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_replaceTrack.html | Error in test execution: Error: Timeout checking for stats for track {592a83e4-40f8-504c-9290-ad147f54f211} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.
35990 06:47:47 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_replaceVideoThenRenegotiate.html | Error in test execution: Error: Timeout checking for stats for track {1eea32d9-bb5d-094d-8482-85be8c35384b} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/test
36623 06:48:18 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_restartIce.html | Error in test execution: Error: Timeout checking for stats for track {37afde50-4ec1-4545-9d17-f37c86b9a024} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js
38219 06:48:54 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_restartIceNoBundle.html | Error in test execution: Error: Timeout checking for stats for track {8627edf9-b888-d642-9035-320ace9ca77c} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochite
39080 06:49:25 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_restartIceNoBundleNoRtcpMux.html | Error in test execution: Error: Timeout checking for stats for track {f225fd7a-3755-274a-aa09-a9daeef07d0b} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/test
39814 06:49:57 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_restartIceNoRtcpMux.html | Error in test execution: Error: Timeout checking for stats for track {69ac3127-4145-8b4d-bedf-7382bad9d478} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochit
40237 06:50:28 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_restrictBandwidthTargetBitrate.html | Error in test execution: Error: Timeout checking for stats for track {f5a28027-2a71-454a-842e-9f6178818418} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/t
40640 06:50:59 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_restrictBandwidthWithTias.html | Error in test execution: Error: Timeout checking for stats for track {7867b4f3-1a07-db44-b324-da972dcf4a49} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/
41427 06:56:13 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_scaleResolution.html | Test timed out.
41961 06:56:46 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_sender_and_receiver_stats.html | Error in test execution: Error: Timeout checking for stats for track {d27694c4-38e1-d845-a76e-dfee55e74566} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1601:49 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/
42543 06:57:18 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_setParameters.html | Error in test execution: Error: Timeout checking for stats for track {0bc606d7-15ce-0345-befd-f08375d172a2} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc
43109 06:57:50 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_simulcastAnswer.html | Error in test execution: Error: Timeout checking for stats for track {6f685fd9-d76b-2848-a8b5-998e97367ec7} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1601:49 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/
43743 06:58:52 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_simulcastOffer.html | Error in test execution: Error: Timeout for element _remote_{ec892bb3-86af-3a48-9ab3-f6d029f04079} waitForMediaElementFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1495:13 ... waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1599:22 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1595:7 ... PC_REMOTE_WA
44371 06:59:23 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_stats.html | Error in test execution: Error: Timeout checking for stats for track {7dbf858f-cfc3-4342-aa3f-60c0474de02b} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600
44791 06:59:55 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_syncSetDescription.html | Error in test execution: Error: Timeout checking for stats for track {56e32192-7dcd-4649-b4fa-602e8407bf8e} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochite
45799 07:00:27 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_trackDisabling.html | Error in test execution: Error: Timeout checking for stats for track {06b34df2-d518-3e4f-829d-777dc1654451} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1601:49 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/p
46317 07:00:58 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_trackDisabling_clones.html | Error in test execution: Error: Timeout checking for stats for track {15572941-f014-8247-9059-dbb9f2abce25} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1601:49 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/moch
50149 07:06:13 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_transceivers.html | Test timed out.
51623 07:06:48 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_twoAudioVideoStreams.html | Error in test execution: Error: Timeout checking for stats for track {265afc0a-be5e-8d47-8f22-4c6ae18b1e5f} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochi
52650 07:07:19 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_twoAudioVideoStreamsCombined.html | Error in test execution: Error: Timeout checking for stats for track {a1bed664-2794-6947-aabf-0cece22ad311} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tes
53197 07:07:51 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_twoVideoStreams.html | Error in test execution: Error: Timeout checking for stats for track {cc320bab-eb90-1941-99fa-d55b198a7127} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/
53707 07:08:22 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_twoVideoTracksInOneStream.html | Error in test execution: Error: Timeout checking for stats for track {2cbaa23f-4687-5341-8f55-e5c84d641d75} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1600:58 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/
54621 07:09:28 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_verifyVideoAfterRenegotiation.html | Error in test execution: Error: Timeout for element _remote_{da9cc4ad-9364-db43-9a5c-5e581f72ef18} waitForMediaElementFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1495:13 ... waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1599:22 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1595:7 .
55003 07:09:59 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_videoCodecs.html | Error in test execution: Error: Timeout checking for stats for track {6d929b7f-1ac4-8044-8e2a-9f74a0bec4b8} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1601:49 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.j
55810 07:10:30 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_videoCodecs.html | Error in test execution: Error: Timeout checking for stats for track {c9b20606-b867-5442-84e6-3c3775d84bef} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1601:49 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.j
56604 07:11:01 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_videoCodecs.html | Error in test execution: Error: Timeout checking for stats for track {f7fa00c4-3b81-8a48-a84d-8220de7bd8bf} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1601:49 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.j
57403 07:11:32 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_videoCodecs.html | Error in test execution: Error: Timeout checking for stats for track {75e5398b-3dca-714a-8079-f38bf3167ef0} after at least30000ms waitForRtpFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1548:11 ... async*waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1601:49 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.j
57994 07:12:33 INFO - TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_videoRenegotiationInactiveAnswer.html | Error in test execution: Error: Timeout for element _remote_{be0a88d3-3f05-1849-a73a-496d0057f43e} waitForMediaElementFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1495:13 ... waitForMediaFlow/<@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1599:22 ... waitForMediaFlow@http://mochi.test:8888/tests/dom/media/tests/mochitest/pc.js:1595
I'm going to try to figure out what push started this mess. Sadly, that's going to require lots of OS X retriggers. Sorry in advance.
Component: WebRTC → WebRTC: Audio/Video
Assignee: nobody → docfaraday
It is very likely that this started with this push:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=145ff65a890becba6c258e88c88916f9fc00ef42
Flags: needinfo?(apehrson)
Oh fun. I'm not gonna have time to look at this for a while, but if you do I'm happy to provide some log modules for narrowing it down. It would also be interesting to know if it fails when run standalone on try.
Blocks: 1299515
In the failure case, I see IProtocol::AllocShmem start failing when the basicScreenshare test runs, and it never succeeds again.
Also notable is that the first time it fails, it takes a minute to do so, whereas subsequent failures happen basically right away.
I think the reason this is failing is because the screenshare size limit got larger in bug 1299515, and the larger buffers required are frequently causing IPC Shmem on OS X to break. I am trying to test this hypothesis now.
It does not seem to be caused by larger buffer sizes. IPC Shmem is breaking though, and never recovers. Trying to figure out why...
IPC Shmem is breaking here: https://searchfox.org/mozilla-central/source/ipc/glue/SharedMemoryBasic_mach.mm#633

I'm not seeing any glaring difference in terms of how many Shmems CamerasParent is creating, nor how large these Shmems tend to be.
I'm going to check if we are failing to destroy Shmems, or not destroying them quickly enough.
See Also: → 1423271
When comparing the pre-regression code to the post-regression code, I do not see larger shmems allocated, or more bytes of shmem allocated at a time, or a larger number of shmems allocated at a time. This does not seem to be caused by any kind of increased shmem usage/load. Perhaps we are exceeding some other system resource?
I've tried reducing the IPC timeout (which is how these failures happen) from 60 seconds to 10 on the pre-regression changeset, and it still works reliably.
Have you been able to look at this recently, Byron?
status-firefox60: --- → affected
status-firefox61: --- → affected
Flags: needinfo?(docfaraday)
I've been busy with the h264 related oranges lately, but now will have some time to look at this again.
Flags: needinfo?(docfaraday)
Depends on: 1452773
I have been unable to reproduce this problem on a loaner. I wonder if only certain machines have this problem?
Hmm, so, bug 1299515 (the suspected regressor) did change so two gUM requests to capture the screen will request two captures at the backend that get shipped to the child over IPC, instead of one at the backend that is split up in the child.

The test looks like it is using two separate gUM screen requests, so there should be higher churn of shmems than before.

Byron, thoughts on the way forward? 1) Make the test one-way or use/implement something that either 2) shares these shmems across processes with some sort of ipc-based refcounts, or 3) does the split in the child instead of the parent, more like before.

Note that for 3, we'd still not share images across multiple children, so that solution would result in more paths and complexity than before.
Flags: needinfo?(apehrson) → needinfo?(docfaraday)
1) Would certainly be worth a try, if the alternative is disabling the test.

2) IPC-based refcounts strikes me as difficult, but we may already have something like this.

3) This seems ideal from a performance perspective, but I do not know how ugly the resulting code would be. We should probably verify that this is indeed a performance problem, perhaps by implementing 1.
Flags: needinfo?(docfaraday)
Hmm, do we need to do the same for test_pc_basicWindowshare.html that comes directly after?

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4f329e25aa6c&selectedJob=180717840
See Also: → 1048455
Comment on attachment 8983400 [details]
Bug 1434983: Make screenshare and windowshare test one-way, to work around limitations on OS X.

https://reviewboard.mozilla.org/r/249314/#review255492
Attachment #8983400 - Flags: review?(apehrson) → review+
Pushed by bcampen@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0a06f4f9bd73
Make screenshare and windowshare test one-way, to work around limitations on OS X. r=pehrsons
https://hg.mozilla.org/mozilla-central/rev/0a06f4f9bd73
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox62: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: