Closed Bug 1101163 Opened 10 years ago Closed 9 years ago

Intermittent failures in Steeplechase WebRTC test suite

Categories

(Core :: WebRTC, defect, P4)

35 Branch
defect

Tracking

()

RESOLVED FIXED

People

(Reporter: sydpolk, Assigned: drno)

Details

Attachments

(2 files, 1 obsolete file)

The following tests fail intermittently in our Jenkins instance:

steeplechase INFO  | {"action":"test_unexpected_fail","message":"Have localcandidate stat(s)","time":1416342417052,"source_file":"dom/media/tests/mochitest/test_peerConnection_basicAudio.html"} - nightly-win64-aurora-linux64, aurora-linux64-nightly-win32, aurora-mac-nightly-win64, nightly-win32_64-aurora-linux64, nightly-win64-aurora-linux64

steeplechase INFO  | {"action":"test_unexpected_fail","message":"Have remotecandidate stat(s)","time":1416342417052,"source_file":"dom/media/tests/mochitest/test_peerConnection_basicAudio.html"} - nightly-win64-aurora-linux64, aurora-linux64-nightly-win32, aurora-mac-nightly-win64, nightly-win32_64-aurora-linux64, nightly-win64-aurora-linux64

steeplechase INFO  | {"action":"test_unexpected_fail","message":"pc_local: ICE failed to switch to 'connected' state: failed","time":1416320006028,"source_file":"dom/media/tests/mochitest/test_peerConnection_basicAudio.html"} - nightly-linux64-nightly-linux64-turn-udp-tcp, nightly-linux64-win32, nightly-mac-aurora-mac10_7_64, nightly-mac-mac10_7_64

steeplechase INFO  | {"action":"test_unexpected_fail","message":"pc_local: ICE failed to switch to 'connected' state: failed","time":1416320006028,"source_file":"dom/media/tests/mochitest/test_peerConnection_basicAudio.html"} - nightly-linux64-nightly-linux64-turn-udp-tcp, nightly-mac-aurora-mac10_7_64

steeplechase INFO  | {"action":"test_unexpected_fail","message":"PeerConnectionWrapper (pcRemote): legal ICE state transition from new to failed","time":1416325420982,"source_file":"dom/media/tests/mochitest/test_peerConnection_basicAudio.html"} - nightly-mac-aurora-mac10_7_64

steeplechase INFO  | {"action":"test_unexpected_fail","message":"Have remotecandidate stat(s)","time":1416325420987,"source_file":"dom/media/tests/mochitest/test_peerConnection_basicAudio.html"} - nightly-mac-aurora-mac10_7_64
(In reply to Syd Polk :sydpolk from comment #0)
> steeplechase INFO  | {"action":"test_unexpected_fail","message":"pc_local:
> ICE failed to switch to 'connected' state:
> failed","time":1416320006028,"source_file":"dom/media/tests/mochitest/
> test_peerConnection_basicAudio.html"} -
> nightly-linux64-nightly-linux64-turn-udp-tcp, nightly-linux64-win32,
> nightly-mac-aurora-mac10_7_64, nightly-mac-mac10_7_64
> 
> steeplechase INFO  | {"action":"test_unexpected_fail","message":"pc_local:
> ICE failed to switch to 'connected' state:
> failed","time":1416320006028,"source_file":"dom/media/tests/mochitest/
> test_peerConnection_basicAudio.html"} -
> nightly-linux64-nightly-linux64-turn-udp-tcp, nightly-mac-aurora-mac10_7_64

For this one I think I have fix. It's a problem in the tests. Basically one side is not ready yet to receive the messages from the other side.
I think it actually runs into a Javascript error, because in this case the test tries to access an undefined object. But somehow these kind of errors get masked by Steeplechase.
Try run for first piece looks green.
Hi Nils, does this need review ?
Assignee: nobody → drno
Flags: needinfo?(drno)
Keywords: checkin-needed
Flags: needinfo?(drno)
Attachment #8525084 - Flags: review?(spolk)
So what logic changes by moving this initialization around?
Flags: needinfo?(drno)
(In reply to Syd Polk :sydpolk from comment #5)
> So what logic changes by moving this initialization around?

When I wrote original code I thought we would need the trickle ICE data structures only when the test itself requires/requests trickle ICE. But in steeplechase the timing difference between two test clients can be that big that trickle ICE messages from the other peer arrive, before the local test has created/requested the needed data structures for storing the information from the other side.
So I only moved the creation of the data structure up to the PeerConnectionTest itself so they get created when we create the object, so we are ready to store incoming ICE candidates right after initializing.

BTW in mochitest on the build servers the missing data structure would have been caught by mochitest as trying to access undefined variables. Steeplechase unfortunately is silent about such programming errors :-(
Flags: needinfo?(drno)
Attachment #8525084 - Flags: review?(spolk) → review+
Keywords: checkin-needed
Apparently I should tried to build first locally... build errors fixed.

Try run: https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=a5893758a134
Attachment #8527131 - Attachment is obsolete: true
More work needed?
backlog: --- → webRTC+
Rank: 45
Flags: needinfo?(drno)
Priority: -- → P4
As we don't see these errors any more I think it safe to close this one (and open new bugs for remaining problems).
Status: NEW → RESOLVED
Closed: 9 years ago
Flags: needinfo?(drno)
Resolution: --- → FIXED
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.