Intermittent TEST-UNEXPECTED-PASS | /webrtc/RTCPeerConnection-setLocalDescription.html | setLocalDescription(offer) should never resolve if connection is closed in parallel - expected FAIL

RESOLVED FIXED in Firefox 56

Status

()

P2
normal
Rank:
20
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: intermittent-bug-filer, Assigned: mjf)

Tracking

({intermittent-failure})

unspecified
mozilla56
intermittent-failure
Points:
---

Firefox Tracking Flags

(firefox56 fixed)

Details

(Whiteboard: [stockwell disabled])

Attachments

(1 attachment)

Comment hidden (Intermittent Failures Robot)
36 failures in 3 days, so far, failures are on debug and e10s.  This is a new failure as of a few days ago, we should get someone to look into this.

Here is a recent osx log:
https://treeherder.mozilla.org/logviewer.html#?repo=try&job_id=116978449&lineNumber=6149

and some related information:
09:30:41     INFO - TEST-FAIL | /webrtc/RTCPeerConnection-setLocalDescription.html | setLocalDescription() with offer not created by own createOffer() should reject with InvalidModificationError - assert_throws: function "function() { throw e }" threw object "InvalidStateError: Calling SetLocal without first calling CreateOffer/Answer is not supported." that is not a DOMException InvalidModificationError: property "code" is equal to 11, expected 13
09:30:41     INFO - promise callback*@http://web-platform.test:8000/webrtc/RTCPeerConnection-setLocalDescription.html:51:12
09:30:41     INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1412:20
09:30:41     INFO - promise callback*promise_test@http://web-platform.test:8000/resources/testharness.js:529:31
09:30:41     INFO - @http://web-platform.test:8000/webrtc/RTCPeerConnection-setLocalDescription.html:48:3
09:30:41     INFO - TEST-FAIL | /webrtc/RTCPeerConnection-setLocalDescription.html | Set created offer other than last offer should reject with InvalidModificationError - assert_throws: function "function() { throw e }" threw object "InvalidSessionDescriptionError: Changing the media-type of m-sections is not allowed" that is not a DOMException InvalidModificationError: property "code" is equal to 0, expected 13
09:30:41     INFO - promise callback*@http://web-platform.test:8000/webrtc/RTCPeerConnection-setLocalDescription.html:70:7
09:30:41     INFO - promise callback*@http://web-platform.test:8000/webrtc/RTCPeerConnection-setLocalDescription.html:68:12
09:30:41     INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1412:20
09:30:41     INFO - promise callback*promise_test@http://web-platform.test:8000/resources/testharness.js:529:31
09:30:41     INFO - @http://web-platform.test:8000/webrtc/RTCPeerConnection-setLocalDescription.html:62:3
09:30:41     INFO - TEST-FAIL | /webrtc/RTCPeerConnection-setLocalDescription.html | Creating and setting offer multiple times should succeed - promise_test: Unhandled rejection with value: object "InvalidStateError: Cannot create offer in state have-local-offer"
09:30:41     INFO - @http://web-platform.test:8000/webrtc/RTCPeerConnection-setLocalDescription.html:78:3
09:30:41     INFO - TEST-UNEXPECTED-PASS | /webrtc/RTCPeerConnection-setLocalDescription.html | setLocalDescription(offer) should never resolve if connection is closed in parallel - expected FAIL
09:30:41     INFO - TEST-INFO | expected FAIL
09:30:41     INFO - ...........
09:30:41     INFO - TEST-OK | /webrtc/RTCPeerConnection-setLocalDescription.html | took 1234ms


it appears that we expect this test to fail but it is passing.

:mreavy, can you find someone on the webrtc team to look into this?
Flags: needinfo?(mreavy)
Whiteboard: [stockwell needswork]
In my opinion this failing new test is only asking for race condition. It calls method one() followed by two() and expects that one() returns a failure, because two() should have closed the PeerConnection object before the one() operation could succeed. That is only asking for trouble in my opinion and not how a reliable test should be written.

I'll file issue upstream to get it fixed.
Created the following issue upstream https://github.com/w3c/web-platform-tests/issues/6612
I think we should skip/ignore this test until it gets fixed upstream.
Flags: needinfo?(mreavy)
Comment hidden (Intermittent Failures Robot)
Same as for bug 1382972: there is a PR upstream to remove this test as it's trying to test a legacy race condition which can't be triggered reliably https://github.com/w3c/web-platform-tests/pull/6616
Rank: 20
Priority: -- → P2
Duplicate of this bug: 1383349
Duplicate of this bug: 1383352
(Assignee)

Updated

2 years ago
Assignee: nobody → mfroman
(Assignee)

Comment 9

2 years ago
For now I'm going to disable this test in our repo.
Comment hidden (mozreview-request)

Comment 11

2 years ago
mozreview-review
Comment on attachment 8890067 [details]
Bug 1383069 - disable wpt test RTCPeerConnection-setLocalDescription.html

https://reviewboard.mozilla.org/r/161130/#review166536

Same here: I would hope that there is a better way to just skip single tests within the one file. If not we can revert to this bigger canon.
Attachment #8890067 - Flags: review?(drno) → review-
Comment hidden (Intermittent Failures Robot)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)

Comment 15

2 years ago
mozreview-review
Comment on attachment 8890067 [details]
Bug 1383069 - disable wpt test RTCPeerConnection-setLocalDescription.html

https://reviewboard.mozilla.org/r/161130/#review166920

LGTM
Attachment #8890067 - Flags: review?(na-g) → review+

Comment 16

2 years ago
Pushed by mfroman@nostrum.com:
https://hg.mozilla.org/integration/autoland/rev/f9869e7ecce4
disable wpt test RTCPeerConnection-setLocalDescription.html r=ng
Comment hidden (Intermittent Failures Robot)

Comment 18

2 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/f9869e7ecce4
Status: NEW → RESOLVED
Last Resolved: 2 years ago
status-firefox56: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla56
Whiteboard: [stockwell needswork] → [stockwell disabled]
Comment hidden (Intermittent Failures Robot)
You need to log in before you can comment on or make changes to this bug.