Closed Bug 1637387 Opened 10 months ago Closed 10 months ago

[wpt-sync] Sync PR 23544 - webrtc wpt: add vp8 and h264 simulcast tests

Categories

(Core :: WebRTC, task, P4)

Product:
Core
Component:
WebRTC
Type:
task

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla78
Tracking Flags:
Tracking Status
firefox78 --- fixed

People

(Reporter: mozilla.org, Unassigned)

References

(
URL
)

Details

(Whiteboard: [wptsync downstream])

Attachments

(1 file)

Notable wpt changes
89.07 KB, text/plain
Details

Sync web-platform-tests PR 23544 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/23544
Details from upstream follow.

Philipp Hancke <philipp.hancke@googlemail.com> wrote:

webrtc wpt: add vp8 and h264 simulcast tests

BUG=None

Change-Id: I78b85c0dedd15ecf46300f114620c896c4851c09
Reviewed-on: https://chromium-review.googlesource.com/2196396
WPT-Export-Revision: 63eea9777dbaf6885e19826facbf26a88058d839

PR 23544 applied with additional changes from upstream: 0de19dc97fb8a46fc83c95153da7bbffb7018c54, a9f1e8c01979f0415e4f18396fa7b46d257a675a, 71940d7ce6a7c4de1c27815e32da20e37d57da2b, cdd4b43f2f89422f36d81546df70eb09f6aa8ed0
Component: web-platform-tests → WebRTC
Product: Testing → Core
Attached file Notable wpt changes — 
# CI Results

Ran 12 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 169 tests and 1 subtests

## Status Summary

### Firefox
OK     : 73[Gecko-android-em-7.0-x86_64-debug-geckoview, Gecko-android-em-7.0-x86_64-opt-geckoview] 76[Gecko-windows7-32-debug, Gecko-windows7-32-opt] 77[Gecko-linux1804-64-debug, Gecko-linux1804-64-opt, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows10-64-debug, Gecko-windows10-64-opt, Gecko-windows10-64-qr-debug, Gecko-windows10-64-qr-opt] 89[GitHub]
PASS   : 782[Gecko-android-em-7.0-x86_64-debug-geckoview, Gecko-android-em-7.0-x86_64-opt-geckoview] 829[GitHub] 880[Gecko-windows7-32-debug, Gecko-windows7-32-opt] 890[Gecko-linux1804-64-debug, Gecko-linux1804-64-opt, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows10-64-debug, Gecko-windows10-64-opt, Gecko-windows10-64-qr-debug, Gecko-windows10-64-qr-opt]
FAIL   : 300[Gecko-android-em-7.0-x86_64-debug-geckoview, Gecko-android-em-7.0-x86_64-opt-geckoview, Gecko-windows7-32-debug, Gecko-windows7-32-opt] 301[Gecko-linux1804-64-debug, Gecko-linux1804-64-opt, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows10-64-debug, Gecko-windows10-64-opt, Gecko-windows10-64-qr-debug, Gecko-windows10-64-qr-opt] 429[GitHub]
TIMEOUT: 7[Gecko-android-em-7.0-x86_64-debug-geckoview, Gecko-android-em-7.0-x86_64-opt-geckoview, Gecko-linux1804-64-debug, Gecko-linux1804-64-opt, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows10-64-debug, Gecko-windows10-64-opt, Gecko-windows10-64-qr-debug, Gecko-windows10-64-qr-opt, Gecko-windows7-32-debug, Gecko-windows7-32-opt] 15[GitHub]
ERROR  : 1[Gecko-android-em-7.0-x86_64-debug-geckoview, Gecko-android-em-7.0-x86_64-opt-geckoview, Gecko-linux1804-64-debug, Gecko-linux1804-64-opt, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows10-64-debug, Gecko-windows10-64-opt, Gecko-windows10-64-qr-debug, Gecko-windows10-64-qr-opt, Gecko-windows7-32-debug, Gecko-windows7-32-opt] 3[GitHub]
NOTRUN : 8[GitHub] 10[Gecko-android-em-7.0-x86_64-debug-geckoview, Gecko-android-em-7.0-x86_64-opt-geckoview, Gecko-linux1804-64-debug, Gecko-linux1804-64-opt, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows10-64-debug, Gecko-windows10-64-opt, Gecko-windows10-64-qr-debug, Gecko-windows10-64-qr-opt, Gecko-windows7-32-debug, Gecko-windows7-32-opt]

### Chrome
OK     : 90
PASS   : 1095
FAIL   : 138
TIMEOUT: 13
ERROR  : 2
NOTRUN : 35

### Safari
OK     : 80
PASS   : 692
FAIL   : 447
TIMEOUT: 27
ERROR  : 6
NOTRUN : 121

## Links
[Gecko CI (Treeherder)](https://treeherder.mozilla.org/#/jobs?repo=try&revision=9cbab0ecd9fe147c3e47f36da598284936f983e7)
[GitHub PR Head](https://wpt.fyi/results/?sha=4722a4dcd2d556082c1c29bc4f33e760411e32e8&label=pr_head)
[GitHub PR Base](https://wpt.fyi/results/?sha=4722a4dcd2d556082c1c29bc4f33e760411e32e8&label=pr_base)

## Details

### Firefox-only Failures
/webrtc/protocol/video-codecs.https.html
  H.264 and VP8 should be negotiated after handshake: FAIL linked bug:Bug 1534687
  H.264 and VP8 should be supported in initial offer: FAIL linked bug:Bug 1534688
  All H.264 codecs MUST include profile-level-id: FAIL linked bug:Bug 1534687
/webrtc/RTCConfiguration-rtcpMuxPolicy.html
  setConfiguration(config) - with { rtcpMuxPolicy: 'invalid' } should throw TypeError: FAIL
  setRemoteDescription throws InvalidAccessError when called with an answer without rtcp-mux and rtcpMuxPolicy is set to require: FAIL
  new RTCPeerConnection({ rtcpMuxPolicy: undefined }) should have default rtcpMuxPolicy require: FAIL
  new RTCPeerConnection({ rtcpMuxPolicy: 'require' }) should succeed: FAIL
  new RTCPeerConnection({ rtcpMuxPolicy: 'negotiate' }) may succeed or throw NotSupportedError: FAIL
  new RTCPeerConnection(config) - with { rtcpMuxPolicy: 'invalid' } should throw TypeError: FAIL
  setConfiguration({ rtcpMuxPolicy: 'require' }) with initial rtcpMuxPolicy negotiate should throw InvalidModificationError: FAIL
  setConfiguration(config) - with { rtcpMuxPolicy: null } should throw TypeError: FAIL
  setRemoteDescription throws InvalidAccessError when called with an offer without rtcp-mux and rtcpMuxPolicy is set to require: FAIL
  setConfiguration({}) with initial rtcpMuxPolicy negotiate should throw InvalidModificationError: FAIL
  setConfiguration({ rtcpMuxPolicy: 'negotiate' }) with initial rtcpMuxPolicy require should throw InvalidModificationError: FAIL
  new RTCPeerConnection() should have default rtcpMuxPolicy require: FAIL
  new RTCPeerConnection(config) - with { rtcpMuxPolicy: null } should throw TypeError: FAIL
/webrtc/RTCPeerConnection-track-stats.https.html: SKIP [`Gecko-android-em-7.0-x86_64-debug-geckoview`, `Gecko-android-em-7.0-x86_64-opt-geckoview`, `Gecko-linux1804-64-debug`, `Gecko-linux1804-64-opt`, `Gecko-linux1804-64-qr-debug`, `Gecko-linux1804-64-qr-opt`, `Gecko-windows10-64-debug`, `Gecko-windows10-64-opt`, `Gecko-windows10-64-qr-debug`, `Gecko-windows10-64-qr-opt`, `Gecko-windows7-32-debug`, `Gecko-windows7-32-opt`], TIMEOUT [`GitHub`]
  RTCRtpSender.getStats() contains only outbound-rtp and related stats: TIMEOUT
  RTCRtpReceiver.getStats() contains only inbound-rtp and related stats: NOTRUN
  O/A exchange yields inbound RTP stream stats for receiving track: FAIL
  O/A exchange yields outbound RTP stream stats for sending track: FAIL
  addTrack() without setLocalDescription() yields track stats: FAIL
  RTCPeerConnection.getStats(receivingTrack) is the same as RTCRtpReceiver.getStats(): NOTRUN
  RTCPeerConnection.getStats(sendingTrack) is the same as RTCRtpSender.getStats(): NOTRUN
  addTrack() with setLocalDescription() yields track stats: FAIL
/webrtc/RTCPeerConnection-connectionState.https.html: SKIP [`Gecko-android-em-7.0-x86_64-debug-geckoview`, `Gecko-android-em-7.0-x86_64-opt-geckoview`, `Gecko-linux1804-64-debug`, `Gecko-linux1804-64-opt`, `Gecko-linux1804-64-qr-debug`, `Gecko-linux1804-64-qr-opt`, `Gecko-windows10-64-debug`, `Gecko-windows10-64-opt`, `Gecko-windows10-64-qr-debug`, `Gecko-windows10-64-qr-opt`, `Gecko-windows7-32-debug`, `Gecko-windows7-32-opt`], TIMEOUT [`GitHub`]
  Initial connectionState should be new: FAIL
  connectionState transitions to connected via connecting: TIMEOUT
  connection with one data channel should eventually have connected connection state: TIMEOUT
  Closing the connection should set connectionState to closed: FAIL
  Closing a PeerConnection should not fire connectionstatechange event: NOTRUN
/mst-content-hint/idlharness.window.html
  MediaStreamTrack interface: videoTrack must inherit property "contentHint" with the proper type: FAIL
  MediaStreamTrack interface: attribute contentHint: FAIL
  MediaStreamTrack interface: audioTrack must inherit property "contentHint" with the proper type: FAIL
/webrtc/idlharness.https.window.html
  RTCRtpTransceiver interface: operation setCodecPreferences(sequence<RTCRtpCodecCapability>): FAIL
  RTCPeerConnection interface: new RTCPeerConnection() must inherit property "setConfiguration(optional RTCConfiguration)" with the proper type: FAIL
  RTCIceCandidate interface: attribute candidate: FAIL
  RTCDTMFSender interface: attribute canInsertDTMF: FAIL
  RTCRtpReceiver interface: calling getCapabilities(DOMString) on new RTCPeerConnection().addTransceiver('audio').receiver with too few arguments must throw TypeError: FAIL
  RTCPeerConnection interface: attribute onconnectionstatechange: FAIL
  RTCIceTransport interface object length: FAIL
  RTCPeerConnection interface: attribute connectionState: FAIL
  RTCIceTransport interface: attribute gatheringState: FAIL
  RTCRtpReceiver interface: operation getCapabilities(DOMString): FAIL
  RTCRtpSender interface: calling getCapabilities(DOMString) on new RTCPeerConnection().addTransceiver('audio').sender with too few arguments must throw TypeError: FAIL
  RTCRtpSender interface: operation getCapabilities(DOMString): FAIL
  RTCIceTransport interface: attribute state: FAIL
  RTCIceTransport interface: existence and properties of interface prototype object's @@unscopables property: FAIL
  RTCCertificate interface: idlTestObjects.certificate must inherit property "getFingerprints()" with the proper type: FAIL
  RTCIceTransport interface: existence and properties of interface prototype object's "constructor" property: FAIL
  RTCRtpTransceiver interface: new RTCPeerConnection().addTransceiver('audio') must inherit property "setCodecPreferences(sequence<RTCRtpCodecCapability>)" with the proper type: FAIL
  RTCCertificate interface: operation getFingerprints(): FAIL
  RTCRtpReceiver interface: new RTCPeerConnection().addTransceiver('audio').receiver must inherit property "getParameters()" with the proper type: FAIL
  RTCRtpReceiver interface: operation getParameters(): FAIL
  RTCPeerConnection interface: new RTCPeerConnection() must inherit property "onconnectionstatechange" with the proper type: FAIL
  RTCRtpTransceiver interface: calling setCodecPreferences(sequence<RTCRtpCodecCapability>) on new RTCPeerConnection().addTransceiver('audio') with too few arguments must throw TypeError: FAIL
  RTCRtpSender interface: calling setParameters(RTCRtpSendParameters) on new RTCPeerConnection().addTransceiver('audio').sender with too few arguments must throw TypeError: FAIL
  RTCIceTransport interface object name: FAIL
  RTCIceCandidate interface: attribute sdpMLineIndex: FAIL
  RTCIceCandidate interface: attribute sdpMid: FAIL
  RTCPeerConnection interface: calling setConfiguration(optional RTCConfiguration) on new RTCPeerConnection() with too few arguments must throw TypeError: FAIL
  RTCPeerConnection interface: new RTCPeerConnection() must inherit property "connectionState" with the proper type: FAIL
/webrtc/protocol/ice-ufragpwd.html
  setRemoteDescription with a ice-ufrag containing a non-ice-char fails: FAIL
  setRemoteDescription with a ice-pwd containing a non-ice-char fails: FAIL
/html/dom/usvstring-reflection.https.html
  source : unpaired surrogate codepoint should be replaced with U+FFFD: FAIL
  anchor : unpaired surrogate codepoint should be replaced with U+FFFD: FAIL
  area : unpaired surrogate codepoint should be replaced with U+FFFD: FAIL
  storage event : unpaired surrogate codepoint should be replaced with U+FFFD: FAIL
/webrtc/RTCPeerConnection-setRemoteDescription-pranswer.html
  setRemoteDescription(answer) from have-remote-pranswer state should succeed: FAIL
  setRemoteDescription(pranswer) multiple times should succeed: FAIL
  setRemoteDescription(pranswer) from stable state should reject with InvalidStateError: FAIL
  setRemoteDescription(pranswer) from have-local-offer state should succeed: FAIL
/webrtc/RTCPeerConnection-setLocalDescription-pranswer.html
  setLocalDescription(answer) from have-local-pranswer state should succeed: FAIL
  setLocalDescription(pranswer) can be applied multiple times while still in have-local-pranswer: FAIL
/webrtc/RTCPeerConnection-getStats.https.html
  getStats() with no argument should return stats report containing peer-connection stats on an empty PC: FAIL

(See attachment for full changes)
Pushed by wptsync@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/67d0405c7895
[wpt PR 23544] - webrtc wpt: add vp8 and h264 simulcast tests, a=testonly
https://hg.mozilla.org/integration/autoland/rev/6090eaf4818d
[wpt PR 23544] - Update wpt metadata, a=testonly

https://hg.mozilla.org/mozilla-central/rev/67d0405c7895
https://hg.mozilla.org/mozilla-central/rev/6090eaf4818d

Status: NEW → RESOLVED
Closed: 10 months ago
status-firefox78: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla78
You need to log in before you can comment on or make changes to this bug.