Closed Bug 1786022 Opened 2 years ago Closed 2 years ago

[wpt-sync] Sync PR 35533 - Deflake external/wpt/webrtc-stats/supported-stats.html.

Categories

(Core :: WebRTC, task, P4)

Product:
Core
Component:
WebRTC
Type:
task

Tracking

()

Status:
RESOLVED FIXED
Milestone:
106 Branch
Tracking Flags:
Tracking Status
firefox106 --- fixed

People

(Reporter: wpt-sync, Unassigned)

References

(Depends on 1 open bug,
URL
)

Details

(Whiteboard: [wptsync downstream])

Sync web-platform-tests PR 35533 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/35533
Details from upstream follow.

Henrik Boström <hbos@chromium.org> wrote:

Deflake external/wpt/webrtc-stats/supported-stats.html.

Sometimes the remote-outbound-rtp is not present, causing all
remote-outbound-rtp expectations (plus inbound-rtp.remoteId) to FAIL.

This is probably caused by the stats getter only awaiting until
remote-inbound-rtp exists, not remote-outbound-rtp, which is racy.

This CL fixes the problem by ensuring at least one remote-outbound-rtp
exists before the early "return stats".

Bug: 1353053
Change-Id: If88c1c58d2fba80ae6529d2322b105b6a8debab2
Reviewed-on: https://chromium-review.googlesource.com/3842252
WPT-Export-Revision: 1dd8895c4658dae53586358085feb9ee23f53c07

Component: web-platform-tests → WebRTC
Product: Testing → Core

CI Results

Ran 9 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 1 tests and 229 subtests

Status Summary

Firefox

OK : 1
PASS: 119
FAIL: 110

Chrome

OK : 1
PASS: 214
FAIL: 15

Safari

OK : 1
PASS: 160
FAIL: 69

Links

Gecko CI (Treeherder)
GitHub PR Head
GitHub PR Base

Details

Firefox-only Failures

  • /webrtc-stats/supported-stats.html [wpt.fyi]
    • inbound-rtp's keyFramesDecoded: FAIL
    • inbound-rtp's totalDecodeTime: FAIL
    • inbound-rtp's totalInterFrameDelay: FAIL
    • inbound-rtp's totalSquaredInterFrameDelay: FAIL
    • inbound-rtp's lastPacketReceivedTimestamp: FAIL
    • inbound-rtp's headerBytesReceived: FAIL
    • inbound-rtp's fecPacketsReceived: FAIL
    • inbound-rtp's fecPacketsDiscarded: FAIL
    • inbound-rtp's concealmentEvents: FAIL
    • inbound-rtp's insertedSamplesForDeceleration: FAIL
    • inbound-rtp's removedSamplesForAcceleration: FAIL
    • inbound-rtp's totalAudioEnergy: FAIL
    • inbound-rtp's totalSamplesDuration: FAIL
    • inbound-rtp's framesDropped: FAIL
    • inbound-rtp's transportId: FAIL
    • outbound-rtp's mediaSourceId: FAIL
    • outbound-rtp's framesPerSecond: FAIL
    • outbound-rtp's keyFramesEncoded: FAIL
    • outbound-rtp's totalPacketSendDelay: FAIL
    • outbound-rtp's qualityLimitationResolutionChanges: FAIL
    • outbound-rtp's transportId: FAIL
    • remote-inbound-rtp's transportId: FAIL
    • peer-connection's dataChannelsOpened: FAIL
    • peer-connection's dataChannelsClosed: FAIL
    • peer-connection's timestamp: FAIL
    • peer-connection's type: FAIL
    • peer-connection's id: FAIL
    • media-source's trackIdentifier: FAIL
    • media-source's kind: FAIL
    • media-source's timestamp: FAIL
    • media-source's type: FAIL
    • media-source's id: FAIL
    • transport's bytesSent: FAIL
    • transport's bytesReceived: FAIL
    • transport's dtlsState: FAIL
    • transport's selectedCandidatePairId: FAIL
    • transport's localCertificateId: FAIL
    • transport's remoteCertificateId: FAIL
    • transport's tlsVersion: FAIL
    • transport's dtlsCipher: FAIL
    • transport's srtpCipher: FAIL
    • transport's timestamp: FAIL
    • transport's type: FAIL
    • transport's id: FAIL
    • candidate-pair's totalRoundTripTime: FAIL
    • candidate-pair's currentRoundTripTime: FAIL
    • candidate-pair's availableOutgoingBitrate: FAIL
    • candidate-pair's requestsReceived: FAIL
    • candidate-pair's requestsSent: FAIL
    • candidate-pair's responsesReceived: FAIL
    • candidate-pair's responsesSent: FAIL
    • local-candidate's transportId: FAIL
    • remote-candidate's transportId: FAIL
    • certificate's fingerprint: FAIL
    • certificate's fingerprintAlgorithm: FAIL
    • certificate's base64Certificate: FAIL
    • certificate's timestamp: FAIL
    • certificate's type: FAIL
    • certificate's id: FAIL

New Tests That Don't Pass

  • /webrtc-stats/supported-stats.html [wpt.fyi]
    • inbound-rtp's trackIdentifier: FAIL (Chrome: PASS, Safari: FAIL)
    • inbound-rtp's mid: FAIL (Chrome: PASS, Safari: FAIL)
    • inbound-rtp's keyFramesDecoded: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's qpSum: FAIL (Chrome: PASS, Safari: FAIL)
    • inbound-rtp's totalDecodeTime: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's totalInterFrameDelay: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's totalSquaredInterFrameDelay: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's lastPacketReceivedTimestamp: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's headerBytesReceived: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's fecPacketsReceived: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's fecPacketsDiscarded: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's totalProcessingDelay: FAIL (Chrome: PASS, Safari: FAIL)
    • inbound-rtp's estimatedPlayoutTimestamp: FAIL (Chrome: FAIL, Safari: FAIL)
    • inbound-rtp's jitterBufferTargetDelay: FAIL (Chrome: PASS, Safari: FAIL)
    • inbound-rtp's jitterBufferMinimumDelay: FAIL (Chrome: PASS, Safari: FAIL)
    • inbound-rtp's concealmentEvents: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's insertedSamplesForDeceleration: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's removedSamplesForAcceleration: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's audioLevel: FAIL (Chrome: PASS, Safari: FAIL)
    • inbound-rtp's totalAudioEnergy: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's totalSamplesDuration: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's decoderImplementation: FAIL (Chrome: PASS, Safari: FAIL)
    • inbound-rtp's framesDropped: FAIL (Chrome: PASS, Safari: PASS)
    • inbound-rtp's transportId: FAIL (Chrome: PASS, Safari: PASS)
    • outbound-rtp's mid: FAIL (Chrome: PASS, Safari: FAIL)
    • outbound-rtp's mediaSourceId: FAIL (Chrome: PASS, Safari: PASS)
    • outbound-rtp's rid: FAIL (Chrome: FAIL, Safari: FAIL)
    • outbound-rtp's targetBitrate: FAIL (Chrome: PASS, Safari: FAIL)
    • outbound-rtp's framesPerSecond: FAIL (Chrome: PASS, Safari: PASS)
    • outbound-rtp's keyFramesEncoded: FAIL (Chrome: PASS, Safari: PASS)
    • outbound-rtp's totalPacketSendDelay: FAIL (Chrome: PASS, Safari: PASS)
    • outbound-rtp's qualityLimitationReason: FAIL (Chrome: PASS, Safari: FAIL)
    • outbound-rtp's qualityLimitationDurations: FAIL (Chrome: PASS, Safari: FAIL)
    • outbound-rtp's qualityLimitationResolutionChanges: FAIL (Chrome: PASS, Safari: PASS)
    • outbound-rtp's encoderImplementation: FAIL (Chrome: PASS, Safari: FAIL)
    • outbound-rtp's transportId: FAIL (Chrome: PASS, Safari: PASS)
    • remote-inbound-rtp's totalRoundTripTime: FAIL (Chrome: PASS, Safari: FAIL)
    • remote-inbound-rtp's fractionLost: FAIL (Chrome: PASS, Safari: FAIL)
    • remote-inbound-rtp's roundTripTimeMeasurements: FAIL (Chrome: PASS, Safari: FAIL)
    • remote-inbound-rtp's framesDropped: FAIL (Chrome: FAIL, Safari: FAIL)
    • remote-inbound-rtp's transportId: FAIL (Chrome: PASS, Safari: PASS)
    • remote-outbound-rtp's reportsSent: FAIL (Chrome: PASS, Safari: FAIL)
    • remote-outbound-rtp's roundTripTime: FAIL (Chrome: FAIL, Safari: FAIL)
    • remote-outbound-rtp's totalRoundTripTime: FAIL (Chrome: PASS, Safari: FAIL)
    • remote-outbound-rtp's roundTripTimeMeasurements: FAIL (Chrome: PASS, Safari: FAIL)
    • remote-outbound-rtp's transportId: FAIL (Chrome: PASS, Safari: FAIL)
    • peer-connection's dataChannelsOpened: FAIL (Chrome: PASS, Safari: PASS)
    • peer-connection's dataChannelsClosed: FAIL (Chrome: PASS, Safari: PASS)
    • peer-connection's timestamp: FAIL (Chrome: PASS, Safari: PASS)
    • peer-connection's type: FAIL (Chrome: PASS, Safari: PASS)
    • peer-connection's id: FAIL (Chrome: PASS, Safari: PASS)
    • media-source's audioLevel: FAIL (Chrome: PASS, Safari: FAIL)
    • media-source's totalAudioEnergy: FAIL (Chrome: PASS, Safari: FAIL)
    • media-source's totalSamplesDuration: FAIL (Chrome: PASS, Safari: FAIL)
    • media-source's echoReturnLoss: FAIL (Chrome: FAIL, Safari: FAIL)
    • media-source's echoReturnLossEnhancement: FAIL (Chrome: FAIL, Safari: FAIL)
    • media-source's width: FAIL (Chrome: PASS, Safari: FAIL)
    • media-source's height: FAIL (Chrome: PASS, Safari: FAIL)
    • media-source's frames: FAIL (Chrome: PASS, Safari: FAIL)
    • media-source's framesPerSecond: FAIL (Chrome: PASS, Safari: FAIL)
    • media-source's trackIdentifier: FAIL (Chrome: PASS, Safari: PASS)
    • media-source's kind: FAIL (Chrome: PASS, Safari: PASS)
    • media-source's timestamp: FAIL (Chrome: PASS, Safari: PASS)
    • media-source's type: FAIL (Chrome: PASS, Safari: PASS)
    • media-source's id: FAIL (Chrome: PASS, Safari: PASS)
    • transport's packetsSent: FAIL (Chrome: PASS, Safari: FAIL)
    • transport's packetsReceived: FAIL (Chrome: PASS, Safari: FAIL)
    • transport's bytesSent: FAIL (Chrome: PASS, Safari: PASS)
    • transport's bytesReceived: FAIL (Chrome: PASS, Safari: PASS)
    • transport's iceRole: FAIL (Chrome: PASS, Safari: FAIL)
    • transport's iceLocalUsernameFragment: FAIL (Chrome: PASS, Safari: FAIL)
    • transport's dtlsState: FAIL (Chrome: PASS, Safari: PASS)
    • transport's iceState: FAIL (Chrome: PASS, Safari: FAIL)
    • transport's selectedCandidatePairId: FAIL (Chrome: PASS, Safari: PASS)
    • transport's localCertificateId: FAIL (Chrome: PASS, Safari: PASS)
    • transport's remoteCertificateId: FAIL (Chrome: PASS, Safari: PASS)
    • transport's tlsVersion: FAIL (Chrome: PASS, Safari: PASS)
    • transport's dtlsCipher: FAIL (Chrome: PASS, Safari: PASS)
    • transport's dtlsRole: FAIL (Chrome: PASS, Safari: FAIL)
    • transport's srtpCipher: FAIL (Chrome: PASS, Safari: PASS)
    • transport's selectedCandidatePairChanges: FAIL (Chrome: PASS, Safari: FAIL)
    • transport's timestamp: FAIL (Chrome: PASS, Safari: PASS)
    • transport's type: FAIL (Chrome: PASS, Safari: PASS)
    • transport's id: FAIL (Chrome: PASS, Safari: PASS)
    • candidate-pair's packetsSent: FAIL (Chrome: PASS, Safari: FAIL)
    • candidate-pair's packetsReceived: FAIL (Chrome: PASS, Safari: FAIL)
    • candidate-pair's totalRoundTripTime: FAIL (Chrome: PASS, Safari: PASS)
    • candidate-pair's currentRoundTripTime: FAIL (Chrome: PASS, Safari: PASS)
    • candidate-pair's availableOutgoingBitrate: FAIL (Chrome: PASS, Safari: PASS)
    • candidate-pair's availableIncomingBitrate: FAIL (Chrome: FAIL, Safari: FAIL)
    • candidate-pair's requestsReceived: FAIL (Chrome: PASS, Safari: PASS)
    • candidate-pair's requestsSent: FAIL (Chrome: PASS, Safari: PASS)
    • candidate-pair's responsesReceived: FAIL (Chrome: PASS, Safari: PASS)
    • candidate-pair's responsesSent: FAIL (Chrome: PASS, Safari: PASS)
    • candidate-pair's consentRequestsSent: FAIL (Chrome: PASS, Safari: FAIL)
    • candidate-pair's packetsDiscardedOnSend: FAIL (Chrome: PASS, Safari: FAIL)
    • candidate-pair's bytesDiscardedOnSend: FAIL (Chrome: PASS, Safari: FAIL)
    • local-candidate's transportId: FAIL (Chrome: PASS, Safari: PASS)
    • local-candidate's url: FAIL (Chrome: FAIL, Safari: FAIL)
    • local-candidate's relayProtocol: FAIL (Chrome: FAIL, Safari: FAIL)
    • remote-candidate's transportId: FAIL (Chrome: PASS, Safari: PASS)
    • remote-candidate's url: FAIL (Chrome: FAIL, Safari: FAIL)
    • remote-candidate's relayProtocol: FAIL (Chrome: FAIL, Safari: FAIL)
    • certificate's fingerprint: FAIL (Chrome: PASS, Safari: PASS)
    • certificate's fingerprintAlgorithm: FAIL (Chrome: PASS, Safari: PASS)
    • certificate's base64Certificate: FAIL (Chrome: PASS, Safari: PASS)
    • certificate's issuerCertificateId: FAIL (Chrome: FAIL, Safari: FAIL)
    • certificate's timestamp: FAIL (Chrome: PASS, Safari: PASS)
    • certificate's type: FAIL (Chrome: PASS, Safari: PASS)
    • certificate's id: FAIL (Chrome: PASS, Safari: PASS)
Pushed by wptsync@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/8b7235ffb9c4 [wpt PR 35533] - Deflake external/wpt/webrtc-stats/supported-stats.html., a=testonly

https://hg.mozilla.org/mozilla-central/rev/8b7235ffb9c4

Status: NEW → RESOLVED
Closed: 2 years ago
status-firefox106: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 106 Branch
You need to log in before you can comment on or make changes to this bug.