1637844 - Perma TEST-UNEXPECTED-PASS | /webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html | [video] getSynchronizationSources() should not contain captureTimestamp if absolute capture time RTP header extension is not offered - expected TIMEOUT

Status: RESOLVED FIXED

(Core :: WebRTC, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: nbeleuzu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=302205544&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YX-HWkokRM-1f7IFqpnc-w/runs/0/artifacts/public/logs/live_backing.log

---

[task 2020-05-14T03:41:55.762Z] 03:41:55 INFO - TEST-PASS | /webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html | [audio] getSynchronizationSources() should not contain captureTimestamp if absolute capture time RTP header extension is offered, but not answered
[task 2020-05-14T03:41:55.763Z] 03:41:55 INFO - TEST-FAIL | /webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html | [audio] getSynchronizationSources() should contain captureTimestamp if absolute capture time RTP header extension is negotiated - assert_false: Absolute capture time RTP header extension is not answered expected false got true
[task 2020-05-14T03:41:55.764Z] 03:41:55 INFO - checkAbsCaptureTimeAndExchangeAnswer@http://web-platform.test:8000/webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html:91:17
[task 2020-05-14T03:41:55.764Z] 03:41:55 INFO - asyncinitiateSingleTrackCall@http://web-platform.test:8000/webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html:144:9
[task 2020-05-14T03:41:55.764Z] 03:41:55 INFO - async@http://web-platform.test:8000/webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html:196:36
[task 2020-05-14T03:41:55.764Z] 03:41:55 INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1988:25
[task 2020-05-14T03:41:55.764Z] 03:41:55 INFO - promise_test/tests.promise_tests</<@http://web-platform.test:8000/resources/testharness.js:592:36
[task 2020-05-14T03:41:55.764Z] 03:41:55 INFO - promise_test/tests.promise_tests<@http://web-platform.test:8000/resources/testharness.js:591:20
[task 2020-05-14T03:41:55.764Z] 03:41:55 INFO - promise callbackpromise_test@http://web-platform.test:8000/resources/testharness.js:590:51
[task 2020-05-14T03:41:55.764Z] 03:41:55 INFO - @http://web-platform.test:8000/webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html:195:15
[task 2020-05-14T03:41:55.765Z] 03:41:55 INFO - TEST-UNEXPECTED-PASS | /webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html | [video] getSynchronizationSources() should not contain captureTimestamp if absolute capture time RTP header extension is not offered - expected TIMEOUT
[task 2020-05-14T03:41:55.765Z] 03:41:55 INFO - TEST-INFO | expected TIMEOUT
[task 2020-05-14T03:41:55.765Z] 03:41:55 INFO -
[task 2020-05-14T03:41:55.765Z] 03:41:55 INFO - TEST-UNEXPECTED-PASS | /webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html | [video] getSynchronizationSources() should not contain captureTimestamp if absolute capture time RTP header extension is offered, but not answered - expected NOTRUN
[task 2020-05-14T03:41:55.765Z] 03:41:55 INFO - TEST-INFO | expected NOTRUN
[task 2020-05-14T03:41:55.766Z] 03:41:55 INFO -
[task 2020-05-14T03:41:55.766Z] 03:41:55 INFO - TEST-UNEXPECTED-FAIL | /webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html | [video] getSynchronizationSources() should contain captureTimestamp if absolute capture time RTP header extension is negotiated - assert_false: Absolute capture time RTP header extension is not answered expected false got true
[task 2020-05-14T03:41:55.766Z] 03:41:55 INFO - checkAbsCaptureTimeAndExchangeAnswer@http://web-platform.test:8000/webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html:91:17
[task 2020-05-14T03:41:55.766Z] 03:41:55 INFO - asyncinitiateSingleTrackCall@http://web-platform.test:8000/webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html:144:9
[task 2020-05-14T03:41:55.767Z] 03:41:55 INFO - async*@http://web-platform.test:8000/webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html:196:36
[task 2020-05-14T03:41:55.767Z] 03:41:55 INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1988:25
[task 2020-05-14T03:41:55.767Z] 03:41:55 INFO - promise_test/tests.promise_tests</<@http://web-platform.test:8000/resources/testharness.js:592:36
[task 2020-05-14T03:41:55.767Z] 03:41:55 INFO - promise_test/tests.promise_tests<@http://web-platform.test:8000/resources/testharness.js:591:20
[task 2020-05-14T03:41:55.767Z] 03:41:55 INFO - promise callback*promise_test@http://web-platform.test:8000/resources/testharness.js:590:51
[task 2020-05-14T03:41:55.768Z] 03:41:55 INFO - @http://web-platform.test:8000/webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html:195:15
[task 2020-05-14T03:41:55.768Z] 03:41:55 INFO - TEST-INFO | expected NOTRUN

Comment 1

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

These are unexpected passes which show up on central: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception&searchStr=wpt&revision=045d696faa87fc06c8040b75dec7cd6358c34067&selectedTaskRun=YX-HWkokRM-1f7IFqpnc-w-0

On autoland, it gets skipped: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=302215515&repo=autoland&lineNumber=1309-1310

This looks related to the test scheduling changes.

Comment 2

[Marco Castelluccio [:marco]]

The skipping is in the harness itself, so it shouldn't be related to the test scheduling change.

Maybe related to the recent work on moving some WPT tests to backlog?

Comment 3

[Marco Castelluccio [:marco]]

(Also, if it's passing, maybe some new WebRTC extension was implemented, and: we should mark it as pass)

Comment 4

[Dan Minor [:dminor]]

Hey Nico, do you mind having a look at this one? Thanks!

Comment 5

[Andrew Halberstadt [:ahal]]

Agree with Marco, I don't think this could be caused by scheduling changes.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

I haven't moved any of tests from that directory to backlog. Most likely the test is working now and we didn't see it on autoland due to skipping expected timeouts on autoland.

Comment 7

[Nico Grunbaum [:ng, @chew:mozilla.org]]

Nothing has changed with regard to captureTime (an extension field we never emit), though I did land a patch yesterday effecting getSynchronizationSources.

This is probably legitimately passing we don't support the absolute capture time RTP extension, and thus we never report captureTime. I think the right course of action given that is to mark it as PASS, so that when / if we support absolute capture time it has the opportunity to fail.

Is it normal for NOTRUN tests to be run under some circumstances like above?

Comment 8

[Intermittent Failures Robot]

107 failures in 970 pushes (0.11 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 103
• try: 4

Platform breakdown:

• android-em-7-0-x86_64: 9
• linux1804-64-qr: 15
• linux1804-64: 9
• macosx1014-64: 4
• windows10-64-qr: 13
• windows10-64: 8
• linux1804-64-asan: 9
• windows7-32: 8
• linux1804-64-ccov: 4
• windows7-32-shippable: 4
• linux1804-64-shippable-qr: 4
• linux1804-64-shippable: 4
• windows10-64-shippable-qr: 4
• windows10-64-shippable: 4
• macosx1014-64-shippable: 4
• linux1804-32-shippable: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1637844&startday=2020-05-14&endday=2020-05-14&tree=all

Comment 9

[James Graham [:jgraham]]

Yes, marking it as expected pass seems like the right thing to do.

A test being marked NOTRUN means that a test declared a subtest, but then no steps of the test were run. That typically happens either if we timeout before getting to the subtest (which can be intermittent) or if there's an uncaught exception before we get to the subtest.

Comment 10

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Attachment: Bug 1637844 - Update video steps in RTCRtpSynchronizationSource-captureTimestamp.html to pass. r=jgraham DONTBUILD

Comment 11

[Intermittent Failures Robot]

95 failures in 1010 pushes (0.094 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• mozilla-central: 47
• try: 48

Platform breakdown:

• linux1804-64-asan: 5
• windows7-32: 7
• windows10-64-qr: 8
• windows10-64: 5
• linux1804-64: 7
• linux1804-64-qr: 9
• android-em-7-0-x86_64: 10
• macosx1014-64: 6
• linux1804-64-ccov: 2
• windows10-64-shippable-qr: 4
• windows10-64-shippable: 3
• linux1804-64-shippable: 4
• linux1804-64-shippable-qr: 4
• windows7-32-shippable: 4
• linux1804-32-shippable: 4
• macosx1014-64-shippable: 4
• linux1804-64-devedition: 2
• windows7-32-devedition: 2
• windows10-64-devedition: 2
• macosx1014-64-devedition: 2
• windows10-64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1637844&startday=2020-05-15&endday=2020-05-15&tree=all

Comment 12

[Intermittent Failures Robot]

54 failures in 263 pushes (0.205 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 34
• try: 20

Platform breakdown:

• windows7-32: 5
• windows10-64: 4
• windows10-64-qr: 6
• linux1804-64-qr: 4
• linux1804-64: 3
• android-em-7-0-x86_64: 6
• macosx1014-64: 2
• linux1804-64-asan: 3
• linux1804-64-ccov: 1
• windows7-32-shippable: 2
• windows10-64-ccov: 2
• linux1804-64-shippable-qr: 2
• linux1804-64-shippable: 2
• linux1804-32-shippable: 1
• macosx1014-64-shippable: 2
• windows10-64-shippable-qr: 1
• windows10-64-shippable: 1
• linux1804-64-devedition: 1
• windows7-32-devedition: 1
• macosx1014-64-devedition: 1
• windows10-64-devedition: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1637844&startday=2020-05-16&endday=2020-05-16&tree=all

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/505be09fd733
Update video steps in RTCRtpSynchronizationSource-captureTimestamp.html to pass. r=jmaher DONTBUILD

Comment 14

[Razvan Maries]

Bakced out for perma failures on captureTimestamp.html.

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedTaskRun=YK1xS-YtSOCr9dQk_KLEwg-0&resultStatus=testfailed%2Cbusted%2Cexception&revision=ee08ec655b70c572b6be02003f62e614b669f6e7

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=302654753&repo=autoland&lineNumber=55007

Backout: https://hg.mozilla.org/integration/autoland/rev/23acc8bd888d53c2398a45d1ec591f45bde9d06b

Comment 15

[Intermittent Failures Robot]

336 failures in 5488 pushes (0.061 failures/push) were associated with this bug in the last 7 days.

This is the #1 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 230
• try: 106

Platform breakdown:

• android-em-7-0-x86_64: 32
• linux1804-64-qr: 35
• linux1804-64: 24
• macosx1014-64: 16
• windows10-64-qr: 34
• windows10-64: 22
• linux1804-64-asan: 20
• windows7-32: 26
• linux1804-64-ccov: 9
• windows7-32-shippable: 14
• linux1804-64-shippable-qr: 14
• linux1804-64-shippable: 14
• windows10-64-shippable-qr: 11
• windows10-64-shippable: 10
• macosx1014-64-shippable: 14
• linux1804-32-shippable: 13
• windows10-64-devedition: 8
• linux1804-64-devedition: 5
• windows7-32-devedition: 5
• macosx1014-64-devedition: 5
• windows10-64-ccov: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1637844&startday=2020-05-11&endday=2020-05-17&tree=all

Comment 16

[Intermittent Failures Robot]

134 failures in 718 pushes (0.187 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 97
• try: 37

Platform breakdown:

• windows7-32: 10
• linux1804-64-asan: 7
• linux1804-64-qr: 18
• linux1804-64: 13
• android-em-7-0-x86_64: 12
• windows10-64: 10
• windows10-64-qr: 14
• macosx1014-64: 6
• linux1804-64-ccov: 4
• windows10-64-shippable: 3
• windows10-64-shippable-qr: 5
• windows7-32-shippable: 5
• linux1804-64-shippable-qr: 5
• linux1804-64-shippable: 5
• linux1804-32-shippable: 5
• windows10-64-ccov: 4
• macosx1014-64-shippable: 4
• windows7-32-devedition: 1
• linux1804-64-devedition: 1
• windows10-64-devedition: 1
• macosx1014-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1637844&startday=2020-05-18&endday=2020-05-18&tree=all

Comment 17

[Intermittent Failures Robot]

124 failures in 1049 pushes (0.118 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 52
• mozilla-central: 72

Platform breakdown:

• windows10-64: 7
• windows10-64-qr: 11
• linux1804-64-qr: 17
• linux1804-64: 12
• macosx1014-64: 6
• android-em-7-0-x86_64: 13
• windows7-32: 8
• linux1804-64-asan: 4
• linux1804-64-ccov: 3
• windows7-32-shippable: 5
• windows10-64-ccov: 3
• windows10-64-shippable: 4
• windows10-64-shippable-qr: 2
• linux1804-32-shippable: 5
• linux1804-64-shippable: 7
• linux1804-64-shippable-qr: 5
• macosx1014-64-shippable: 5
• linux1804-64-devedition: 2
• windows10-64-devedition: 1
• windows7-32-devedition: 2
• macosx1014-64-devedition: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1637844&startday=2020-05-19&endday=2020-05-19&tree=all

Comment 18

[Intermittent Failures Robot]

74 failures in 959 pushes (0.077 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 50
• try: 24

Platform breakdown:

• windows7-32: 5
• linux1804-64-asan: 3
• windows10-64-qr: 7
• windows10-64: 7
• linux1804-64: 7
• linux1804-64-qr: 10
• android-em-7-0-x86_64: 6
• macosx1014-64: 3
• linux1804-64-ccov: 2
• windows10-64-shippable-qr: 3
• macosx1014-64-shippable: 3
• linux1804-32-shippable: 3
• linux1804-64-shippable-qr: 3
• windows10-64-ccov: 2
• linux1804-64-shippable: 2
• windows7-32-shippable: 2
• windows10-64-shippable: 2
• windows7-32-devedition: 1
• linux1804-64-devedition: 1
• macosx1014-64-devedition: 1
• windows10-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1637844&startday=2020-05-20&endday=2020-05-20&tree=all

Comment 19

[Andreea Pavel [:apavel]]

Failures here are fixed by bug 1637486.

Comment 20

[Intermittent Failures Robot]

337 failures in 4727 pushes (0.071 failures/push) were associated with this bug in the last 7 days.

This is the #1 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 219
• try: 118

Platform breakdown:

• windows7-32: 23
• linux1804-64-asan: 14
• linux1804-64-qr: 45
• linux1804-64: 32
• android-em-7-0-x86_64: 32
• windows10-64: 24
• windows10-64-qr: 32
• macosx1014-64: 15
• linux1804-64-ccov: 9
• windows10-64-shippable: 9
• windows10-64-shippable-qr: 10
• windows7-32-shippable: 13
• linux1804-64-shippable-qr: 14
• linux1804-64-shippable: 15
• linux1804-32-shippable: 13
• windows10-64-ccov: 9
• macosx1014-64-shippable: 13
• windows7-32-devedition: 4
• linux1804-64-devedition: 4
• windows10-64-devedition: 3
• macosx1014-64-devedition: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1637844&startday=2020-05-18&endday=2020-05-24&tree=all