1633563 - Intermittent TinderboxPrint: Per-test run of .../tests/webrtc/<test><br/>: FAILURE

Status: RESOLVED INCOMPLETE

(Core :: WebRTC, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: apavel [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=299637635&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/DgBQdZQfRIOw5a493XCB6A/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/DgBQdZQfRIOw5a493XCB6A/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

[task 2020-04-27T20:11:27.021Z] 20:11:27 INFO - TEST-OK | /webrtc/RTCPeerConnection-iceGatheringState.html | took 8322ms
[task 2020-04-27T20:11:27.022Z] 20:11:27 INFO - No more tests
[task 2020-04-27T20:11:27.029Z] 20:11:27 INFO - PID 3072 | 1588018287012 Marionette INFO Stopped listening on port 49382
[task 2020-04-27T20:11:27.110Z] 20:11:27 INFO - PID 3072 | [Child 5064: Main Thread]: E/signaling [main|PeerConnectionImpl] PeerConnectionImpl.cpp:2633: GetStats: Found no pipelines matching selector.
[task 2020-04-27T20:11:27.111Z] 20:11:27 INFO - PID 3072 | [Child 5064: Main Thread]: I/signaling [main|PeerConnectionImpl] PeerConnectionImpl.cpp:2022: CloseInt: Closing PeerConnectionImpl 9f77ea5c4c78c3b6; ending call
[task 2020-04-27T20:11:27.111Z] 20:11:27 INFO - PID 3072 | [Child 5064: Main Thread]: I/jsep [1588018285106000 (id=27917287426 url=http://web-platform.test:8000/webrtc/RTCPeerConnection-iceGatheringState.html)]: stable -> closed
[task 2020-04-27T20:11:33.230Z] 20:11:33 INFO - PID 3072 | [Parent 3560, Gecko_IOThread] WARNING: file /builds/worker/checkouts/gecko/ipc/chromium/src/base/process_util_win.cc, line 166
[task 2020-04-27T20:11:33.437Z] 20:11:33 INFO - Browser exited with return code 0
[task 2020-04-27T20:11:33.437Z] 20:11:33 INFO - PROCESS LEAKS None
[task 2020-04-27T20:11:33.437Z] 20:11:33 INFO - Closing logging queue
[task 2020-04-27T20:11:33.437Z] 20:11:33 INFO - queue closed
[task 2020-04-27T20:11:33.438Z] 20:11:33 INFO - PROCESS LEAKS None
[task 2020-04-27T20:11:33.505Z] 20:11:33 INFO - Got 0 unexpected results
[task 2020-04-27T20:11:33.505Z] 20:11:33 INFO - SUITE-END | took 89s
[task 2020-04-27T20:11:33.548Z] 20:11:33 INFO - Closing logging queue
[task 2020-04-27T20:11:33.549Z] 20:11:33 INFO - queue closed
[task 2020-04-27T20:11:33.636Z] 20:11:33 INFO - ## All results ##
[task 2020-04-27T20:11:33.636Z] 20:11:33 INFO -
[task 2020-04-27T20:11:33.636Z] 20:11:33 INFO - ### /webrtc/RTCPeerConnection-iceGatheringState.html ###
[task 2020-04-27T20:11:33.636Z] 20:11:33 INFO - | Subtest | Results | Messages |
[task 2020-04-27T20:11:33.636Z] 20:11:33 INFO - |------------------------------------------------------------------------------------------------------------------------------|---------|------------------------------------------------------------|
[task 2020-04-27T20:11:33.636Z] 20:11:33 INFO - | | OK | |
[task 2020-04-27T20:11:33.637Z] 20:11:33 INFO - | Initial iceGatheringState should be new | PASS | |
[task 2020-04-27T20:11:33.637Z] 20:11:33 INFO - | iceGatheringState should eventually become complete after setLocalDescription | PASS | |
[task 2020-04-27T20:11:33.637Z] 20:11:33 INFO - | setLocalDescription(reoffer) with no new transports should not cause iceGatheringState to change | PASS | |
[task 2020-04-27T20:11:33.637Z] 20:11:33 INFO - | rolling back an ICE restart when gathering is complete should not result in iceGatheringState changes | PASS | |
[task 2020-04-27T20:11:33.637Z] 20:11:33 INFO - | setLocalDescription(reoffer) with a new transport should cause iceGatheringState to go to "checking" and then "complete" | PASS | |
[task 2020-04-27T20:11:33.637Z] 20:11:33 INFO - | sRD does not cause ICE gathering state changes | PASS | |
[task 2020-04-27T20:11:33.637Z] 20:11:33 INFO - | setLocalDescription(rollback) of original offer should cause iceGatheringState to reach "new" when starting in "complete" | PASS | |
[task 2020-04-27T20:11:33.637Z] 20:11:33 INFO - | setLocalDescription(rollback) of original offer should cause iceGatheringState to reach "new" when starting in "gathering" | PASS | |
[task 2020-04-27T20:11:33.638Z] 20:11:33 INFO - | renegotiation that closes all transports should result in ICE gathering state "new" | PASS | |
[task 2020-04-27T20:11:33.638Z] 20:11:33 INFO - | connection with one data channel should eventually have connected connection state | FAIL | can't access property "transport", pc2.sctp is undefined |
[task 2020-04-27T20:11:33.638Z] 20:11:33 INFO -
[task 2020-04-27T20:11:33.638Z] 20:11:33 INFO - ## Slow tests ##
[task 2020-04-27T20:11:33.638Z] 20:11:33 INFO -
[task 2020-04-27T20:11:33.638Z] 20:11:33 INFO - | Test | Result | Longest duration (ms) | Timeout (ms) |
[task 2020-04-27T20:11:33.638Z] 20:11:33 INFO - |----------------------------------------------------|--------|-----------------------|--------------|
[task 2020-04-27T20:11:33.638Z] 20:11:33 INFO - | /webrtc/RTCPeerConnection-iceGatheringState.html | OK | 9321 | 10000 |
[task 2020-04-27T20:11:33.638Z] 20:11:33 INFO -
[task 2020-04-27T20:11:33.639Z] 20:11:33 INFO - ::: Running tests in a loop 10 times : FAIL
[task 2020-04-27T20:11:33.639Z] 20:11:33 INFO - :::
[task 2020-04-27T20:11:33.639Z] 20:11:33 ERROR - ::: Test verification FAIL
[task 2020-04-27T20:11:33.639Z] 20:11:33 INFO - :::
[task 2020-04-27T20:11:33.731Z] 20:11:33 ERROR - Return code: 1
[task 2020-04-27T20:11:33.732Z] 20:11:33 ERROR - # TBPL FAILURE #

Comment 1

[Intermittent Failures Robot]

1 failures in 5423 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• windows7-32-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-04-27&endday=2020-05-03&tree=all

Comment 2

[Intermittent Failures Robot]

12 failures in 5488 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 12

Platform breakdown:

• windows7-32-shippable: 3
• windows10-64: 1
• windows7-32: 1
• linux1804-64: 1
• macosx1014-64: 1
• linux1804-64-shippable: 1
• windows10-64-shippable: 1
• macosx1014-64-shippable: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-05-11&endday=2020-05-17&tree=all

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 4

[Dorel Luca [:dluca]]

Recent log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=306804692&repo=autoland&lineNumber=3781

Comment 5

[Intermittent Failures Robot]

2 failures in 4035 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform breakdown:

• windows10-64-shippable: 1
• windows7-32-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-06-15&endday=2020-06-21&tree=all

Comment 6

[Razvan Maries]

There are 22 total failures in the last 7 days: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2020-06-19&endday=2020-06-26&tree=trunk&bug=1633563

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=307683016&repo=autoland&lineNumber=1203

Dan, could you assign this to someone?

Comment 7

[Dan Minor [:dminor]]

So part of the problem here is that it appears the bug name "Test verification FAIL" will match any wpt test that fails on test verification.

There are webrtc test failures here, but also things like:

TinderboxPrint: Per-test run of .../2d.text.measure.actualBoundingBox.html<br/>: FAILURE
TinderboxPrint: Per-test run of .../feature-policy-header-policy-disallowed-for-all.https.sub.html<br/>: FAILURE
TinderboxPrint: Per-test run of .../cookieListItem_attributes.tentative.https.any.js<br/>: FAILURE
TinderboxPrint: Per-test run of .../require-corp-about-blank.html<br/>: FAILURE

etc.

It looks like the most common webrtc related failure is:

TinderboxPrint: Per-test run of .../tests/webrtc/simplecall.https.html<br/>: FAILURE

but that's far from the majority of the failures being tracked by this bug. Byron has been looking at our wpt intermittents, so I'm going to needinfo him, but I think the first step is to rename this so at the very least only WebRTC related stuff ends up in here. I think one bug per failing test would be more useful.

Comment 8

[Intermittent Failures Robot]

22 failures in 5252 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 22

Platform breakdown:

• macosx1014-64: 8
• linux1804-64: 5
• linux1804-64-shippable: 3
• macosx1014-64-shippable: 2
• windows10-64: 2
• windows7-32: 1
• windows10-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-06-22&endday=2020-06-28&tree=all

Comment 9

[Byron Campen [:bwc]]

Rate seems to be way down.

Comment 10

[Intermittent Failures Robot]

4 failures in 5265 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4

Platform breakdown:

• linux1804-64: 1
• macosx1014-64: 1
• macosx1014-64-shippable: 1
• linux1804-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-07-06&endday=2020-07-12&tree=all

Comment 11

[Intermittent Failures Robot]

3 failures in 4310 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform breakdown:

• macosx1014-64: 1
• macosx1014-64-shippable: 1
• linux1804-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-07-20&endday=2020-07-26&tree=all

Comment 12

[Intermittent Failures Robot]

1 failures in 4618 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• macosx1014-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-08-03&endday=2020-08-09&tree=all

Comment 13

[Intermittent Failures Robot]

2 failures in 4046 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform breakdown:

• windows7-32: 1
• linux1804-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-08-24&endday=2020-08-30&tree=all

Comment 14

[Intermittent Failures Robot]

8 failures in 2984 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 8

Platform breakdown:

• windows7-32: 3
• macosx1014-64: 4
• linux1804-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-08-31&endday=2020-09-06&tree=all

Comment 15

[Intermittent Failures Robot]

4 failures in 3410 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4

Platform breakdown:

• macosx1014-64: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-09-07&endday=2020-09-13&tree=all

Comment 16

[Intermittent Failures Robot]

3 failures in 3760 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform breakdown:

• macosx1014-64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-09-14&endday=2020-09-20&tree=all

Comment 17

[Intermittent Failures Robot]

9 failures in 4019 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 9

Platform breakdown:

• macosx1014-64: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-09-21&endday=2020-09-27&tree=all

Comment 18

[Intermittent Failures Robot]

9 failures in 4195 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 9

Platform breakdown:

• macosx1014-64: 3
• linux1804-64: 2
• windows7-32: 2
• windows10-64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-09-28&endday=2020-10-04&tree=all

Comment 19

[Intermittent Failures Robot]

11 failures in 4130 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 11

Platform breakdown:

• macosx1014-64: 6
• windows7-32: 2
• linux1804-64: 2
• windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-10-05&endday=2020-10-11&tree=all

Comment 20

[Intermittent Failures Robot]

1 failures in 3694 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• macosx1014-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633563&startday=2020-10-12&endday=2020-10-18&tree=all

Comment 21

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.