1643058 - webrtc web-platform-tests need a cleanup

Status: RESOLVED FIXED

(Core :: WebRTC: Signaling, task, P2)

Description

[Byron Campen [:bwc]]

There are lots of disabled webrtc wpt, with no bug numbers in the ini files. The ini files need to be evaluated, tests re-enabled if possible, and bug links added if not.

Comment 1

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=7883b898d7c9faa9ef4e2bc05f8fc76ac1c6d035

Comment 2

[Byron Campen [:bwc]]

Seeing lots of instances of bug 1641788 in that push.

Comment 3

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=cfba37f3d049bb81e58b413080b28ad300f2ecdb

Comment 4

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=34292503a9bb141def47d2875f2767ddc569fd5c

Comment 5

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0824379ff4fee9ec80a6f2b471f6ac9ba812e776

Comment 6

[Byron Campen [:bwc]]

Push without fuzzy --full: Only seeing known intermittents. https://treeherder.mozilla.org/#/jobs?repo=try&revision=fd392d1975c6e8ac19e8d77cc984d24fb4ae2ac9

Push with fuzzy --full: Mostly known intermittents here, but there are some failures on android that I don't see in bugzilla anywhere: https://treeherder.mozilla.org/#/jobs?repo=try&revision=d1edc0bf860fc9a0e8bc43812e5b31c7e777e5a1

Here's a push on the base revision to check out those Android failures: https://treeherder.mozilla.org/#/jobs?repo=try&revision=32debcb7d02700c2c724ac230492a103488e42d6

Comment 7

[Byron Campen [:bwc]]

The failures on fuzzy --full android happen on the base revision too. So, try looks good. Let's see if that holds up after a rebase!

Comment 8

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e7999e830f67f35f71d0722c473ae1a15157ad81

fuzzy --full: https://treeherder.mozilla.org/#/jobs?repo=try&revision=5bf417ca1cb6c748b6f9c0516918ce6a1069bb13

Comment 9

[Byron Campen [:bwc]]

Attachment: Bug 1643058: Clean up ini files.

Comment 10

[Byron Campen [:bwc]]

Attachment: Bug 1643058: Try to work around bug 1641237 by adding some restart-after on longer tests.

Depends on D79527

Comment 11

[Byron Campen [:bwc]]

Attachment: Bug 1643058: Clean up PeerConnections in these tests.

Depends on D79528

Comment 12

[Byron Campen [:bwc]]

Attachment: Bug 1643058: Mark these tests as long, because they do ICE.

Depends on D79529

Comment 13

[Byron Campen [:bwc]]

Attachment: Bug 1643058: Timestamp mDNS lookups.

Depends on D79530

Comment 14

[Byron Campen [:bwc]]

Attachment: Bug 1643058: Add a restart-after to avoid timeouts in the next test on win 7 debug.

Depends on D79531

Comment 15

[Byron Campen [:bwc]]

Try looks good.

Comment 16

[Byron Campen [:bwc]]

Try from rebase:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1c5a530a0eb86690e2dda358c9f6a1dfcd33e8b0

With --full: https://treeherder.mozilla.org/#/jobs?repo=try&revision=29be5cd47d90094d7f45ce5d55d17c83b4c97288

Comment 17

[Byron Campen [:bwc]]

Lots of bug 1645663 in those pushes. Also seeing some intermittents that bug 1645219 fixes.

Comment 18

[Byron Campen [:bwc]]

Hit merge conflicts when trying to land, so here are some more try pushes!

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0dcb3cba323c512c5930e8368fe9c16d94952a61

Comment 19

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9ac5a4ae2e5306ea26659b9a6e9b2038ba1b48d0

Running full wpt suite, not just webrtc: https://treeherder.mozilla.org/#/jobs?repo=try&revision=1ff29dadfe1514b2e059d381ced3ffd570d864e0

fuzzy --full: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a64b9fef305d1aecc17daaf3134b3748ca57cdfc

Comment 20

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8778f5cb6315ee7c810295dd746229c27fbeac2b

Comment 21

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b726e3034fe0251d08363b555111687bfd290d7a

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e6edf6df8183
Clean up ini files. r=jib
https://hg.mozilla.org/integration/autoland/rev/c4f71ccf6934
Try to work around bug 1641237 by adding some restart-after on longer tests. r=jib
https://hg.mozilla.org/integration/autoland/rev/7d90312af10f
Clean up PeerConnections in these tests. r=jib
https://hg.mozilla.org/integration/autoland/rev/12c6d279b173
Mark these tests as long, because they do ICE. r=jib
https://hg.mozilla.org/integration/autoland/rev/fcc0942206d9
Timestamp mDNS lookups. r=dminor
https://hg.mozilla.org/integration/autoland/rev/6dd69c93d9e2
Add a restart-after to avoid timeouts in the next test on win 7 debug. r=jib

Comment 23

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/24323 for changes under testing/web-platform/tests

Comment 24

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/24323
* Community-TC (pull_request) (https://community-tc.services.mozilla.com/tasks/groups/G9FV0ksDRWeATECNuafG7A)

Comment 25

[Dorel Luca [:dluca]]

Backed out 6 changesets (bug 1643058) for WPT failures in webrtc/RTCRtpReceiver-getSynchronizationSources.https.html. CLOSED TREE

Log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=307330485&repo=autoland&lineNumber=11620

Push with failure:
https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=6dd69c93d9e2a9f06e7404f4655207adf6a22fdd

Backout:
https://hg.mozilla.org/integration/autoland/rev/8245e9943d9de53b9f3a3a3193704b9b4d49400f

Comment 26

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR was closed without merging

Comment 27

[Byron Campen [:bwc]]

Yeah, the merge was bad. Trying again...

Comment 28

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=5b3d91734d0185f4e5bf438d6c4dd1961120099e

Comment 29

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=cf0156fd361451cb06a56e424702d3b09a69ea33

Whole suite: https://treeherder.mozilla.org/#/jobs?repo=try&revision=58f33f8c190ce768a548822a60b97b8bf1c83504

Comment 30

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=88f03cafe97ac8dc1e10c947b8bd54191ce04434

Comment 31

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=eb5941e5e5422fd141e5a4a51c35815fc2f5c6b2

Comment 32

[Byron Campen [:bwc]]

Attachment: Bug 1643058: Make these tests fail instead of timing out if ICE fails.

Depends on D79532

Comment 33

[Byron Campen [:bwc]]

Attachment: Bug 1643058: Add bug link for this failure.

Depends on D80947

Comment 34

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ebad65f70e2d
Clean up ini files. r=jib
https://hg.mozilla.org/integration/autoland/rev/b32e18c3d542
Try to work around bug 1641237 by adding some restart-after on longer tests. r=jib
https://hg.mozilla.org/integration/autoland/rev/4767df198c29
Clean up PeerConnections in these tests. r=jib
https://hg.mozilla.org/integration/autoland/rev/cf062b9c00b4
Mark these tests as long, because they do ICE. r=jib
https://hg.mozilla.org/integration/autoland/rev/9c677d996512
Timestamp mDNS lookups. r=dminor
https://hg.mozilla.org/integration/autoland/rev/141ade17eb3c
Add a restart-after to avoid timeouts in the next test on win 7 debug. r=jib
https://hg.mozilla.org/integration/autoland/rev/36f54439ad2e
Make these tests fail instead of timing out if ICE fails. r=jib
https://hg.mozilla.org/integration/autoland/rev/78d7201bf2a0
Add bug link for this failure. r=jib

Comment 35

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/24323
* Community-TC (pull_request) (https://community-tc.services.mozilla.com/tasks/groups/Z-QCfGlhS4Skt_ZpISLsdA)

Comment 36

[Byron Campen [:bwc]]

/webrtc/RTCPeerConnection-createDataChannel.html seems to be having unexpected errors on Chrome, but the only change there was marking the test as long.
/webrtc/RTCPeerConnection-setRemoteDescription-tracks.https.html is the same story.
/webrtc/datachannel-emptystring.html just had some necessary cleanup added.
/webrtc/no-media-call.html just had some necessary cleanup added.

Looks like Chrome needs to update their test expectations.

Comment 37

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/ebad65f70e2d
https://hg.mozilla.org/mozilla-central/rev/b32e18c3d542
https://hg.mozilla.org/mozilla-central/rev/4767df198c29
https://hg.mozilla.org/mozilla-central/rev/cf062b9c00b4
https://hg.mozilla.org/mozilla-central/rev/9c677d996512
https://hg.mozilla.org/mozilla-central/rev/141ade17eb3c
https://hg.mozilla.org/mozilla-central/rev/36f54439ad2e
https://hg.mozilla.org/mozilla-central/rev/78d7201bf2a0

Comment 38

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by jgraham