1524642 - RTCRtpTransceiver wpt has been quietly disabled

Status: RESOLVED FIXED

(Core :: WebRTC: Signaling, enhancement, P1)

Description

[Byron Campen [:bwc]]

It sure would be great if someone would let me know before they disable one of my tests...

Comment 1

[Byron Campen [:bwc]]

When I try to re-enable our tests, I trip over bug 1522773.

Comment 2

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=db2dc7d5dab4808eedafe6eaf018b376230467fa

Comment 3

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=02aab7f0a9de1e365e6d2322bba63e9a013f4860

Comment 4

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=cf10c716afda6ff1b1c487471453a2d34dee8c66

Comment 5

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=fbac9be320e2db23f35aa8193f4f5ad0817ce706

Comment 6

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b7c02174c732d820a3792c12bda0678bd9993401

Comment 7

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=bb6389c54f6e47565faa579501dcdf32d466ea34

Comment 8

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2aa6aafbb076aa97984402230481cc219c65d0d8

Comment 9

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ae091bc123045ac0e4dfe3fdb45b3a679074e111

Comment 10

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=67304302473794908ec9791caac986d21a0172ce

Comment 11

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6054127a4c6f908d38dd0e63955e625fe3e7b20d

Comment 12

[Andreas Pehrson [:pehrsons]]

I'm assuming the webkit patch in bug 1514432 busted this test for us and made it quietly disabled on the following metadata sync. James, can problems like this be mitigated somehow? Like automatic filing of a bug when a WPT-sync makes our support for a WPT-test worse?

Comment 13

[James Graham [:jgraham]]

The intent is that there is always a bug when something is disabled. In this case a harness change caused a bunch of new instability, blocking all wpt imports. In the scramble to fix that some bugs that ought to have been filed weren't.

So this situation already isn't supposed to happen, and I'm also working on tooling to give people better insights into the status of web-platform-tests in general so it will be even harder for something like this to happen again without people becoming aware.

Comment 14

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=37abc16172aad9bf17da3d08d6f3eab8a0979b07

Comment 15

[Byron Campen [:bwc]]

(In reply to Andreas Pehrson [:pehrsons] from comment #12)

I'm assuming the webkit patch in bug 1514432 busted this test for us and made it quietly disabled on the following metadata sync. James, can problems like this be mitigated somehow? Like automatic filing of a bug when a WPT-sync makes our support for a WPT-test worse?

This was one of the problems, but there are much worse things that broke. I am trying to fix/compensate for them here.

Comment 16

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=67adbbade1395d92a4723fd8ddd380e00c7d5b7d

Comment 17

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b30110080e88f085c2090ef66e285a1225702a3c

Comment 18

[Byron Campen [:bwc]]

I wonder why so many try pushes aren't running CI...

https://treeherder.mozilla.org/#/jobs?repo=try&revision=232543cb549957855126d6d152362d9ff11c83b9

Comment 19

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6d3a5dbca6ddfb5a77497221ff9490724d7fdae7

Comment 20

[Byron Campen [:bwc]]

Attachment: Bug 1524642 - Part 1: Re-enable/fix some webrtc wpt. r?jib

Comment 21

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a5af552d17813489c50c0a1ce20fc7e25219abf5

Comment 22

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b799bbd328123c1d315de4e433e97a24f90af518

Comment 23

[Byron Campen [:bwc]]

Attachment: Bug 1524642 - Part 2: In getReceivers/Senders, do not include stopped transceivers. r?jib

Depends on D19548

Comment 24

[Byron Campen [:bwc]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0ca5430661c1df2aee4907aa63149d12a347a073

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1318991460b3
Part 1: Re-enable/fix some webrtc wpt. r=jib
https://hg.mozilla.org/integration/autoland/rev/804dc60558df
Part 2: In getReceivers/Senders, do not include stopped transceivers. r=jib

Comment 26

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/1318991460b3
https://hg.mozilla.org/mozilla-central/rev/804dc60558df

Comment 27

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/15416 for changes under testing/web-platform/tests

Comment 28

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/15416
* Taskcluster (pull_request) (https://tools.taskcluster.net/task-group-inspector/#/DM56lTwGQbmX5A_x_WCNwg)

Comment 29

[James Graham [:jgraham]]

This didn't merge upstream for failures in Firefox stability checking. I can force-merge the changes if you like, but it seems like there are still real issues to figure out with this test.