1421775 - Intermittent dom/media/tests/mochitest/test_peerConnection_transceivers.html | Test timed out.

Status: RESOLVED WORKSFORME

(Firefox for Android Graveyard :: Audio/Video, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: bclary [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=148552619&repo=autoland

https://autophone.s3.amazonaws.com/v1/task/dO1oxfltQzaQGGMWV1oCDw/runs/0/artifacts/public/build/b7aae30f-96db-44e5-883a-0f586c032546-autophone.log

Comment 1

[Tiberius Oros[:tiberius_oros]]

There have been 40 failures in the last 2 days.
This fails only on opt build type.

Occurrences per platform:

- 34 on android-6-0-armv8-api16
- 5 on android-4-4-armv7-api16
- 1 on windows10-64

This is a recent log file:
https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=148964096&lineNumber=902

And a snippet with the failure:

2017-12-01 01:50:35,659     nexus-6p-05 INFO     UnitTestJob mozilla-central 20171130233148 opt api-23 android-api-16 mochitest-dom-media-tests 339 INFO Adding ICE candidate: {"candidate":"candidate:3 1 TCP 2105524479 2620:101:80fc:120:31ee:9756:bb08:8d68 9 typ host tcptype active","sdpMid":"sdparta_0","sdpMLineIndex":0}
900
2017-12-01 01:50:35,659     nexus-6p-05 INFO     UnitTestJob mozilla-central 20171130233148 opt api-23 android-api-16 mochitest-dom-media-tests 340 INFO Adding ICE candidate: null
901
2017-12-01 01:50:35,659     nexus-6p-05 INFO     UnitTestJob mozilla-central 20171130233148 opt api-23 android-api-16 mochitest-dom-media-tests Buffered messages finished
902
2017-12-01 01:50:35,659     nexus-6p-05 INFO     UnitTestJob mozilla-central 20171130233148 opt api-23 android-api-16 mochitest-dom-media-tests 341 INFO TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_transceivers.html | Test timed out. 
903
2017-12-01 01:50:35,659     nexus-6p-05 INFO     UnitTestJob mozilla-central 20171130233148 opt api-23 android-api-16 mochitest-dom-media-tests     reportError@SimpleTest/TestRunner.js:121:7
904
2017-12-01 01:50:35,659     nexus-6p-05 INFO     UnitTestJob mozilla-central 20171130233148 opt api-23 android-api-16 mochitest-dom-media-tests     TestRunner._checkForHangs@SimpleTest/TestRunner.js:142:7

Comment 2

[Intermittent Failures Robot]

52 failures in 792 pushes (0.066 failures/push) were associated with this bug in the last 7 days. 

This is the #12 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 25
* mozilla-inbound: 18
* mozilla-central: 9

Platform breakdown:
* android-6-0-armv8-api16: 44
* android-4-4-armv7-api16: 7
* windows10-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2017-11-27&endday=2017-12-03&tree=all

Comment 3

[John Lin [:jhlin][:jolin]]

Byron, it looks like there were some time-outs in autophone tests since you landed 'test_peerConnection_transceivers.html' last week. Could you please take a look? Thanks a lot.

Comment 4

[Byron Campen [:bwc]]

Attachment: Bug 1421775: Await negotiationneeded before SLD for the other PC so we don't miss it.

Review commit: https://reviewboard.mozilla.org/r/205192/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/205192/

Comment 5

[Byron Campen [:bwc]]

I think I know what is happening here.

Comment 6

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Comment on attachment 8934280 [details]
Bug 1421775: Await negotiationneeded before SLD for the other PC so we don't miss it.

https://reviewboard.mozilla.org/r/205192/#review210770

Comment 7

[Byron Campen [:bwc]]

Under normal circumstances, I would try to elicit the failure with lots of retriggers, to check that the fix worked. However, since autophone is running on actual hardware in a lab, I'm just going to land this now. We can re-open if the fix doesn't work.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7cc99d649d75
Await negotiationneeded before SLD for the other PC so we don't miss it. r=jib

Comment 9

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/7cc99d649d75

Comment 10

[Intermittent Failures Robot]

17 failures in 157 pushes (0.108 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 11
* mozilla-central: 4
* mozilla-inbound: 2

Platform breakdown:
* android-6-0-armv8-api16: 15
* osx-10-10: 1
* android-4-4-armv7-api16: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2017-12-06&endday=2017-12-06&tree=all

Comment 11

[Byron Campen [:bwc]]

Ok, this does not look fixed.

Comment 12

[Byron Campen [:bwc]]

Attachment: Bug 1421775 - Attempt 2: Don't try to update negotiationneeded until after the stuff we do in parallel with sRD/sLD finishes.

Review commit: https://reviewboard.mozilla.org/r/206350/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/206350/

Comment 13

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Comment on attachment 8935458 [details]
Bug 1421775 - Attempt 2: Don't try to update negotiationneeded until after the stuff we do in parallel with sRD/sLD finishes.

https://reviewboard.mozilla.org/r/206350/#review212116

Comment 14

[Intermittent Failures Robot]

24 failures in 155 pushes (0.155 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* mozilla-inbound: 11
* autoland: 9
* mozilla-central: 4

Platform breakdown:
* android-6-0-armv8-api16: 15
* android-4-4-armv7-api16: 6
* osx-10-10: 2
* android-7-1-armv8-api16: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2017-12-07&endday=2017-12-07&tree=all

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a61bb7e683d9
Attempt 2: Don't try to update negotiationneeded until after the stuff we do in parallel with sRD/sLD finishes. r=jib

Comment 16

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/a61bb7e683d9

Comment 17

[Intermittent Failures Robot]

36 failures in 164 pushes (0.22 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 18
* mozilla-inbound: 13
* mozilla-central: 5

Platform breakdown:
* android-6-0-armv8-api16: 25
* android-4-4-armv7-api16: 9
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2017-12-08&endday=2017-12-08&tree=all

Comment 19

[Intermittent Failures Robot]

19 failures in 55 pushes (0.345 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 10
* mozilla-inbound: 6
* mozilla-central: 3

Platform breakdown:
* android-6-0-armv8-api16: 19

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2017-12-09&endday=2017-12-09&tree=all

Comment 20

[Intermittent Failures Robot]

145 failures in 889 pushes (0.163 failures/push) were associated with this bug in the last 7 days. 

This is the #4 most frequent failure this week. 

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **   

Repository breakdown:
* autoland: 75
* mozilla-inbound: 47
* mozilla-central: 23

Platform breakdown:
* android-6-0-armv8-api16: 115
* android-4-4-armv7-api16: 21
* osx-10-10: 6
* android-7-1-armv8-api16: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2017-12-04&endday=2017-12-10&tree=all

Comment 21

[Byron Campen [:bwc]]

This seems to have fixed itself somehow...

Comment 22

[Byron Campen [:bwc]]

Ok, so it definitely stopped happening on autophone on mozilla-inbound after the landing of Attempt 2. The remaining instances seem to be some other problem.

Comment 23

[Byron Campen [:bwc]]

So the remaining occurrences of this bug seem to be timeouts that happen _after_ TEST-OK has happened, and then warnings about finish() being called multiple times...

Comment 24

[Intermittent Failures Robot]

3 failures in 85 pushes (0.035 failures/push) were associated with this bug yesterday.    

** This test has failed more than 200 times in the last 30 days. It should be disabled until it can be fixed. ** 

Repository breakdown:
* autoland: 2
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2017-12-15&endday=2017-12-15&tree=all

Comment 25

[Intermittent Failures Robot]

4 failures in 423 pushes (0.009 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 2
* try: 1
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2017-12-11&endday=2017-12-17&tree=all

Comment 26

[Intermittent Failures Robot]

1 failures in 116 pushes (0.009 failures/push) were associated with this bug yesterday.    

** This test has failed more than 200 times in the last 30 days. It should be disabled until it can be fixed. ** 

Repository breakdown:
* try: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2017-12-19&endday=2017-12-19&tree=all

Comment 27

[Intermittent Failures Robot]

4 failures in 590 pushes (0.007 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 2
* try: 1
* mozilla-inbound: 1

Platform breakdown:
* osx-10-10: 3
* windows10-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2017-12-18&endday=2017-12-24&tree=all

Comment 28

[Intermittent Failures Robot]

11 failures in 462 pushes (0.024 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 6
* try: 2
* mozilla-inbound: 2
* mozilla-central: 1

Platform breakdown:
* windows10-64: 9
* windows10-64-nightly: 1
* linux64-qr: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2018-01-01&endday=2018-01-07&tree=all

Comment 29

[Intermittent Failures Robot]

19 failures in 788 pushes (0.024 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 11
* mozilla-inbound: 4
* try: 3
* mozilla-central: 1

Platform breakdown:
* windows10-64: 18
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2018-01-08&endday=2018-01-14&tree=all

Comment 30

[Byron Campen [:bwc]]

This seems to have stopped happening on the 13th. Not sure why. I guess we can keep an eye on it.

Comment 31

[Intermittent Failures Robot]

3 failures in 657 pushes (0.005 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1
* mozilla-central: 1
* autoland: 1

Platform breakdown:
* osx-10-10: 1
* linux64-stylo-disabled: 1
* linux64-qr: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2018-01-15&endday=2018-01-21&tree=all

Comment 32

[Intermittent Failures Robot]

6 failures in 701 pushes (0.009 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 3
* mozilla-inbound: 2
* mozilla-central: 1

Platform breakdown:
* windows10-64-qr: 5
* windows10-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2018-01-22&endday=2018-01-28&tree=all

Comment 33

[Intermittent Failures Robot]

1 failures in 702 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 1

Platform breakdown:
* windows10-64-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2018-02-05&endday=2018-02-11&tree=all

Comment 34

[Intermittent Failures Robot]

2 failures in 685 pushes (0.003 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 2

Platform breakdown:
* windows10-64-nightly: 1
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2018-02-12&endday=2018-02-18&tree=all

Comment 35

[Intermittent Failures Robot]

1 failures in 675 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 1

Platform breakdown:
* windows10-64-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2018-02-19&endday=2018-02-25&tree=all

Comment 36

[Intermittent Failures Robot]

1 failures in 831 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2018-02-26&endday=2018-03-04&tree=all

Comment 37

[Intermittent Failures Robot]

9 failures in 747 pushes (0.012 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 7
* mozilla-inbound: 1
* autoland: 1

Platform breakdown:
* linux64-qr: 4
* osx-10-10: 2
* linux64: 2
* linux64-stylo-disabled: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1421775&startday=2018-03-05&endday=2018-03-11&tree=all

Comment 38

[Intermittent Failures Robot]

1 failures in 521 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1421775&startday=2018-06-18&endday=2018-06-24&tree=trunk