1411822 - Intermittent /webrtc/RTCDTMFSender-ontonechange-long.https.html | insertDTMF with duration greater than 6000 should be clamped to 6000 - assert_approx_equals: Expect tonechange event for "" to be fired approximately after 6070 seconds expected 6070 +/- 5

Status: RESOLVED FIXED

(Core :: WebRTC, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: philringnalda [at] gmail.com

https://treeherder.mozilla.org/logviewer.html#?job_id=139540178&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/cbl28PWaS6u242uMLE9tOw/runs/0/artifacts/public/logs/live_backing.log

Comment 2

[Intermittent Failures Robot]

32 failures in 912 pushes (0.035 failures/push) were associated with this bug in the last 7 days.   

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 16
* mozilla-inbound: 9
* mozilla-central: 7

Platform breakdown:
* linux32-stylo-disabled: 11
* linux32: 10
* linux64: 6
* osx-10-10: 3
* linux64-stylo-disabled: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1411822&startday=2017-10-23&endday=2017-10-29&tree=all

Comment 3

[Henrietta Maior [:henrietta_maior]]

In the last 7 days, there have been 56 failures.
Most of the failures occur on Linux, but a few of them are on OS X 10.10. All of them on debug.

Here is a recent example of the log: https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=141254506&lineNumber=5530

And the log part where the test failed:
[task 2017-11-01T06:14:25.748Z] 06:14:25     INFO - PID 15773 | --DOMWINDOW == 11 (0xd15dcc00) [pid = 15821] [serial = 32] [outer = (nil)] [url = about:blank]
32237
[task 2017-11-01T06:14:25.748Z] 06:14:25     INFO - PID 15773 | --DOMWINDOW == 10 (0xd15d6c00) [pid = 15821] [serial = 26] [outer = (nil)] [url = about:blank]
32238
[task 2017-11-01T06:14:25.748Z] 06:14:25     INFO - PID 15773 | --DOMWINDOW == 9 (0xd15bd800) [pid = 15821] [serial = 25] [outer = (nil)] [url = about:blank]
32239
[task 2017-11-01T06:14:25.749Z] 06:14:25     INFO - PID 15773 | --DOMWINDOW == 8 (0xe18d9c00) [pid = 15821] [serial = 5] [outer = (nil)] [url = http://web-platform.test:8000/testharness_runner.html]
32240
[task 2017-11-01T06:14:26.012Z] 06:14:26     INFO - 
32241
[task 2017-11-01T06:14:26.012Z] 06:14:26     INFO - TEST-UNEXPECTED-FAIL | /webrtc/RTCDTMFSender-ontonechange-long.https.html | insertDTMF with duration greater than 6000 should be clamped to 6000 - assert_approx_equals: Expect tonechange event for "" to be fired approximately after 6070 seconds expected 6070 +/- 50 but got 6182
32242
[task 2017-11-01T06:14:26.012Z] 06:14:26     INFO - test_tone_change_events/</</onToneChange<@https://web-platform.test:8443/webrtc/RTCDTMFSender-helper.js:79:9
32243
[task 2017-11-01T06:14:26.013Z] 06:14:26     INFO - Test.prototype.step@https://web-platform.test:8443/resources/testharness.js:1489:20
32244
[task 2017-11-01T06:14:26.013Z] 06:14:26     INFO - Test.prototype.step_func/<@https://web-platform.test:8443/resources/testharness.js:1513:20
32245
[task 2017-11-01T06:14:26.014Z] 06:14:26     INFO - TEST-OK | /webrtc/RTCDTMFSender-ontonechange-long.https.html | took 6839ms

:jgraham, can you please help prioritize this bug?

Comment 4

[James Graham [:jgraham]]

Putting this in the WebRTC component. I suspect the issue is that the test is timing sensitive and debug builds can be slow. Hopefully someone working on WebRTC can fix the test to be more tolerant.

Comment 5

[Henrietta Maior [:henrietta_maior]]

:jgraham, thanks!

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

:drno, is this something you could look at?

Comment 7

[Intermittent Failures Robot]

62 failures in 857 pushes (0.072 failures/push) were associated with this bug in the last 7 days. 

This is the #27 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 28
* mozilla-inbound: 23
* mozilla-central: 6
* try: 5

Platform breakdown:
* linux32-stylo-disabled: 37
* linux32: 15
* linux64: 5
* linux64-ccov: 2
* osx-10-10: 1
* macosx64-stylo-disabled: 1
* linux64-stylo-disabled: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1411822&startday=2017-10-30&endday=2017-11-05&tree=all

Comment 8

[Nils Ohlmeier [:drno]]

Same problem as in bug 1398308: the expectations from the upstream tests are too tight.

Comment 9

[Nils Ohlmeier [:drno]]

Created upstream PR https://github.com/w3c/web-platform-tests/pull/8082

Comment 10

[James Graham [:jgraham]]

For this kind of change, unless we expect it to be controversial, it makes sense to land it directly in m-c so we get the benefit immediately and then later let the two-way sync handle upstreaming it.

Comment 11

[James Graham [:jgraham]]

(by m-c I obviously mean inbound or autoland)

Comment 12

[Nils Ohlmeier [:drno]]

Attachment: Bug 1411822: increase DTMF timeout range value.

Review commit: https://reviewboard.mozilla.org/r/196838/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/196838/

Comment 13

[Nils Ohlmeier [:drno]]

(In reply to James Graham [:jgraham] from comment #10)
> For this kind of change, unless we expect it to be controversial, it makes
> sense to land it directly in m-c so we get the benefit immediately and then
> later let the two-way sync handle upstreaming it.

I did not know that there is a two-way sync.
I also don't know if this is controversial. I assume not. So here we go.

Comment 14

[Michael Froman [:mjf]]

Comment on attachment 8925704 [details]
Bug 1411822: increase DTMF timeout range value.

https://reviewboard.mozilla.org/r/196838/#review202072

Looks good to me.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2167381ba501
increase DTMF timeout range value. r=mjf

Comment 16

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/2167381ba501

Comment 17

[Intermittent Failures Robot]

9 failures in 849 pushes (0.011 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 5
* autoland: 3
* try: 1

Platform breakdown:
* linux32-stylo-disabled: 4
* linux64: 2
* linux32: 2
* linux64-stylo-disabled: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1411822&startday=2017-11-06&endday=2017-11-12&tree=all