Open
Bug 1305136
Opened 8 years ago
Updated 1 year ago
Intermittent dom/media/webaudio/test/test_audioContextSuspendResumeClose.html | Test timed out.
Categories
(Core :: Web Audio, defect, P5)
Core
Web Audio
Tracking
()
NEW
Tracking | Status | |
---|---|---|
firefox52 | --- | wontfix |
firefox-esr52 | --- | unaffected |
firefox53 | --- | unaffected |
firefox54 | --- | disabled |
firefox55 | --- | disabled |
People
(Reporter: intermittent-bug-filer, Unassigned)
References
Details
(Keywords: intermittent-failure, leave-open, Whiteboard: [stockwell disabled])
Attachments
(1 file, 2 obsolete files)
1.97 KB,
patch
|
karlt
:
review+
|
Details | Diff | Splinter Review |
Filed by: rvandermeulen [at] mozilla.com https://treeherder.mozilla.org/logviewer.html#?job_id=1656546&repo=mozilla-beta https://queue.taskcluster.net/v1/task/VTlOT7lHTluSC3aXZFK7xw/runs/0/artifacts/public%2Flogs%2Flive_backing.log
Updated•8 years ago
|
Rank: 35
Priority: -- → P3
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 12•7 years ago
|
||
:mreavy, this bug has increased in frequency around Feb 8th/9th, it looks to be primarily on linux- can we get someone to look into this sometime this week. Our goal is for all high frequency bugs to be resolved within 2 weeks (in the case end of the month)
Flags: needinfo?(mreavy)
Comment 13•7 years ago
|
||
Paul, Dan -- Looks like something landed in the last 2 weeks to bump the frequency of this. Thoughts? Dan -- Can I ask you to take this one? (At least to get the frequency back to where it was -- ideally I'd love to fix the root problem, but I'm not asking for that.)
Assignee: nobody → dminor
Rank: 35 → 21
Flags: needinfo?(padenot)
Flags: needinfo?(mreavy)
Flags: needinfo?(dminor)
Priority: P3 → P2
Comment 14•7 years ago
|
||
I don't know off hand what happened in this period. I would turn on MSG logging on (I think you need to modify part of the harness to do that). I recently landed a patch that makes MSG logging more useful. Look for GraphDriver messages and other driver things. Maybe add logging in `ApplyAudioCOntextOperation{Impl,Completed,}.
Flags: needinfo?(padenot)
Comment 16•7 years ago
|
||
All failures for the past two weeks have been here [1], which is inside testScriptProcessNodeSuspended, which tests that a ScriptProcessorNode does not receive events when it's context is suspended. I'll spend some time to try to identify a root cause for this failure, but ScriptProcessorNode is a deprecated feature of Web Audio and so at some point it might make more sense to remove this particular test case than to sink a bunch of time into a fix. That assumes, of course, that the problem doesn't just move to another test case once this one is removed. [1] http://searchfox.org/mozilla-central/rev/d3307f19d5dac31d7d36fc206b00b686de82eee4/dom/media/webaudio/test/test_audioContextSuspendResumeClose.html#234
Comment 17•7 years ago
|
||
Actually no, when I had a closer look at the log, there are multiple failures, they just aren't showing up in the Treeherder summary.
Comment hidden (Intermittent Failures Robot) |
Comment 19•7 years ago
|
||
I realize my credibility might be a little reduced after Comment 16, but it appears the problem is actually here [1]. I ran it on try with full logging here [2] and I'm only seeing "0 Offline" in the log. A successful run would contain both "0 Offline" and "O Realtime". I added some logging for finish() calls and it looks like we're still waiting on one last call to finish() when the test times out. [1] http://searchfox.org/mozilla-central/rev/d3307f19d5dac31d7d36fc206b00b686de82eee4/dom/media/webaudio/test/test_audioContextSuspendResumeClose.html#77 [2] https://treeherder.mozilla.org/#/jobs?repo=try&revision=71b7fe75f8337cf4fd9bd0cac2f4c53540196a5a&selectedJob=77331849
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 22•7 years ago
|
||
FWIW, the spike on Trunk looks like it has a pretty clear correlation to when bug 1336098 landed.
Blocks: 1336098
status-firefox52:
--- → affected
status-firefox53:
--- → affected
status-firefox54:
--- → affected
Flags: needinfo?(padenot)
Comment 23•7 years ago
|
||
Much to my annoyance, it's still a low-frequency intermittent on Beta (but not Aurora). Looks like something fixed it in mid/late-November until the recent spike. Anyway, I'm probably just going to disable this test on 52 soon since we're running out of time to backport any real fixes at this point.
Comment 24•7 years ago
|
||
There are a number of other media failures that stopped on trunk in mid-November but live on in 52. See my comment in bug 1293170 and some others.
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 27•7 years ago
|
||
(In reply to Ryan VanderMeulen [:RyanVM] from comment #22) > FWIW, the spike on Trunk looks like it has a pretty clear correlation to > when bug 1336098 landed. The only reasonable explanation for that correlation is timing changes due to the added test, increasing exposure of a latent bug.
Comment 28•7 years ago
|
||
(In reply to Dan Minor [:dminor] from comment #19) > I realize my credibility might be a little reduced after Comment 16, but it > appears the problem is actually here [1]. I ran it on try with full logging > here [2] and I'm only seeing "0 Offline" in the log. A successful run would > contain both "0 Offline" and "O Realtime". I added some logging for finish() > calls and it looks like we're still waiting on one last call to finish() > when the test times out. > > [1] > http://searchfox.org/mozilla-central/rev/ > d3307f19d5dac31d7d36fc206b00b686de82eee4/dom/media/webaudio/test/ > test_audioContextSuspendResumeClose.html#77 > [2] > https://treeherder.mozilla.org/#/ > jobs?repo=try&revision=71b7fe75f8337cf4fd9bd0cac2f4c53540196a5a&selectedJob=7 > 7331849 Have you had a chance to log in the functions that deal with the promises, so we have the log from the test interleaved with the log from gecko ?
Flags: needinfo?(padenot) → needinfo?(dminor)
Comment 29•7 years ago
|
||
Sorry, I didn't get to that last week, just started run here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=09b7402238b77de5551fc412b80ab6744856e09b
Flags: needinfo?(dminor)
Comment 30•7 years ago
|
||
It looks like the extra logging may be causing this to not reproduce. Here's a try run with the level reduced from 5 to 4: https://treeherder.mozilla.org/#/jobs?repo=try&revision=26dc18abfa4df7c4bb1d308597277afefc0878b0
Comment hidden (Intermittent Failures Robot) |
Comment 32•7 years ago
|
||
Here's a try run with some simple fprintf logging (look for '>>>>') that reproduces which makes it look the like the decode never even started for the normal AudioContext case: https://treeherder.mozilla.org/#/jobs?repo=try&author=dminor@mozilla.com&selectedJob=79134145 I tried adding some extra "todos" to the loadFile function in this try run here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=d29386a3d674cfb8dd91af6088ec95af3fd27300 to see if we were actually starting the decode process, but that seems to make this never reproduce. This might be a workaround if we were confident this was a test harness issue and not a real bug, but I'm not sure that is the case here. Here's a try run with MediaStreamGraph logging enabled that reproduces: https://treeherder.mozilla.org/#/jobs?repo=try&author=dminor@mozilla.com&selectedJob=79094920. Unfortunately, the log is not interleaved with the test execution or timestamped. I'll try diffing against a successful run to see if anything shows up.
Comment 33•7 years ago
|
||
Do we decode this with MediaDecoder? Have you tried adding logs there?
Comment 34•7 years ago
|
||
Based on this try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=eca47218b9740defcb8677e8468270318ef8f069 it looks like adding these extra "todos" might reduce the frequency of this intermittent while we investigate the root cause.
Attachment #8840068 -
Flags: review?(padenot)
Comment 35•7 years ago
|
||
(In reply to Andreas Pehrson [:pehrsons] (Telenor) from comment #33) > Do we decode this with MediaDecoder? Have you tried adding logs there? I've been adding logs to the MediaDecodeTask stuff in MediaBufferDecoder.cpp, but so far it seems like either it all succeeds or it doesn't even start, so maybe the problem is before that point. Suggestions welcome :)
Comment 36•7 years ago
|
||
Also possible that this call to close() is not completing and we never get to the decode at all: http://searchfox.org/mozilla-central/rev/b1044cf7c2000c3e75e8181e893236a940c8b6d2/dom/media/webaudio/test/test_audioContextSuspendResumeClose.html#304
Updated•7 years ago
|
Attachment #8840068 -
Flags: review?(padenot) → review+
Updated•7 years ago
|
Keywords: leave-open
Comment 37•7 years ago
|
||
Pushed by dminor@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/7d15bc99dbf1 Add additional status messages to loadFile function; r=padenot
Comment 38•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/7d15bc99dbf1
Comment 39•7 years ago
|
||
In this run [1], the last statement in the log is produced here [2] which makes it look like the ac.resume() just below is not completing. I've kicked off another try run with the MediaStreamGraph logs mixed in with the test output rather than in a separate file here [3] but so far no luck with getting it to reproduce with the extra logs in place. [1] https://treeherder.mozilla.org/#/jobs?repo=try&author=dminor@mozilla.com&selectedJob=79646925 [2] http://searchfox.org/mozilla-central/source/dom/media/webaudio/test/test_audioContextSuspendResumeClose.html#276 [3] https://treeherder.mozilla.org/#/jobs?repo=try&revision=0af8b59e282d7f78a1b4cebf3ebd191b34100e97
Comment hidden (Intermittent Failures Robot) |
Updated•7 years ago
|
Whiteboard: [stockwell needswork]
Comment 41•7 years ago
|
||
I discussed this with Maire and we thought it made sense to disable this test on Linux only as that seems to be the only place where we are running into trouble. I've had no luck getting this to reproduce locally or on Try with any sort of logging enabled.
Attachment #8840068 -
Attachment is obsolete: true
Attachment #8842524 -
Flags: review?(karlt)
Comment 42•7 years ago
|
||
(In reply to Dan Minor [:dminor] from comment #39) > In this run [1], the last statement in the log is produced here [2] which > makes it look like the ac.resume() just below is not completing. [1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=eaae566d5f462a4f44e29f8dc3d56f967cf547b5&selectedJob=79646925 [2] http://searchfox.org/mozilla-central/rev/31b6089ce26fa76459642765115605d50a6c67b4/dom/media/webaudio/test/test_audioContextSuspendResumeClose.html#276
Comment 43•7 years ago
|
||
Comment on attachment 8842524 [details] [diff] [review] Disable the test on Linux only Are all the subtests having intermittent timeouts? Can you point to evidence of this, or can logging be pushed to m-c to more clearly identify the remaining subtests before timeout? Is there a reason why all these tests run concurrently? Would it be reasonable to move the tests that are timing out to a different file? If not, I suggest removing only the problem subtests from the tests array when running on Linux.
Flags: needinfo?(dminor)
Attachment #8842524 -
Flags: review?(karlt)
Comment hidden (Intermittent Failures Robot) |
Comment 45•7 years ago
|
||
All of the failures I've seen are in the "testAudioContext" test case, and on Linux only, so I think it makes sense to disable just that test on Linux for now.
Attachment #8842524 -
Attachment is obsolete: true
Flags: needinfo?(dminor)
Attachment #8843916 -
Flags: review?(karlt)
Comment 46•7 years ago
|
||
Comment on attachment 8843916 [details] [diff] [review] Disable the testAudioContext test case on Linux only Thanks! I agree that there is no obvious indication that the order of the tests should matter (from a quick skim through).
Attachment #8843916 -
Flags: review?(karlt) → review+
Updated•7 years ago
|
Whiteboard: [stockwell needswork] → [stockwell disabled]
Comment 47•7 years ago
|
||
Pushed by dminor@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/0060ab5eefa8 Disable testAudioContext test in audioContextSuspendResumeClose.html on Linux; r=karlt
Comment 48•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/0060ab5eefa8
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 52•7 years ago
|
||
It looks like the patch to disable the test never made it to Aurora. It should be fine to land this a=test-only.
Keywords: checkin-needed
Updated•7 years ago
|
status-firefox55:
--- → disabled
status-firefox-esr52:
--- → unaffected
Comment 53•7 years ago
|
||
uplift |
https://hg.mozilla.org/releases/mozilla-aurora/rev/5725d27c15a6
Keywords: checkin-needed
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 63•7 years ago
|
||
Mass change P2->P3 to align with new Mozilla triage process.
Priority: P2 → P3
Comment 64•7 years ago
|
||
Bulk priority update of open intermittent test failure bugs. P3 => P5 https://bugzilla.mozilla.org/show_bug.cgi?id=1381960
Priority: P3 → P5
Comment 65•7 years ago
|
||
Unassigning myself, this continues to occur at low frequency on esr-52.
Assignee: dminor → nobody
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Updated•1 year ago
|
Severity: normal → S3
You need to log in
before you can comment on or make changes to this bug.
Description
•