Closed Bug 1032037 Opened 6 years ago Closed 6 years ago

perma-red on Travis for Music player tests - Playback tests

Categories

(Firefox OS Graveyard :: Gaia::Music, defect)

x86
macOS
defect
Not set

Tracking

(b2g-v2.1 fixed)

RESOLVED FIXED
Tracking Status
b2g-v2.1 --- fixed

People

(Reporter: rudyl, Unassigned)

References

Details

(Keywords: regression)

Attachments

(1 file)

I've seen this kind of failure a couple of times,
1) Music player tests Playback tests Check that progress bar updates when re-shown:

Example failing jobs,
1. https://travis-ci.org/mozilla-b2g/gaia/jobs/28749803
2. https://travis-ci.org/mozilla-b2g/gaia/jobs/28747151


====
1) Music player tests Playback tests Check that progress bar updates when re-shown:

AssertionError: Progress bar not updated!

at Context.<anonymous> (/home/travis/build/mozilla-b2g/gaia/apps/music/test/marionette/Player_test.js:112:7)

at callFn (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:223:21)

at Test.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:216:7)

at Runner.runTest (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:373:10)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:451:12

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:298:14)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:308:7

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:246:23)

at /home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:270:7

at done (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:185:5)

at callFn (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:228:7)

at Hook.Runnable.run (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runnable.js:216:7)

at next (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:258:10)

at Object._onImmediate (/home/travis/build/mozilla-b2g/gaia/node_modules/mocha/lib/runner.js:275:5)

at processImmediate [as _immediateCallback] (timers.js:330:15)
Can we get a regression window on this? Nothing much has changed in the music app lately.
Can't do a window on this - this is a test driven bug, not end user behavior.
Jim,

Do you agree we should disable the test on Travis and investigate it later or all build result will be red right now?

Thank you.
Attachment #8448595 - Flags: review?(squibblyflabbetydoo)
Comment on attachment 8448595 [details] [review]
Patch to disable the test

Also ask for Dominic's review to get the Travis passed as soon as possible.

Dominic, could you help on this?
Attachment #8448595 - Flags: review?(dkuo)
Comment on attachment 8448595 [details] [review]
Patch to disable the test

Thanks Rudy, if this is a perma-red test then we should skip it first to make master green(it will take to long to find the gecko regression...), please land it then Jim and I can try to bring back this test after we fixed it.
Attachment #8448595 - Flags: review?(squibblyflabbetydoo)
Attachment #8448595 - Flags: review?(dkuo)
Attachment #8448595 - Flags: review+
Landed to Gaia master,
https://github.com/mozilla-b2g/gaia/commit/ef1599767fb41f9a417a77a56f88c95912687a70

Dominic, thanks for the review.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
bug 1032037 filed for re-enabling the test.
(In reply to Jason Smith [:jsmith] from comment #2)
> Can't do a window on this - this is a test driven bug, not end user behavior.

Wait, why can't we do this? We'd just have to bisect the tree and then run the music tests until we find when it started failing. Given that we have historical logs of our test runs, it should be easy to get a rough idea of where to look even before doing the bisection.

I really don't agree that we should be disabling tests without doing even cursory investigation of why they broke.
Duplicate of this bug: 1033235
Uplifted to v2.0,
https://github.com/mozilla-b2g/gaia/commit/085cdbf16dfd5249786016ac8ceafa1a54e88497

--
Sorry that I don't have free cycles to do the bisect, especially when we doubt that this failure might be caused by Gecko change or something. We need to get Travis back on track since Gaia-Try is not totally ready yet.
(In reply to Jim Porter (:squib) from comment #8)
> (In reply to Jason Smith [:jsmith] from comment #2)
> > Can't do a window on this - this is a test driven bug, not end user behavior.
> 
> Wait, why can't we do this? We'd just have to bisect the tree and then run
> the music tests until we find when it started failing. Given that we have
> historical logs of our test runs, it should be easy to get a rough idea of
> where to look even before doing the bisection.
> 
> I really don't agree that we should be disabling tests without doing even
> cursory investigation of why they broke.

What Jason is saying is that it's not QA's job.
You need to log in before you can comment on or make changes to this bug.