Intermittent ERROR test_basic_stream_playback.py TestBasicStreamPlayback.test_video_playback_partial

RESOLVED FIXED

Status

Testing Graveyard
external-media-tests
RESOLVED FIXED
a year ago
5 months ago

People

(Reporter: Treeherder Bug Filer, Unassigned)

Tracking

({intermittent-failure})

Version 3
intermittent-failure

Details

(Whiteboard: [stockwell disabled])

See Also: → bug 1344961
Not sure what's going on here. The harness is failing to detect the video element on the page, but the screenshot indicates that the page is loaded and the video is present. There is a potential lag between the screenshot being taken and the failure, but I'm dubious as to if that's what's going on. If this happens again, I'll instrument the tests to try and better identify what's going on here.
Hrm, had a look and found some more of these. Going to add further logging to the tests to try and identify what's going on.

Comment 3

a year ago
11 failures in 790 pushes (0.014 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-aurora: 8
* try: 1
* mozilla-central: 1
* graphics: 1

Platform breakdown:
* linux64: 5
* linux32: 4
* windowsxp: 1
* linux32-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-03-06&endday=2017-03-12&tree=all

Comment 4

a year ago
40 failures in 777 pushes (0.051 failures/push) were associated with this bug in the last 7 days. 

This is the #41 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* try: 11
* autoland: 11
* mozilla-inbound: 9
* mozilla-central: 7
* oak: 1
* graphics: 1

Platform breakdown:
* linux64: 17
* linux32: 15
* windows7-32-vm: 4
* windows10-64-vm: 3
* linux32-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-03-13&endday=2017-03-19&tree=all
Linux media tests on TaskCluster are in the process of being disabled to address this failure rate.
Whiteboard: [stockwell needswork]

Comment 6

a year ago
24 failures in 174 pushes (0.138 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* mozilla-central: 7
* autoland: 7
* mozilla-inbound: 5
* try: 3
* mozilla-aurora: 2

Platform breakdown:
* linux64: 16
* linux32: 4
* windows7-32-vm: 2
* windows10-64-vm: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-03-21&endday=2017-03-21&tree=all

Comment 7

a year ago
20 failures in 186 pushes (0.108 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* mozilla-inbound: 12
* mozilla-aurora: 3
* mozilla-central: 2
* try: 1
* oak: 1
* autoland: 1

Platform breakdown:
* linux64: 10
* linux32: 7
* windows10-64-vm: 2
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-03-22&endday=2017-03-22&tree=all
the failure rate has gone way down after disabling the tests on linux, thanks for doing that.
Whiteboard: [stockwell needswork] → [stockwell disabled]

Comment 9

a year ago
72 failures in 898 pushes (0.08 failures/push) were associated with this bug in the last 7 days. 

This is the #12 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-inbound: 21
* mozilla-central: 18
* mozilla-aurora: 13
* autoland: 12
* try: 4
* oak: 3
* graphics: 1

Platform breakdown:
* linux64: 40
* linux32: 18
* windows7-32-vm: 7
* windows10-64-vm: 7

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-03-20&endday=2017-03-26&tree=all
19 failures in 891 pushes (0.021 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-central: 18
* try: 1

Platform breakdown:
* windows10-64-vm: 11
* windows7-32-vm: 8

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-05-22&endday=2017-05-28&tree=all
17 failures in 175 pushes (0.097 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* mozilla-central: 17

Platform breakdown:
* windows7-32-vm: 9
* windows10-64-vm: 8

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-06-22&endday=2017-06-22&tree=all
this started failing June 21st around midnight on windows opt, we should look into this with 20+ failures/day unless this is a weird spike for a day.

:singingtree, do you know if much has changed with these tests recently?  Can you help look into this?
Flags: needinfo?(bvandyk)
these are only failing on m-c, this makes it difficult to figure out the root cause of failure, but I assume they are only running on mozilla-central.
Whiteboard: [stockwell disabled] → [stockwell needswork]
I'm uncertain as to why we'd see a specific uptick in failures at that time. These tests are about to be retired as part of Bug 1362252 and the more immediate Bug 1372059. I will examine the increased failure rate in the coming week to see if I can identify issues causing (legitimate) test failures. Given the above, we don't need to worry about further failures, but I'll see if the tests can deliver value in their swan song.
Flags: needinfo?(bvandyk)
28 failures in 892 pushes (0.031 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-central: 26
* try: 2

Platform breakdown:
* windows7-32-vm: 14
* windows10-64-vm: 13
* linux64-ccov: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-06-19&endday=2017-06-25&tree=all
:singingtree, when will these be disabled?  we still see many failures
Flags: needinfo?(bvandyk)

Comment 17

11 months ago
The taskcluster tests were removed from tree ~23 June. Looking at brasstacks I see a falloff at that date. Can you confirm that there is still issue?
Flags: needinfo?(bvandyk) → needinfo?(jmaher)
I apologize for asking for info on this and not paying attention to the dates- this has been silent and being removed is a good explanation for it.
Status: NEW → RESOLVED
Last Resolved: 11 months ago
Flags: needinfo?(jmaher)
Resolution: --- → FIXED
Whiteboard: [stockwell needswork] → [stockwell disabled]

Updated

5 months ago
Product: Testing → Testing Graveyard
You need to log in before you can comment on or make changes to this bug.