1345810 - Intermittent ERROR test_basic_stream_playback.py TestBasicStreamPlayback.test_video_playback_partial

Status: RESOLVED FIXED

(Testing Graveyard :: external-media-tests, defect)

Description

[Treeherder Bug Filer]

Filed by: cbook [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=82556837&repo=mozilla-aurora

https://queue.taskcluster.net/v1/task/do4YC-frQcyfvdj6t3-K9Q/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Not sure what's going on here. The harness is failing to detect the video element on the page, but the screenshot indicates that the page is loaded and the video is present. There is a potential lag between the screenshot being taken and the failure, but I'm dubious as to if that's what's going on. If this happens again, I'll instrument the tests to try and better identify what's going on here.

Comment 2

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Hrm, had a look and found some more of these. Going to add further logging to the tests to try and identify what's going on.

Comment 3

[Intermittent Failures Robot]

11 failures in 790 pushes (0.014 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-aurora: 8
* try: 1
* mozilla-central: 1
* graphics: 1

Platform breakdown:
* linux64: 5
* linux32: 4
* windowsxp: 1
* linux32-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-03-06&endday=2017-03-12&tree=all

Comment 4

[Intermittent Failures Robot]

40 failures in 777 pushes (0.051 failures/push) were associated with this bug in the last 7 days. 

This is the #41 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* try: 11
* autoland: 11
* mozilla-inbound: 9
* mozilla-central: 7
* oak: 1
* graphics: 1

Platform breakdown:
* linux64: 17
* linux32: 15
* windows7-32-vm: 4
* windows10-64-vm: 3
* linux32-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-03-13&endday=2017-03-19&tree=all

Comment 5

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Linux media tests on TaskCluster are in the process of being disabled to address this failure rate.

Comment 6

[Intermittent Failures Robot]

24 failures in 174 pushes (0.138 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* mozilla-central: 7
* autoland: 7
* mozilla-inbound: 5
* try: 3
* mozilla-aurora: 2

Platform breakdown:
* linux64: 16
* linux32: 4
* windows7-32-vm: 2
* windows10-64-vm: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-03-21&endday=2017-03-21&tree=all

Comment 7

[Intermittent Failures Robot]

20 failures in 186 pushes (0.108 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* mozilla-inbound: 12
* mozilla-aurora: 3
* mozilla-central: 2
* try: 1
* oak: 1
* autoland: 1

Platform breakdown:
* linux64: 10
* linux32: 7
* windows10-64-vm: 2
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-03-22&endday=2017-03-22&tree=all

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

the failure rate has gone way down after disabling the tests on linux, thanks for doing that.

Comment 9

[Intermittent Failures Robot]

72 failures in 898 pushes (0.08 failures/push) were associated with this bug in the last 7 days. 

This is the #12 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-inbound: 21
* mozilla-central: 18
* mozilla-aurora: 13
* autoland: 12
* try: 4
* oak: 3
* graphics: 1

Platform breakdown:
* linux64: 40
* linux32: 18
* windows7-32-vm: 7
* windows10-64-vm: 7

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-03-20&endday=2017-03-26&tree=all

Comment 10

[Intermittent Failures Robot]

19 failures in 891 pushes (0.021 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-central: 18
* try: 1

Platform breakdown:
* windows10-64-vm: 11
* windows7-32-vm: 8

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-05-22&endday=2017-05-28&tree=all

Comment 11

[Intermittent Failures Robot]

17 failures in 175 pushes (0.097 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* mozilla-central: 17

Platform breakdown:
* windows7-32-vm: 9
* windows10-64-vm: 8

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-06-22&endday=2017-06-22&tree=all

Comment 12

[Joel Maher ( :jmaher ) (UTC -8)]

this started failing June 21st around midnight on windows opt, we should look into this with 20+ failures/day unless this is a weird spike for a day.

:singingtree, do you know if much has changed with these tests recently?  Can you help look into this?

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

these are only failing on m-c, this makes it difficult to figure out the root cause of failure, but I assume they are only running on mozilla-central.

Comment 14

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

I'm uncertain as to why we'd see a specific uptick in failures at that time. These tests are about to be retired as part of Bug 1362252 and the more immediate Bug 1372059. I will examine the increased failure rate in the coming week to see if I can identify issues causing (legitimate) test failures. Given the above, we don't need to worry about further failures, but I'll see if the tests can deliver value in their swan song.

Comment 15

[Intermittent Failures Robot]

28 failures in 892 pushes (0.031 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-central: 26
* try: 2

Platform breakdown:
* windows7-32-vm: 14
* windows10-64-vm: 13
* linux64-ccov: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1345810&startday=2017-06-19&endday=2017-06-25&tree=all

Comment 16

[Joel Maher ( :jmaher ) (UTC -8)]

:singingtree, when will these be disabled?  we still see many failures

Comment 17

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

The taskcluster tests were removed from tree ~23 June. Looking at brasstacks I see a falloff at that date. Can you confirm that there is still issue?

Comment 18

[Joel Maher ( :jmaher ) (UTC -8)]

I apologize for asking for info on this and not paying attention to the dates- this has been silent and being removed is a good explanation for it.