1232340 - EME tests time out on Netflix

Status: RESOLVED FIXED

(Testing Graveyard :: external-media-tests, defect)

Description

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

Since early December, during Netflix EME tests (http://pf-jenkins.qa.mtv2.mozilla.com:8080/view/mario-mse/job/mn-eme-netflix-nightly-win7_32_64/530/console) the browser is redirected to a "Netflix System Requirments" page. 

I think this may be a problem with the CDM not having enough time to install/update itself, which is why I'm filing this under firefox-media-tests instead of audio/video playback. Perhaps the profile used for the tests needs to be updated or the timeout should be longer?

Comment 1

[Syd Polk :sydpolk]

This does not seem to be the problem anymore. We are having timeouts, but they appear to be media playback problems:

http://pf-jenkins.qa.mtv2.mozilla.com:8080/job/mn-eme-netflix-nightly-win_32_64/76/artifact/logs/media_tests.html

Comment 2

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

(In reply to Syd Polk :sydpolk from comment #1)
> This does not seem to be the problem anymore. We are having timeouts, but
> they appear to be media playback problems:
> 
> http://pf-jenkins.qa.mtv2.mozilla.com:8080/job/mn-eme-netflix-nightly-
> win_32_64/76/artifact/logs/media_tests.html

This is a Windows 8 job, and I agree that looks like media playback problems, but my original report is about Windows 7, where we still see the same issue today (redirect to "Netflix System Requirements"). Please see: http://pf-jenkins.qa.mtv2.mozilla.com:8080/view/mario-mse/job/mn-eme-netflix-nightly-win7_32_64/531/artifact/logs/media_tests.html

If you look at the corresponding gecko.log, the CDM download seems to start in the middle of test_video_playback.py:test_playback_starts.

Comment 3

[Syd Polk :sydpolk]

OK, this was a little painful. Evidently, the job never had "-c config/treeherder_submission.py" added to it, so the run_media_tests.py script started rasing. Don't know why this did not raise before...

I have confirmed that the download of the gmp piece had not happened for the first video. And since marionette copies the profile every time, it never got updated... so I updated it by hand. And then the test passed.

Verified that the other netflix jobs have the correct command-line parameters.
Also updated profile on t-win81-64-02 while I was at it. Did not seem to help timeouts. Maybe real bug here, but I think I should see if it repros on real machines first.

Comment 4

[Chris Pearce [:cpearce (Not reading bugmail)]]

(In reply to Syd Polk :sydpolk from comment #3)
> I have confirmed that the download of the gmp piece had not happened for the
> first video. And since marionette copies the profile every time, it never
> got updated... so I updated it by hand. And then the test passed.

We really want at least one test where downloading the GMP is part of the test; as we want to ensure that the download/server doesn't regress. Can the profile be reset so that the download has to happen for the test to pass?