EME tests time out on Netflix

RESOLVED FIXED

Status

Testing
external-media-tests
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: maja_zf, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

2 years ago
Since early December, during Netflix EME tests (http://pf-jenkins.qa.mtv2.mozilla.com:8080/view/mario-mse/job/mn-eme-netflix-nightly-win7_32_64/530/console) the browser is redirected to a "Netflix System Requirments" page. 

I think this may be a problem with the CDM not having enough time to install/update itself, which is why I'm filing this under firefox-media-tests instead of audio/video playback. Perhaps the profile used for the tests needs to be updated or the timeout should be longer?
(Reporter)

Updated

2 years ago
Flags: needinfo?(spolk)

Comment 1

2 years ago
This does not seem to be the problem anymore. We are having timeouts, but they appear to be media playback problems:

http://pf-jenkins.qa.mtv2.mozilla.com:8080/job/mn-eme-netflix-nightly-win_32_64/76/artifact/logs/media_tests.html
Flags: needinfo?(spolk)
(Reporter)

Comment 2

2 years ago
(In reply to Syd Polk :sydpolk from comment #1)
> This does not seem to be the problem anymore. We are having timeouts, but
> they appear to be media playback problems:
> 
> http://pf-jenkins.qa.mtv2.mozilla.com:8080/job/mn-eme-netflix-nightly-
> win_32_64/76/artifact/logs/media_tests.html

This is a Windows 8 job, and I agree that looks like media playback problems, but my original report is about Windows 7, where we still see the same issue today (redirect to "Netflix System Requirements"). Please see: http://pf-jenkins.qa.mtv2.mozilla.com:8080/view/mario-mse/job/mn-eme-netflix-nightly-win7_32_64/531/artifact/logs/media_tests.html

If you look at the corresponding gecko.log, the CDM download seems to start in the middle of test_video_playback.py:test_playback_starts.
Flags: needinfo?(spolk)

Comment 3

2 years ago
OK, this was a little painful. Evidently, the job never had "-c config/treeherder_submission.py" added to it, so the run_media_tests.py script started rasing. Don't know why this did not raise before...

I have confirmed that the download of the gmp piece had not happened for the first video. And since marionette copies the profile every time, it never got updated... so I updated it by hand. And then the test passed.

Verified that the other netflix jobs have the correct command-line parameters.
Also updated profile on t-win81-64-02 while I was at it. Did not seem to help timeouts. Maybe real bug here, but I think I should see if it repros on real machines first.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Flags: needinfo?(spolk)
Resolution: --- → FIXED
(In reply to Syd Polk :sydpolk from comment #3)
> I have confirmed that the download of the gmp piece had not happened for the
> first video. And since marionette copies the profile every time, it never
> got updated... so I updated it by hand. And then the test passed.

We really want at least one test where downloading the GMP is part of the test; as we want to ensure that the download/server doesn't regress. Can the profile be reset so that the download has to happen for the test to pass?
You need to log in before you can comment on or make changes to this bug.