Open Bug 1342449 Opened 8 years ago Updated 3 years ago

determine if media-test and media-tests[-youtube|-twitch] can be run as e10s

Categories

(Testing :: General, defect, P3)

Product:
Testing
Component:
General
Type:
defect

Tracking

(Not tracked)

People

(Reporter: jmaher, Unassigned)

References

Details

in preparation of turning off non-e10s tests for Firefox 57 (and currently for osx debug), we find that the media-tests* are only run as non-e10s. I would like to see these running as e10s. I really don't know who owns these tests, :mreavy, would you have some background on these tests?
Flags: needinfo?(mreavy)
Hi Joel, Can you be more specific about which tests you're referring to? Can you give me a path for the tests? They are likely Playback tests, but the path should tell me for sure. Thanks.
Flags: needinfo?(mreavy) → needinfo?(jmaher)
sorry for not being so specific, these run in the VP() group on treeherder: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=media-tests b-y, b-m, b-t if that doesn't help, I would need to figure out where the sources for those tests live.
Flags: needinfo?(jmaher)
Thanks, Joel. I think the person you want to talk with is cpearce. Chris -- Here is Joel's question: > in preparation of turning off non-e10s tests for Firefox 57 (and currently > for osx debug), we find that the media-tests* are only run as non-e10s. > > I would like to see these running as e10s. I really don't know who owns > these tests, :mreavy, would you have some background on these tests?
Flags: needinfo?(cpearce)
Chris -- I think the tests in question are dom/media/test/external/external_media_tests . Also, why does this log show errors and crashes, butyet still looks green in treeherder: https://treeherder.mozilla.org/logviewer.html#?job_id=80000607&repo=mozilla-central ?
that is really confusing, I see that the harness has return(10), I wonder if the script used to run this test is not handling errors properly? this looks like the runner script: https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/scripts/firefox_media_tests_taskcluster.py?q=firefox_media_tests_taskcluster.py&redirect_type=direct SingingTree, can you help determine why we are not getting errors as mentioned in comment 4 ?
Flags: needinfo?(bvandyk)
Roger. Am investigating now.
Flags: needinfo?(cpearce)
Flags: needinfo?(bvandyk)
I think the issue here is based on my misapprehension about how return codes are handled by the base class runner. My expectation was that the base class would set return codes, however this does not appear to be the case. I'm checking in to see if this behaviour can be changed in the base, which is my preferred solution right now. However, failing that it can be fixed in the TC runner. Should be a relatively quick fix once a course has been decided upon, which I wouldn't think would take more than a couple of days, if that.
Code to fix bug 1343129 is landed, and should fix the return code problem. Thank you for bringing it to my attention, and please, let me know if there are any other issues or changes needed that I can assist with.
:cpearce, still a pending question here regarding if these will be run in e10s mode?
Flags: needinfo?(cpearce)
I'm think I'm also better suited to deal with e10s issues. I've tested them locally and they seem to work correctly under e10s. Do you need config changes to get these running under e10s on TaskCluster?
Flags: needinfo?(cpearce) → needinfo?(jmaher)
glad these work under e10s. The plan is in June (Firefox v.57) to turn off all non-e10s testing (unless we are testing things that use a shell vs a browser). to schedule these as e10s, just change: https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/test/tests.yml#147 from: e10s: false to: e10s: true <- only e10s or: e10s: both <- run both e10s and non-e10s I would leave this up to you or your team to determine where you are getting value from the tests. I always advocate for e10s only, but if there is a need for the next few months to have both, that would seem reasonable.
Flags: needinfo?(jmaher)
Thanks, Joel. I'll grab this bug and see that it's addressed in the near future.
Assignee: nobody → bvandyk
Priority: -- → P3

Sorry, there was a problem with the detection of inactive users. I'm reverting the change.

Assignee: nobody → brycebugemail

Unassigning bugs assigned to Bryce because he no longer works at Mozilla.

Assignee: brycebugemail → nobody
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.