Closed Bug 1383922 Opened 7 years ago Closed 7 years ago

Allow try-test-paths syntax to refer to wpt subtypes

Categories

(Testing :: web-platform-tests, enhancement)

Product:
Testing
Component:
web-platform-tests
Version:
Version 3
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(firefox57 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla57
Tracking Flags:
Tracking Status
firefox57 --- fixed

People

(Reporter: impossibus, Assigned: impossibus)

References

Details

Attachments

(1 file)

Bug 1383922 - Allow try-test-paths syntax to refer to wpt subtypes
59 bytes, text/x-review-board-request
jgraham
: review+
Details 
It's possible to use --try-test-paths syntax to schedule wdspec tests on platforms where they aren't configured to run, for example: try: -b do -p all -u web-platform-tests-1 --try-test-paths web-platform-tests:testing/web-platform/tests/webdriver

This may occur when the wpt-sync service pushes to try to check the tests affected by a change that we want to downstream from the web-platform-tests repository.
Comment on attachment 8889657 [details]
Bug 1383922 - Allow try-test-paths syntax to refer to wpt subtypes

https://reviewboard.mozilla.org/r/160700/#review166186

I'm a little worried about the special-casing here. For example I'm thinking of enabling web-paltform-tests-reftests on QuantumRender builds, and I don't want try to error out because it tries to run testharness tests on that platform. I think we have access to the taskgraph, somehow? Can we use that to make a more generic solution here?
Attachment #8889657 - Flags: review?(james)
In that case, I think the best option is to generate try syntax that refers to the different kinds of wpt jobs. Either teach |mach try| or implement it directly in the wpt-sync service.

So, given [testing/web-platform/tests/2dcontext, testing/web-platform/tests/cookies], generate this:

try: -b do -p linux -u web-platform-tests-1,web-platform-tests-e10s-1,web-platform-tests-reftests-1,web-platform-tests-reftests-e10s-1 --try-test-paths web-platform-tests-reftests:testing/web-platform/tests/2dcontext web-platform-tests:testing/web-platform/tests/cookies 

Instead of this, which lumps everything under "web-platform-tests" -- this is what ./mach try does now:

try: -b do -p linux -u web-platform-tests-1,web-platform-tests-e10s-1 --try-test-paths web-platform-tests:testing/web-platform/tests/2dcontext web-platform-tests:testing/web-platform/tests/cookies web-platform-tests:testing/web-platform/tests/webdriver
Generating the right jobs for each wpt test type seems like a good idea to me.
Summary: Handle wdspec tests requested via try-test-paths with non-linux64 builds → Allow try-test-paths syntax to refer to wpt subtypes
Comment on attachment 8889657 [details]
Bug 1383922 - Allow try-test-paths syntax to refer to wpt subtypes

https://reviewboard.mozilla.org/r/160700/#review168704

I admit I'm not sure where this works out which web-platform-tests are of which type, but based on the try push it apparently works, and I at least understand why it works given that it manages that, so it seems fine to me :)
Attachment #8889657 - Flags: review?(james) → review+
Pushed by mjzffr@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/4e799d2c0555
Allow try-test-paths syntax to refer to wpt subtypes r=jgraham
https://hg.mozilla.org/mozilla-central/rev/4e799d2c0555
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox57: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla57
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: