Add web-platform-tests to buildbot config

RESOLVED FIXED

Status

defect
RESOLVED FIXED
6 years ago
Last year

People

(Reporter: jgraham, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 1 obsolete attachment)

In bug 945222 I am working on a test runner for the web-platform-tests. An early version of this along with a mozharness script has now landed on ash, but I presume to try it out I need to add it to the buildbot configs (and in any case I need to in order to land the change).

AIUI I need to look in http://hg.mozilla.org/build/buildbot-configs/mozilla-tests/config.py. It seems straightforward to add it to the list of testsuites and the list of jobs for each slave type (at the moment only for the linux ones), but I'm not sure if anything more is needed e.g. something to limit it to ash.
Yes, that's about right.  You should enable the tests on cedar, rather than ash, if you're using a new mozharness script; cedar is used for greening up test suites; ash for testing out mozharness changes that could impact production if they're pushed live, but that wouldn't be the case if you're creating a brand new mozharness script.

You can add them to a specific tree using something like the block here:

http://hg.mozilla.org/build/buildbot-configs/file/f9df7bca20c3/mozilla-tests/config.py#l1613
So if I land this on cedar, I have to land the untested (on infrastructure) mozharness script in production?
Yes, but a new mozharness script that isn't being used by anything else won't cause any problems except on cedar, if something goes wrong.

If you're modifying mozharness core code, though, we should test it on ash first.
Depends on: 961138
Attachment #8363117 - Flags: review?(aki)
Comment on attachment 8363117 [details] [diff] [review]
Add web-platform-tests to cedar

I would have preferred an explicit set like this:
http://hg.mozilla.org/build/buildbot-configs/file/7c742fd377c3/mozilla-tests/config.py#l1558

but it looks like cedar is special cased all over already.
Attachment #8363117 - Flags: review?(aki) → review+
(This will add these tests to macosx64, linux, linux64, and linux64-asan.)
Could be too many platform variants for initial testing I guess. jgriffin?
I think since we're just running them on cedar, the platform list is fine.

But, I believe we'll also need code similar to http://hg.mozilla.org/build/buildbot-configs/file/7c742fd377c3/mozilla-tests/config.py#l744 where we specify which config files to use.

Additionally, we don't want to schedule these tests on both Fedora and Ubuntu for linux, so we should add some code to specifically exclude Fedora slave_platform variants.
Oh, yeah, that's right, sorry.
Attachment #8363117 - Attachment is obsolete: true
Attachment #8363771 - Flags: review?(aki)
Attachment #8363771 - Flags: review?(aki) → review+
bbot-cfg patch is in production! :)
These are running in cedar; I think we can open new bugs for scheduling them elsewhere when needed.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.