1296397 - Perform a firefox checkout when running wpt tests

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Gregory Szorc [:gps]]

Before we actually run WPT tests from a source checkout (bug 1286900), I want automation to start performing a checkout as part of the task.

The main reason I want to do this is to validate that automation is capable of performing the checkout without issue. I'm particularly interested in what the new load to hg.mozilla.org will do to those servers. I'm pretty sure we have the capacity. But TC's patterns of spawning hundreds of workers at the same time could make things... interesting.

Comment 1

[Gregory Szorc [:gps]]

Attachment: Bug 1296397 - Add source checkout to web platform test tasks;

This commit does a few things. First, it introduces a property on the
"test_description" schema that, if defined, will cause run-task to
perform a gecko checkout to the specified path. The presence of the
property also configures the needed scopes and caches.

Second, we introduce the property on web platform test tasks so a
Gecko checkout is present. We also add volumes for the Mercurial
paths to the Docker images. We strictly only need this for
desktop1604-test since WPT tests don't run on desktop-test. However,
desktop-test and desktop1604-test are nearly mirror images of each
other and I feel it is best to keep them in sync.

Review commit: https://reviewboard.mozilla.org/r/72694/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/72694/

Comment 2

[Gregory Szorc [:gps]]

Comment on attachment 8782588 [details]
Bug 1296397 - Add source checkout to web platform test tasks;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72694/diff/1-2/

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8782588 [details]
Bug 1296397 - Add source checkout to web platform test tasks;

https://reviewboard.mozilla.org/r/72694/#review70400

I'm OK with this landing as-is, even if I think it's a little "too much rope", as it is a good way to start judging total load on hg.m.o.

::: taskcluster/taskgraph/transforms/tests/test_description.py:118
(Diff revision 2)
>          int,
>          {'by-test-platform': {basestring: int}},
>      ),
>  
> +    # Where to put a VCS checkout of the Firefox repo.
> +    Optional('gecko-checkout'): basestring,

I'd prefer to have this be a boolean -- I don't think there will ever be a good reason to do checkouts in arbitrary directories, so allowing specification of a directory just means writing the same directory name several times, for a flexibility we'd rather not ever be used.

Comment 4

[Gregory Szorc [:gps]]

Comment on attachment 8782588 [details]
Bug 1296397 - Add source checkout to web platform test tasks;

https://reviewboard.mozilla.org/r/72694/#review70400

> I'd prefer to have this be a boolean -- I don't think there will ever be a good reason to do checkouts in arbitrary directories, so allowing specification of a directory just means writing the same directory name several times, for a flexibility we'd rather not ever be used.

I like when someone calls YAGNI on me. I'll make this simpler.

Comment 5

[Gregory Szorc [:gps]]

Comment on attachment 8782588 [details]
Bug 1296397 - Add source checkout to web platform test tasks;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72694/diff/2-3/

Comment 6

[Gregory Szorc [:gps]]

Comment on attachment 8782588 [details]
Bug 1296397 - Add source checkout to web platform test tasks;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72694/diff/3-4/

Comment 7

[Gregory Szorc [:gps]]

Comment on attachment 8782588 [details]
Bug 1296397 - Add source checkout to web platform test tasks;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72694/diff/4-5/

Comment 8

[Armen [:armenzg]]

Is this waiting for the build refactoring to make it into central?

Comment 9

[Gregory Szorc [:gps]]

Yes, this was stalled by Dustin's work and changes to TaskCluster's log parsing. Both landed today. I'm in the process of rebasing my commits.

Comment 10

[Gregory Szorc [:gps]]

Attachment: Bug 1296397 - Move VCS task setup functionality to standalone function;

Tasks that aren't using run-task may want to configure a task for
optimal VCS setup. Also, having all the code in the same place makes
it easier to keep track of.

Review commit: https://reviewboard.mozilla.org/r/78548/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/78548/

Comment 11

[Gregory Szorc [:gps]]

Comment on attachment 8782588 [details]
Bug 1296397 - Add source checkout to web platform test tasks;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72694/diff/5-6/

Comment 12

[Gregory Szorc [:gps]]

Comment on attachment 8790950 [details]
Bug 1296397 - Move VCS task setup functionality to standalone function;

Tasks that aren't using run-task may want to configure a task for
optimal VCS setup. Also, having all the code in the same place makes
it easier to keep track of.

Review commit: https://reviewboard.mozilla.org/r/78548/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/78548/

Comment 13

[Gregory Szorc [:gps]]

Comment on attachment 8782588 [details]
Bug 1296397 - Add source checkout to web platform test tasks;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72694/diff/5-6/

Comment 14

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8790950 [details]
Bug 1296397 - Move VCS task setup functionality to standalone function;

https://reviewboard.mozilla.org/r/78548/#review77278

Comment 15

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8782588 [details]
Bug 1296397 - Add source checkout to web platform test tasks;

https://reviewboard.mozilla.org/r/72694/#review77296

::: taskcluster/taskgraph/transforms/tests/test_description.py:123
(Diff revision 6)
>          int,
>          {'by-test-platform': {basestring: int}},
>      ),
>  
> +    # Whether to perform a gecko checkout.
> +    Optional('checkout'): bool,

I'd prefer `Required('checkout', default=False)` here, just to avoid the implied equivalence of property omission and False.

Comment 16

[Gregory Szorc [:gps]]

Comment on attachment 8790950 [details]
Bug 1296397 - Move VCS task setup functionality to standalone function;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/78548/diff/1-2/

Comment 17

[Gregory Szorc [:gps]]

Comment on attachment 8782588 [details]
Bug 1296397 - Add source checkout to web platform test tasks;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/72694/diff/6-7/

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ab5198e198b5
Move VCS task setup functionality to standalone function; r=dustin
https://hg.mozilla.org/integration/autoland/rev/4b447ca3c1ae
Add source checkout to web platform test tasks; r=dustin

Comment 19

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/ab5198e198b5
https://hg.mozilla.org/mozilla-central/rev/4b447ca3c1ae