1302204 - update marionette-harness kind to use the perks of bug 1286075

Status: RESOLVED INVALID

(Firefox Build System :: Task Configuration, task)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

https://reviewboard.mozilla.org/r/77452/diff/2 breaks the marionette-harness kind out of the legacy kind, but doesn't do the greatest job of it.

There are some comments there illustrating things to change.  The changes probably span the task-graph generation, the shell scripts that invoke mozharness, and mozharness itself.  Ideally with a little bit of work this can be written as a job description with run.using = 'mozharness'.  At that point it can probably be folded back into the "source-checks" kind, unless there's good reason to keep the separate "marionette-harness" kind.

Comment 1

[Gregory Szorc [:gps]]

Attachment: Bug 1302204 - Convert marionette-harness kind and task to a source-check

INCOMPLETE:

* mozlog py.test plugin isn't loading for unknown reasons
* we need to print the TBPL log output or treeherder parsing will get
  confused
* exit code needs to be sane (see mozharness script)
* artifact paths need fixed

(These should all likely be addressed by converting the mozharness
script to a mach command and having TC run that.)

Per the TODOs in the code, we convert this kind and task to a
source-check kind. We add it to the python-tests group, since that
is what it's doing.

mozharness is overkill for this task: we basically create a virtualenv
and run py.test. So we convert the task to not use mozharness.

Review commit: https://reviewboard.mozilla.org/r/80926/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/80926/

Comment 2

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

Comment on attachment 8794471 [details]
Bug 1302204 - Convert marionette-harness kind and task to a source-check

https://reviewboard.mozilla.org/r/80926/#review80382

::: taskcluster/ci/source-check/python-tests.yml:48
(Diff revision 1)
> +        using: run-task
> +        # TODO convert to mach command
> +        command: >
> +            cd /home/worker/checkouts/gecko &&
> +            python2.7 python/virtualenv/virtualenv.py venv &&
> +            (cd testing/config && ../../venv/bin/pip install --no-deps -r marionette_harness_test_requirements.txt) &&

Note: for pytest-mozlog plugin to get picked up by pytest, need to run mozlog's setup.py. See Bug 1301177.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

It would be great to run these tests in the desktop-test docker image, too, rather than desktop-build.  The latter usually doesn't run on test workers, so it's causing cache misses and also some issues with cache permissions.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

removing caches from current kinds.yml in bug 1311442

Comment 5

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

This doesn't apply anymore since Bug 1335873.

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

yay, thanks!!