1388016 - Add a task for running the mozbase python tests against Python 3

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Once bug 1388013 is resolved we should create a new task for running the mozbase Python tests against Python 3. This will allow us to start adding support for Python 3 into our mozbase packages.

Ideally we should avoid duplicating the task definition by generating tasks for Python 2 and 3 at the same time.

Comment 1

[(not currently active) Ted Mielczarek]

I'm currently digging through the taskcluster taskgraph code for other reasons, so I might as well put some links here. The standalone mozbase Python test task is defined here:
https://dxr.mozilla.org/mozilla-central/rev/52285ea5e54c73d3ed824544cef2ee3f195f05e6/taskcluster/ci/source-test/python-tests.yml#82

Some generic info for the 'source-test' task kind is defined in the kind.yml alongside it:
https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/source-test/kind.yml

Specifically, the kind.yml file instructs the taskgraph to run those task definitions through the source_test transforms:
https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/transforms/source_test.py

If you wanted to make it so we generated Python 2 + Python 3 tasks from the same task definition, you could do something like the `expand_platforms` transform, which makes a task run on a specified set of platforms:
https://dxr.mozilla.org/mozilla-central/rev/52285ea5e54c73d3ed824544cef2ee3f195f05e6/taskcluster/taskgraph/transforms/source_test.py#78

You could add a field to the `source_test_description_schema` earlier in the file like `python-versions: [int]`. You'd need to add an additional transform function that checked `job['python-versions']` and yielded a task per-version listed there, and use that to add the `--python3` argument to the test command.

Then in the python-tests.yml file you could make individual task types request Python 3 tasks like:
```
python-versions:
    - 2
    - 3
```

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Bug 1388016 - Add support for specifying the Python version to use in TaskCluster;

Review commit: https://reviewboard.mozilla.org/r/251986/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/251986/

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Bug 1388016 - Run mozterm tests against Python 2 and 3;

Review commit: https://reviewboard.mozilla.org/r/251988/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/251988/

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Bug 1388016 - Run mozbase tests against Python 2 and 3;

Review commit: https://reviewboard.mozilla.org/r/251990/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/251990/

Comment 5

[Andrew Halberstadt [:ahal]]

Comment on attachment 8986688 [details]
Bug 1388016 - Add support for specifying the Python version to use in TaskCluster;

https://reviewboard.mozilla.org/r/251986/#review258472

Thanks, lgtm!

::: taskcluster/ci/config.yml:8
(Diff revision 1)
>  treeherder:
>      group-names:
>          'cram': 'Cram tests'
>          'js-bench': 'JavaScript shell benchmarks'
>          'mocha': 'Mocha unit tests'
>          'py': 'Python unit tests'

Please remove this group

::: taskcluster/ci/source-test/python.yml:27
(Diff revision 1)
>  taskgraph-tests:
>      description: taskcluster/taskgraph unit tests
> +    python-version:
> +        - 2
>      treeherder:
>          symbol: py(tg)

Might as well remove the `py()` from all these symbols. I'm pretty sure it'll just work with your existing code.

::: taskcluster/taskgraph/transforms/job/python_test.py:33
(Diff revision 1)
> +@run_job_using(
> +    'native-engine',
> +    'python-test',
> +    schema=python_test_schema,
> +    defaults={'python-version': 2, 'subsuite': 'default'})

You can probably remove this, I don't think there are (or will be) any `native-engine` python test tasks. That is only used for things that need to run on physical hardware (like Talos)

::: taskcluster/taskgraph/transforms/source_test.py:109
(Diff revision 1)
> [e-mail]
> +def split_python(config, jobs):

I guess this is here because it wasn't possible to split a task by the time it got to the code in `python_test.py`?

Comment 6

[Andrew Halberstadt [:ahal]]

Comment on attachment 8986689 [details]
Bug 1388016 - Run mozterm tests against Python 2 and 3;

https://reviewboard.mozilla.org/r/251988/#review258522

::: taskcluster/ci/source-test/python.yml:139
(Diff revision 1)
>  
>  mozterm:
>      description: python/mozterm unit tests
>      python-version:
>          - 2
> +        - 3

Won't make this an issue, but fyi you can use:

    python-version: [2, 3]

if you find that more readable.

Comment 7

[Andrew Halberstadt [:ahal]]

Comment on attachment 8986690 [details]
Bug 1388016 - Run mozbase tests against Python 2 and 3;

https://reviewboard.mozilla.org/r/251990/#review258524

Awesome, thanks! I think this will end up becoming a nice source of [good first bug]! (Assuming there are parts that aren't time sensitive)

Comment 8

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8986688 [details]
Bug 1388016 - Add support for specifying the Python version to use in TaskCluster;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/251986/diff/1-2/

Comment 9

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8986689 [details]
Bug 1388016 - Run mozterm tests against Python 2 and 3;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/251988/diff/1-2/

Comment 10

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8986690 [details]
Bug 1388016 - Run mozbase tests against Python 2 and 3;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/251990/diff/1-2/

Comment 11

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8986688 [details]
Bug 1388016 - Add support for specifying the Python version to use in TaskCluster;

https://reviewboard.mozilla.org/r/251986/#review258472

> I guess this is here because it wasn't possible to split a task by the time it got to the code in `python_test.py`?

As far as I could tell, yes. I've just tried implementing this in `python_test.py` and I didn't get the additional tasks generated.

Comment 12

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8986688 [details]
Bug 1388016 - Add support for specifying the Python version to use in TaskCluster;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/251986/diff/2-3/

Comment 13

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8986689 [details]
Bug 1388016 - Run mozterm tests against Python 2 and 3;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/251988/diff/2-3/

Comment 14

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8986690 [details]
Bug 1388016 - Run mozbase tests against Python 2 and 3;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/251990/diff/2-3/

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/81962d389280
Add support for specifying the Python version to use in TaskCluster; r=ahal
https://hg.mozilla.org/integration/autoland/rev/09747ac119be
Run mozterm tests against Python 2 and 3; r=ahal
https://hg.mozilla.org/integration/autoland/rev/06fcd1d66bf3
Run mozbase tests against Python 2 and 3; r=ahal

Comment 16

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/81962d389280
https://hg.mozilla.org/mozilla-central/rev/09747ac119be
https://hg.mozilla.org/mozilla-central/rev/06fcd1d66bf3