1338531 - [mozlog] suite_start message is never sent when using pytest-mozlog and pytest-xdist

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Running pytest tests with pytest-mozlog and pytest-xdist does not send the suite_start message, therefore all test_start messages fail with the error: "Got test_start message before suite_start for test XXXX"

Steps to replicate:
1. Install mozlog from mozilla-central and pytest from PyPI
2. Run a suite with "--log-mach=- -n=1"

Expected:
1. "SUITE_START" and "TEST_START" messages appear in the console.

Actual:
1. "ERROR Got test_start message before suite_start for test XXX".
2. "ERROR Got suite_end message before suite_start".
3. SUITE_START and TEST_START messages do not appear in the console.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Bug 1338531 - [mozlog] Collect tests when using pytest-xdist and send suite_start when the first test starts.

When using pytest-xdist we need to gather the tests for each slave. Lack of a better hook means I'm then using the first test start to log the suite_start message including the collected tests.

Review commit: https://reviewboard.mozilla.org/r/112124/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/112124/

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8836802 [details]
Bug 1338531 - [mozlog] Collect tests when using pytest-xdist and send suite_start when the first test starts.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/112124/diff/1-2/

Comment 3

[Andrew Halberstadt [:ahal]]

Comment on attachment 8836802 [details]
Bug 1338531 - [mozlog] Collect tests when using pytest-xdist and send suite_start when the first test starts.

https://reviewboard.mozilla.org/r/112124/#review113732

::: testing/mozbase/mozlog/mozlog/pytest_mozlog/plugin.py:66
(Diff revision 2)
> +    def pytest_xdist_node_collection_finished(self, node, ids):
> +        '''Called after each pytest-xdist node collection is completed'''
> +        self._log_suite_start(ids)

I'm not familiar with pytest-xdist.. Will this pass in a different set of tests each time?

If so, then only the first batch will be reported in the suite_start message. In that case it would probably be better not to log any tests at all.

If not, then please close this issue.

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8836802 [details]
Bug 1338531 - [mozlog] Collect tests when using pytest-xdist and send suite_start when the first test starts.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/112124/diff/2-3/

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8836802 [details]
Bug 1338531 - [mozlog] Collect tests when using pytest-xdist and send suite_start when the first test starts.

https://reviewboard.mozilla.org/r/112124/#review113746

::: testing/mozbase/mozlog/mozlog/pytest_mozlog/plugin.py:66
(Diff revision 2)
> +    def pytest_xdist_node_collection_finished(self, node, ids):
> +        '''Called after each pytest-xdist node collection is completed'''
> +        self._log_suite_start(ids)

I checked with the owner of pytest-xdist beforehand, and every node has an identical set of tests. If they ever collected a different set of tests, pytest would raise an exception anway.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9a6977ca4eae
[mozlog] Collect tests when using pytest-xdist and send suite_start when the first test starts. r=ahal

Comment 7

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/9a6977ca4eae