1472171 - Upload raw structured logs as artifacts from Python unit tests

Status: ASSIGNED

(Testing :: Python Test, enhancement)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Tests run via |mach python-test| do not currently produce and upload structured logs, and therefore results are not available in ActiveData. As we're already using mozlog's pytest plugin, we should be able to generate a raw structured log and upload the resulting file as an artifact.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Bug 1472171 - Upload raw structured logs as artifacts from Python unit tests;

Review commit: https://reviewboard.mozilla.org/r/253934/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/253934/

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

ahal: This is basically your patch applied with no changes, but it appears to be working. I've pushed to try, and the results look good: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7a39c85f0e9180456d817c4be26f98a088222448&selectedJob=185578055

The py3 jobs are uploading empty log files, which is due to mozlog not yet supporting py3, but the result of the jobs themselves don't appear to be affected.

Comment 3

[Andrew Halberstadt [:ahal]]

Comment on attachment 8988727 [details]
Bug 1472171 - Upload raw structured logs as artifacts from Python unit tests;

https://reviewboard.mozilla.org/r/253934/#review260670

I think having a single suite for the entire run should be a blocker to getting this landed. As it stands, it will cause a lot of extra variability in ActiveData (would probably double or triple the number of suite records) and might make a bunch of queries/dashboards less useful as a result. Tbh, I'm not even sure that ActiveData would pick this up automatically or not.

I also probably shouldn't review my own code :).