1011434 - [mozlog.structured] Add a framework for structured logging command line scripts and a script for printing unstable tests

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[James Graham [:jgraham]]

After doing a run with multiple repetitions, it is useful to see which tests gave different results across runs. This adds a script to do that and a framework for adding more command line scripts to mozlog.structured.

Comment 1

[James Graham [:jgraham]]

Attachment: Add a framework for structured logging command line scripts and a script for printing unstable tests.

Comment 2

[Andrew Halberstadt [:ahal]]

Comment on attachment 8423774 [details] [diff] [review]
Add a framework for structured logging command line scripts and a script for printing unstable tests.

Review of attachment 8423774 [details] [diff] [review]:
-----------------------------------------------------------------

I don't really like the pattern of storing arbitrary console scripts in mozlog since mozlog itself is not a CLI application. It seems like the CLI application that uses mozlog should be responsible for creating whatever scripts are needed. If this script is super common and would be useful in the general case, maybe it could live in moztest? Or at least create a new submodule for them under mozlog.structured so they are hidden away?

Comment 3

[Andrew Halberstadt [:ahal]]

James, I think you may have missed my comment in irc, but I would be ok with e.g:
"from mozlog.structured.scripts import unstable" (possibly s/scripts/tools ?)

I think my biggest complaint about the current patch is just that unstable.py is a confusing name, and it's not clear that it's a utility script.

Comment 4

[James Graham [:jgraham]]

Attachment: Add a framework for structured logging command line scripts and a script for printing unstable tests.

Comment 5

[Andrew Halberstadt [:ahal]]

Comment on attachment 8426276 [details] [diff] [review]
Add a framework for structured logging command line scripts and a script for printing unstable tests.

Review of attachment 8426276 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks!

Comment 6

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/911a468a29b0