1443557 - Default to mach formatter when terminal / tty is present

Status: RESOLVED FIXED

(Testing :: Mozbase, enhancement)

Description

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

The mach formatter is easier for humans with its colored output and other nice things.

For some reason, it's not enabled by default.  It seems like most developers would benefit from having it on, but may not be aware of it.

Can we enable it by default, at least when a terminal / tty is present?

Comment 1

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

:ahal, is there some historical / compatibility reason why we default to the raw formatter?

Comment 2

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Hmm, I guess bug 1421799 improved the mach formatting, and the summary of that bug mentioned making it the default, but that didn't seem to happen at the time.  Anyway, seems like :ahal should know more.

Comment 3

[Matthew N. [:MattN]]

This is related to bug 1132044 and unless something has changed, bug 1432683 probably should be fixed first.

Comment 5

[Andrew Halberstadt [:ahal]]

Sorry, bug 1421799 was supposed to make 'mach' the default, but the 'tbpl' format is still the default:
https://searchfox.org/mozilla-central/source/testing/mach_commands.py#67

I must have either forgot to make this change, or blown over it while resolving conflicts. I never noticed because I had:

[test]
format=mach

in my ~/.mozbuild/machrc. So I think there are two issues here:
1) mach format is non-default (I'll use this bug to fix)
2) mach format breaks test-verify (bug 1432683)

I think bug 1132044 can be duped to this one.

Comment 6

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1443557 - Start using the 'mach' formatter by default in |mach test| and |mach mochitest|,

This was originally supposed to be fixed in bug 1421799, but I either forgot to
switch the flag, or accidentally clobbered it while resolving conflicts.

Review commit: https://reviewboard.mozilla.org/r/227156/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/227156/

Comment 7

[Geoff Brown [:gbrown]]

Comment on attachment 8958217 [details]
Bug 1443557 - Start using the 'mach' formatter by default in |mach test| and |mach mochitest|,

https://reviewboard.mozilla.org/r/227156/#review232992

::: testing/mach_commands.py:67
(Diff revision 1)
>          format_desc = "The default format to use when running tests with `mach test`."
>          format_choices = log_formatters.keys()
>          level_desc = "The default log level to use when running tests with `mach test`."
>          level_choices = [l.lower() for l in log_levels]
>          return [
> -            ('test.format', 'string', format_desc, 'tbpl', {'choices': format_choices}),
> +            ('test.format', 'string', format_desc, 'mach', {'choices': format_choices}),

This also changes the format for 'mach test <path-to-reftest>'. Is that OK? Why do we use tbpl by default for 'mach reftest' still, but now mach for 'mach test <reftest>'?

Comment 8

[Andrew Halberstadt [:ahal]]

Comment on attachment 8958217 [details]
Bug 1443557 - Start using the 'mach' formatter by default in |mach test| and |mach mochitest|,

https://reviewboard.mozilla.org/r/227156/#review232992

> This also changes the format for 'mach test <path-to-reftest>'. Is that OK? Why do we use tbpl by default for 'mach reftest' still, but now mach for 'mach test <reftest>'?

I know it's a little weird, but it is intentional.

We still use the `tbpl` format when running |mach reftest| because the `mach` format isn't compatible with the reftest-analyzer. However, when running |mach test|, the reftest output is dumped alongside all the other test output from other suites. So it would look really weird if part of the logs were in one format, and the rest of the logs were in another. Plus the combined log isn't useful for the reftest-analyzer anyway, so using `tbpl` in this case wouldn't have much benefit.

Ideally we get the reftest-analyzer to read the raw structured log and then it doesn't matter what log format we use for reftest, but another bug for another time.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fc99177ac89b
Start using the 'mach' formatter by default in |mach test| and |mach mochitest|, r=gbrown

Comment 10

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/fc99177ac89b