1362811 - Make REFTEST prefix optional in unstructured reftest analyzer

Status: RESOLVED FIXED

(Testing :: Reftest, enhancement)

Description

[James Graham [:jgraham]]

This prefix can't be produced by the standard structured logging formatters because it doesn't know that the tests are reftests. Therefore the reftest harness has a hack to add this prefix, and the unstructured analyzer doesn't work with wpt reftests. This can simply be solved by making the prefix optional (but using other features of mozharness logs to identify the start of the TEST-UNEXPECTED string in this case.

Comment 1

[James Graham [:jgraham]]

Attachment: Bug 1362811 - Don't require REFTEST prefix in unstructured reftest analyzer,

This prefix can't be produced by the standard structured logging
formatters because it doesn't know that the tests are
reftests. Therefore the reftest harness has a hack to add this prefix,
and the unstructured analyzer doesn't work with wpt reftests. This can
simply be solved by making the prefix optional, but looking for the
"INFO - " prefix that mozharness adds, when present, to identify the
start of the TEST- string.

Review commit: https://reviewboard.mozilla.org/r/136822/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/136822/

Comment 2

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Comment on attachment 8865162 [details]
Bug 1362811 - Don't require REFTEST prefix in unstructured reftest analyzer,

https://reviewboard.mozilla.org/r/136822/#review140212

Please test that this works for analyzing locally-run reftest output as well as a failure log on treeherder.

r=dbaron with that

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c528edf75b2e
Don't require REFTEST prefix in unstructured reftest analyzer, r=dbaron

Comment 4

[Iris Hsiao [:ihsiao]]

https://hg.mozilla.org/mozilla-central/rev/c528edf75b2e