1471717 - Too many timestamps in linux and android unit test logs

Status: RESOLVED INACTIVE

(Testing :: General, enhancement, P3)

Description

[Geoff Brown [:gbrown]]

Tasks that run in linux docker containers -- notably linux and android unit tests -- have a timestamp added to every line in the log by run-task:

https://dxr.mozilla.org/mozilla-central/rev/681eb7dfa324dd50403c382888929ea8b8b11b00/taskcluster/scripts/run-task#91

such as:

setup 2018-06-27T12:18:35.132Z] run-task started in /builds/worker


mozharness scripts also add a timestamp to everything they log, so we see:

[task 2018-06-27T12:22:31.480Z] 12:22:31     INFO - Using env: (same as previous command)


Timestamps are useful, but I don't think there is much value in 2 timestamps which are almost always identical (modulo formatting). On the downside, dual timestamps complicate reading and add load to log processing.

Comment 1

[Geoff Brown [:gbrown]]

Attachment: override mozharness log format for android and linux tests

A simple config override of the mozharness log format eliminates the extra timestamps from most tasks.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1438bc7da109f825171a4d8d9d22c1b46a3e6a2b

https://treeherder.mozilla.org/logviewer.html#?job_id=185244307&repo=try&lineNumber=1255

[task 2018-06-27T20:42:12.318Z]     INFO -  TEST-PASS | ShowSSEConfig | took 11ms
[task 2018-06-27T20:42:12.319Z]     INFO -  TEST-START | TestPrintf

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e82cf1aca043
Remove extra timestamp from mozharness logs for linux and android unit tests; r=jmaher

Comment 3

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/e82cf1aca043

Comment 4

[Geoff Brown [:gbrown]]

Re-opening for bug 1473565. I'll effectively back out changes until those issues can be addressed.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/22a241245097
Disable mozharness log format changes for android and linux; r=me,a=test-only

Comment 6

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/22a241245097

Comment 7

[Andreea Pavel [:apavel]]

(In reply to Eliza Balazs [:ebalazs_] from comment #6)
> https://hg.mozilla.org/mozilla-central/rev/22a241245097

:gbrown since this got to central, it should have been backed out of central and i cannot see that in treeherder. Can you please point out where the backout push? 

Thank you.

Comment 8

[Geoff Brown [:gbrown]]

https://hg.mozilla.org/mozilla-central/rev/e82cf1aca043 was the original push, which caused trouble in bug 1473565.
https://hg.mozilla.org/mozilla-central/rev/22a241245097 was not a "real" backout, but effectively backed out e82cf1aca043 by commenting out the change. Linux and Android test logs are back to normal now.

Comment 9

[BugBot [:suhaib / :marco]]

The leave-open keyword is there and there is no activity for 6 months.
:gbrown, maybe it's time to close this bug?

Comment 10

[Geoff Brown [:gbrown]]

I still hope to get back to this.