1210488 - archiver_client.py needs to output errors that can be parsed by Treeherder

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

I'm not sure if this should start as a Treeherder bug or some releng-y place, but this Linux Static build failure in https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&fromchange=65257cf4c5f8&group_state=expanded&filter-searchStr=Linux%20x64%20debug%20Static%20Checking%20Build%20%28S%29&tochange=65257cf4c5f8 doesn't show a failure summary.

It'd be nice if it did, whether that means Treeherder parses out the existing S3 error or the build system makes that S3 error into a parsable format for Treeherder.

Comment 1

[Ed Morley [:emorley]]

The line in question is:

2015-10-01 10:13:32,179 An s3 URL could not be determined even though archiver task completed. Checkarchiver logs for errors. Task status: no status available at this point.

It needs some kind of error prefix, since otherwise we're just going to have to hardcode every possible string it may output in Treeherder's log parser.

This is output by:
https://hg.mozilla.org/build/tools/file/cfec1b98cad7/buildfarm/utils/archiver_client.py#l101

Jordan, would you mind taking a look? :-)

Comment 2

[Jordan Lund (:jlund)]

> It needs some kind of error prefix, since otherwise we're just going to have
> to hardcode every possible string it may output in Treeherder's log parser.
> 
> This is output by:
> https://hg.mozilla.org/build/tools/file/cfec1b98cad7/buildfarm/utils/
> archiver_client.py#l101
> 
> Jordan, would you mind taking a look? :-)

it is strange that we ended up with a 'good' status and a 'bad' state:
2015-10-01 10:13:32,179 current task status: no status available at this point. state: SUCCESS

This was from a few days ago. It be nice to catch this in action. I'll file a patch now so to add static output error lines.

Comment 3

[Jordan Lund (:jlund)]

Attachment: 151005_1210488_add_static_error_lines_to_archiver.patch

will something like this work?

Comment 4

[Ed Morley [:emorley]]

Comment on attachment 8669760 [details] [diff] [review]
151005_1210488_add_static_error_lines_to_archiver.patch

Whilst we could add "Archiver failure" to the list of error terms, that list is just going to grow over time.

Ideally for things that don't have error lines prefixed with the mozharness error level output, we'd use "Automation Error: " or one of the others in:
https://github.com/mozilla/treeherder/blob/beeef2ed50ff7c4a9fffc3012184ba3a9b5863e5/treeherder/log_parser/parsers.py#L362-L410

Comment 5

[Jordan Lund (:jlund)]

Attachment: 151007_1210488_add_static_error_lines_to_archiver.patch

same as before but this time with a regex that TH knows about

Comment 6

[Ed Morley [:emorley]]

Comment on attachment 8670887 [details] [diff] [review]
151007_1210488_add_static_error_lines_to_archiver.patch

Looks good, thank you :-)

Comment 7

[Jordan Lund (:jlund)]

Comment on attachment 8670887 [details] [diff] [review]
151007_1210488_add_static_error_lines_to_archiver.patch

thanks!

https://hg.mozilla.org/build/tools/rev/d0022b4cdbc0