611448 - Buildbot JSON needs logURL field

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P4)

Description

[Markus Stange [:mstange]]

Getting the URL of the log that's uploaded for a build is trickier than it should be. At the moment it's possible for most builds by connecting the packageUrl, build_url, builddir and buildnumber fields, as demonstrated here:
http://hg.mozilla.org/users/dbaron_mozilla.com/buildbot-json-tools/file/28b24c910790/filter-buildbot-json.py#l47

However, that neither works for Talos runs (no builddir field) nor for busted builds (no packageUrl or build_url field).

Comment 1

[Chris AtLee [:catlee]]

The problem right now is that the log is uploaded asynchronously after the build is completed.  We could pre-compute what the log url _should_ be, but if the actual log transfer fails for some reason the logUrl property wouldn't be correct.  This might be a decent intermediate step to take though.

If we change how we're doing log uploading from a asynchronous status plugin that's triggered after the build is finished to a master-side step that runs as the last step, we will be able to set this property before the build is finished, so it would show up in the list of build properties as well.

Comment 2

[Markus Stange [:mstange]]

(In reply to comment #1)
> The problem right now is that the log is uploaded asynchronously after the
> build is completed.  We could pre-compute what the log url _should_ be, but if
> the actual log transfer fails for some reason the logUrl property wouldn't be
> correct.  This might be a decent intermediate step to take though.

Yes, that would be a big step forward.

Not knowing the log filename for Talos runs and failed builds is the biggest obstacle to a Tinderbox-free TBPL at the moment.

Comment 3

[Markus Stange [:mstange]]

This is not urgent anymore. The Run._log function at http://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/file/tip/dataimport/import-buildbot-data.py seems to work pretty well.

Comment 4

[Phil Ringnalda (:philor)]

(In reply to Chris AtLee [:catlee] from comment #1)
> We could pre-compute what the log url _should_ be, but
> if the actual log transfer fails for some reason the logUrl property
> wouldn't be correct.

It would, however, be exactly as good as tbpl's current reverse-engineering. It also would have saved you from the backout of bug 671450, and would save me the comment I'm trying to write in bug 685299, which is a truly awful "reverse engineering of tbpl's reverse engineering of buildbot's upload path rules so that another chunk of buildbot can upload where tbpl expects it to."

Comment 5

[Chris AtLee [:catlee]]

this property will be called 'log_url'