1180052 - The classification data sent to ElasticSearch should include the treeherder job_id

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P2)

Description

[Ed Morley [:emorley]]

To minimise the impact of switching off bug comments, it would be great to fix OF bug 1180050.

However that requires that Treeherder actually submit valid log URLs to ElasticSearch in the first place. We don't currently, since we only submitted what was actually being used at the time, to reduce complexity.

Example submission:

"_source": {
    "buildtype": "opt",
    "os": "b2g-linux64",
    "tree": "mozilla-inbound",
    "who": "[e-mail]",
    "buildname": "non-buildbot b2g-linux64 test [TC] - Gaia JS Integration Test",
    "date": "2015-06-09",
    "logfile": "00000000",
    "bug": "1147867",
    "machinename": "unknown",
    "rev": "4986f8464f9c",
    "starttime": "0",
    "timestamp": "1433994368",
    "type": "[TC] - Gaia JS Integration Test"
},

We need "logfile" to be either a link to the raw log (ie the log file on ftp.m.o or whereever the TC logs are), or else a link to Treeherder's log viewer for that job. The latter might be better for the user, and also easier for us to generate (since we can just pass job_id to the URL, and don't need to fetch the job_log_url artifacts in ElasticsearchDocRequest()).

Comment 1

[Ed Morley [:emorley]]

Attachment: Include the job_id in the submission to ElasticSearch

Rather than including the log url or a direct link to the log viewer, it makes more sense to include the treeherder job_id, then OrangeFactor can construct the log viewer URL (or URL for artifacts etc) itself.

The dummy "logfile" property is not used anywhere in OrangeFactor, so can be safely removed.

As an added bonus of not just putting a log URL in the existing "logfile" key, is that we don't have to specialcase "00000000" in OrangeFactor, and can simply check to see if the "job_id" key exists or not.

Comment 2

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/15ebebe7655497d7a8e4ebf8d3a6cef880169835
Bug 1180052 - Rename submit_timestamp to classification_timestamp

...when referring to the datetime that a classification was made. This
avoids confusion in ElasticsearchDocRequest, since previously we had
two similarly named variables: 'job_data["submit_timestamp"]' and
'self.submit_timestamp', the former referring to the time the job was
scheduled, the latter to the time the classification was submitted.

https://github.com/mozilla/treeherder/commit/1a9aa94eb49972e12be3a4922d5c86f89c88f742
Bug 1180052 - Include the job_id in the submission to ElasticSearch

So that OrangeFactor can use it to link to the log viewer (and more).
Also stop sending the dummy 'logfile' key, since it's not used by
OrangeFactor.