1124720 - Make test-informant listen for test jobs instead of build jobs

Status: RESOLVED FIXED

(Testing Graveyard :: Test Informant, defect)

Description

[Andrew Halberstadt [:ahal]]

In order to make the switch from manifest parsing to structured logs, we first need to make test informant's pulse listener listen to test jobs instead of build jobs.

That means using the pulse routing key 'unittest.<branch>.#' instead of 'build.<branch>.#'. To see what a pulse message for a test job looks like, see [1].

Test jobs have all the same keys as build jobs required by the model [2], but there are two differences:

1) The config file url (mozinfo.json) will need to be guessed using the 'installer_url' property instead of the 'testsurl' property [3]
2) There will be many test jobs for the same build, so we'll only want to make sure we only parse the manifests if that build hasn't already been added to the database [4].

I think this bug is a good first step because all the changes required for this are also required for the switch to structured logs. This acts as a kind of checkpoint. Once this is working well, we can move on to the next step which is actually using the structured logs.

I know this is likely all confusing, so feel free to ping me if you are stuck, and take as much time as needed.

[1] https://tools.taskcluster.net/pulse-inspector/#!%28%28exchange:exchange/build/normalized,routingKeyPattern:unittest.mozilla-inbound.%23%29%29
[2] https://github.com/mozilla/test-informant/blob/master/informant/models.py#L20
[3] https://github.com/mozilla/test-informant/blob/master/informant/worker.py#L65
[4] https://github.com/mozilla/test-informant/blob/master/informant/worker.py#L72

Comment 1

[Alice Scarpa [:adusca]]

Attachment: Listening for unittest

Comment 2

[Andrew Halberstadt [:ahal]]

Comment on attachment 8554571 [details] [review]
Listening for unittest

Pushed this to the structured_log branch:
https://github.com/mozilla/test-informant/commit/49f1537cc12858ab069ef7388fd674ac73751904