1124613 - Log parser should store the search terms used to query for bug suggestions

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Data Ingestion, defect, P3)

Description

[Ed Morley [:emorley]]

Currently the bug suggestions artefact (to be renamed in bug 1091055) only lists:
* bugs found (under open_recent and all_others)
* the error line with mozharness prefix removed (under 'search', though that's not a great name - bug 1091060 will rename)

What we don't have is the actual search terms (we can extract more than one, if the first did not find anything) that were extracted from that string and used to query the bugscache. 

Having this would allow us to:
1) Debug missing bug suggestions
2) Make more intelligent guesses in the intermittent bug filer as to what search terms must be included in the bug summary - since we'll have the terms from the job used to file the bug.
3) Perhaps tweak the highlighting used in the UI when showing the failure line.

Comment 1

[Ed Morley [:emorley]]

Attachment: Log parser: Save the search terms used for bug suggestions

Comment 2

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder-service

https://github.com/mozilla/treeherder-service/commit/5e4b356d259fa65fbf60c3f13efae3883eecf444
Bug 1124613 - Log parser: Save the search terms used for bug suggestions

For debugging & also for when filing new intermittent failure bugs, it
is useful to see which search terms were extracted from a log failure
line, and used to query the bugscache for bug suggestions. In the future
this could be used by an intermittent bug filer to verify the bug
summary contained the term extracted for failures of that type.

Comment 3

[Ed Morley [:emorley]]

Thank you for the review :-)

Comment 4

[Ed Morley [:emorley]]

Example of this working:
https://treeherder.mozilla.org/api/project/mozilla-central/artifact/?job_id=1241338&name=Bug+suggestions&type=json

[{
  "blob": [{
    "search": "746 INFO TEST-UNEXPECTED-FAIL | browser/base/content/test/plugins/browser_CTP_iframe.js | Test 1, Waited too long for the overlay to become invisible. -",
    "search_terms": ["browser_CTP_iframe.js"],
    "bugs": {
...
    }
  }],
  "type": "json",
  "id": 7692633,
  "name": "Bug suggestions",
  "job_id": 1241338
}]