1666195 - Add a button to "filter by this test path" near the failure line in the failure summary

Status: RESOLVED FIXED

(Tree Management :: Treeherder, enhancement)

Description

[Bogdan Tara[:bogdan_tara | bogdant]]

This suggestion would help the sheriffing activity while the manifest scheduling is enabled.

The improvement would add an easily and direct accessible mean to filter by a test path.

Other helpers could be:

• exposing the "filter by test path" button
• adding a button (in the failure summary, near the failure line) which sends the test path to the clipboard.

Comment 1

[Cameron Dawson [:camd]]

This was addressed in https://github.com/mozilla/treeherder/pull/6889

I did not add a button to just copy the path, as there is already a button to copy the whole string of the failed line. It seemed like having two copy buttons could be confusing. But we could consider adding one at a later time.

Comment 2

[Andreea Pavel [:apavel]]

(In reply to Cameron Dawson [:camd] from comment #1)

This was addressed in https://github.com/mozilla/treeherder/pull/6889

I did not add a button to just copy the path, as there is already a button to copy the whole string of the failed line. It seemed like having two copy buttons could be confusing. But we could consider adding one at a later time.

Hi, I think Bogdan's point here was to have a button that directly filters by test path, meaning after it's clicked TH already displays only the jobs that run on that path. This helping us skip a few steps in investigating failures using the existing process of test path filtering.

Comment 3

[Marco Castelluccio [:marco]]

(In reply to Andreea Pavel [:apavel] from comment #2)

(In reply to Cameron Dawson [:camd] from comment #1)

This was addressed in https://github.com/mozilla/treeherder/pull/6889

I did not add a button to just copy the path, as there is already a button to copy the whole string of the failed line. It seemed like having two copy buttons could be confusing. But we could consider adding one at a later time.

Hi, I think Bogdan's point here was to have a button that directly filters by test path, meaning after it's clicked TH already displays only the jobs that run on that path. This helping us skip a few steps in investigating failures using the existing process of test path filtering.

I agree this would be helpful, when debugging mozci I often wished I had something like that.

Comment 4

[Andreea Pavel [:apavel]]

@Marco the button has already been added in TH.
However, add new jobs using test path filter is not working as it should.

EX: filter by test path https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&test_paths=browser%2Fcomponents%2Fplaces%2Ftests%2Fbrowser&tochange=f01f746874d4b81eb44ec92a588edfb72b7329a8&fromchange=ea5674f9a511fda3697a159e2be34fad0623dda3&selectedTaskRun=HRZz_BKfTiaVoL2_J0Sf8Q.0

I couldn't add tests on 5e89d54f4951124dd90bb600f32aa29f232f32eb and 1359af4d35d17f5db3e8e35e34e2f86d2a5b4ea2, however I could backfill bc1 from f01f746874d4b81eb44ec92a588edfb72b7329a8 which runs on the same path too (see test groups)

https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&tochange=f01f746874d4b81eb44ec92a588edfb72b7329a8&fromchange=ea5674f9a511fda3697a159e2be34fad0623dda3&searchStr=os%2Cx%2C10.14%2Cwebrender%2Cdebug%2Cmochitests%2Ctest-macosx1014-64-qr%2Fdebug-mochitest-browser-chrome-e10s%2Cbc1&selectedTaskRun=HRZz_BKfTiaVoL2_J0Sf8Q.0

@Cameron I don't know if this is due to manifest scheduling or a TH issue?

Comment 5

[Cameron Dawson [:camd]]

Attachment: Screen Shot 2020-12-11 at 4.13.00 PM.png

The new filter button is in the middle between the copy and the log viewer icon. Is this doing something different than what you'd asked for?

Comment 6

[Cameron Dawson [:camd]]

(In reply to Marco Castelluccio [:marco] from comment #3)

(In reply to Andreea Pavel [:apavel] from comment #2)

(In reply to Cameron Dawson [:camd] from comment #1)

This was addressed in https://github.com/mozilla/treeherder/pull/6889

I did not add a button to just copy the path, as there is already a button to copy the whole string of the failed line. It seemed like having two copy buttons could be confusing. But we could consider adding one at a later time.

Hi, I think Bogdan's point here was to have a button that directly filters by test path, meaning after it's clicked TH already displays only the jobs that run on that path. This helping us skip a few steps in investigating failures using the existing process of test path filtering.

I agree this would be helpful, when debugging mozci I often wished I had something like that.

(In reply to Andreea Pavel [:apavel] from comment #4)

@Marco the button has already been added in TH.
However, add new jobs using test path filter is not working as it should.

EX: filter by test path https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&test_paths=browser%2Fcomponents%2Fplaces%2Ftests%2Fbrowser&tochange=f01f746874d4b81eb44ec92a588edfb72b7329a8&fromchange=ea5674f9a511fda3697a159e2be34fad0623dda3&selectedTaskRun=HRZz_BKfTiaVoL2_J0Sf8Q.0

I couldn't add tests on 5e89d54f4951124dd90bb600f32aa29f232f32eb and 1359af4d35d17f5db3e8e35e34e2f86d2a5b4ea2, however I could backfill bc1 from f01f746874d4b81eb44ec92a588edfb72b7329a8 which runs on the same path too (see test groups)

https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&tochange=f01f746874d4b81eb44ec92a588edfb72b7329a8&fromchange=ea5674f9a511fda3697a159e2be34fad0623dda3&searchStr=os%2Cx%2C10.14%2Cwebrender%2Cdebug%2Cmochitests%2Ctest-macosx1014-64-qr%2Fdebug-mochitest-browser-chrome-e10s%2Cbc1&selectedTaskRun=HRZz_BKfTiaVoL2_J0Sf8Q.0

@Cameron I don't know if this is due to manifest scheduling or a TH issue?

This is due to bug 1669513 because we only currently check the paths of the originally run tasks. We need to get task maps for every time tasks are added with backfilling, add new jobs, or whatever else. I'm investigating this bug now, fwiw.

Comment 7

[Andreea Pavel [:apavel]]

Thank you for looking into this!
Your work here is much appreciated.

Comment 8

[Marco Castelluccio [:marco]]

(In reply to Andreea Pavel [:apavel] from comment #4)

@Marco the button has already been added in TH.

Sorry, I hadn't noticed. Let me reclose :)