1234247 - Stop using fetch-missing-pushlogs for pending/running job ingestion

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Data Ingestion, defect, P2)

Description

[Ed Morley [:emorley]]

In order that we can remove the fetch-missing-pushlog tasks, we need to:
1) Improve logging of instances where it's being used
2) Fix causes of those (if any) now that bug 1090289 is fixed.

However even with improved logging, we know that one cause of us running the task, is that job ingestion raced with the standard pushlog ingestion -- and had we waited just a bit longer we'd have ingested the push anyway.

Bug 1090441 proposed trying to filter out jobs that have only just been scheduled, but that's fiddly given the way we batch up resultset queries.

Much easier would be to just stop running fetch-missing-pushlogs for pending/running jobs, and leave it for completed jobs as a last resort fallback, giving plenty of time for the standard pushlog ingestion to do its thing.

Comment 1

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:partially-stop-fetch-missing-pushlogs > mozilla:master

Comment 2

[Cameron Dawson [:camd]]

Comment on attachment 8748302 [details] [review]
[treeherder] mozilla:partially-stop-fetch-missing-pushlogs > mozilla:master

lgtm.  :)

Comment 3

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/025b6f7f4c37aa920cd5128e670ef2d8998de635
Bug 1234247 - Only fetch missing pushlogs for completed jobs

The long-term aim is to make the fetch missing pushlogs tasks redundant,
by fixing the causes of missing pushes (if they're not already fixed).

However currently it's hard to identify whether it's still required,
since the standard pushlog task can race with job ingestion, causing us
to fetch 'missing' pushes that would have been otherwise retrieved fine.

By stopping the retrieval of missing pushes for pending/running jobs, we
give the standard pushlog retrieval time to run, prior to the job being
completed. If the push is still missing once we ingest the completed
job, we'll run the fetch missing pushes task as before - as a last
resort fall-back.