1368985 - job_loader:MissingPushException: No push found in {go-bouncer,socorro} for revision X

Status: NEW

(Tree Management :: Treeherder: Data Ingestion, defect, P2)

Description

[Ed Morley [:emorley]]

Bug 1351644 added several repositories to Treeherder, however for some of these we're receiving jobs from pulse but no pushes. This results in the Pulse jobs ingestion retrying job ingestion multiple times (hoping that the push will have been ingested in the meantime), causing queue backlogs and noise on New Relic.

Repositories affected are:
* go-bouncer
* socorro
* mozillians-tests
* bmo-master

https://rpm.newrelic.com/accounts/677903/applications/14179757/filterable_errors?tw%5Bend%5D=1496228477&tw%5Bstart%5D=1495623677#/table?top_facet=transactionUiName&primary_facet=project&barchart=barchart&filters=%5B%7B%22key%22%3A%22error.class%22%2C%22value%22%3A%22treeherder.etl.job_loader%3AMissingPushException%22%2C%22like%22%3Afalse%7D%5D&_k=1ubxa3

Strangely I've now checked Treeherder and there are pushes for those repos?

Comment 1

[Armen [:armenzg]]

You can ./manage.py ingest task and maybe some code tweaking to emulate some of what comes via Pulse tasks.
I remember one of the projects using a branch called staging.tmp. It might have been with android-components.

Comment 2

[Sarah Clements [:sclements]]

This pollutes the logs and we end up with a high exception count. This is where this happens: https://github.com/mozilla/treeherder/blob/master/treeherder/etl/job_loader.py#L104

Comment 3

[Armen [:armenzg]]

A specific repo fix -> bug 1656036.

Comment 4

[Armen [:armenzg]]

The root solution bug 1656244.