Closed Bug 1072422 Opened 10 years ago Closed 10 years ago

Use 'fromchange' for pushlog ingestion, rather than relying on <10 pushes since last poll

Categories

(Tree Management :: Treeherder, defect, P1)

defect

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 1072291

People

(Reporter: emorley, Unassigned)

Details

In bug 1071577, we switched away from using &pushID=N when polling json-pushes, to using json-pushes with &maxhours=24. Unfortunately maxhours is not a valid param, so we're effectively using json-pushes with no start params at all, which means we're requesting the last 10 pushes made to each repo.

This increases network transfer & processing costs, and also means that if there were more than 10 pushes since the last successful poll, we'd miss pushes. (10 pushes in a few mins is unlikely in normal operation, but if ingestion went down for 15-20+ mins, there's a chance we might hit it, given push rates at peak times).

As an added bonus, using &fromchange doesn't have the same pitfalls as pushID - in that we correctly get a 404 if the repo was reset (ie bug 1065771 doesn't apply). eg:
https://hg.mozilla.org/mozilla-central/json-pushes?full=1&fromchange=invalid
Summary: Use 'fromchange' for pushlog ingestion, rather than relying on fewer than 10 pushes since last poll → Use 'fromchange' for pushlog ingestion, rather than relying on <10 pushes since last poll
This is now happening as part of bug 1072291.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.