Use 'fromchange' for pushlog ingestion, rather than relying on <10 pushes since last poll

RESOLVED DUPLICATE of bug 1072291

Status

Tree Management
Treeherder
P1
normal
RESOLVED DUPLICATE of bug 1072291
3 years ago
3 years ago

People

(Reporter: emorley, Unassigned)

Tracking

Details

(Reporter)

Description

3 years ago
In bug 1071577, we switched away from using &pushID=N when polling json-pushes, to using json-pushes with &maxhours=24. Unfortunately maxhours is not a valid param, so we're effectively using json-pushes with no start params at all, which means we're requesting the last 10 pushes made to each repo.

This increases network transfer & processing costs, and also means that if there were more than 10 pushes since the last successful poll, we'd miss pushes. (10 pushes in a few mins is unlikely in normal operation, but if ingestion went down for 15-20+ mins, there's a chance we might hit it, given push rates at peak times).

As an added bonus, using &fromchange doesn't have the same pitfalls as pushID - in that we correctly get a 404 if the repo was reset (ie bug 1065771 doesn't apply). eg:
https://hg.mozilla.org/mozilla-central/json-pushes?full=1&fromchange=invalid
(Reporter)

Updated

3 years ago
Summary: Use 'fromchange' for pushlog ingestion, rather than relying on fewer than 10 pushes since last poll → Use 'fromchange' for pushlog ingestion, rather than relying on <10 pushes since last poll
(Reporter)

Comment 1

3 years ago
This is now happening as part of bug 1072291.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1072291
(Reporter)

Updated

3 years ago
No longer blocks: 1059400, 1071577
You need to log in before you can comment on or make changes to this bug.