1079784 - Lower frequency of TBPL UI polling to reduce load on json-pushes

Status: RESOLVED FIXED

(Tree Management Graveyard :: TBPL, defect)

Description

[Ed Morley [:emorley]]

Treeherder's backend polls json-pushes as part of initial data ingestion, and then the UI requests the pushlog via treeherder's API. This means that unlike TBPL, load on json-pushes doesn't scale with the number of clients.

We've been seeing timeouts when treeherder polls json-pushes - which is partly due to bug 1076826 - however one way to help mitigate in the meantime, would be to reduce TBPL UI's polling frequency slightly. This would mean that all of the people who have not yet switched to TBPL don't contribute to the DB deadlocks that are a possible cause.

Now I'm not sure how the load on json-pushes from TBPL clients compares against releng's hgpollers or other consumers of the pushlog, so this may not actually make a huge difference, but worth a try (and for non-sheriffs, TBPL polling time is less noticeable).

Comment 1

[Ed Morley [:emorley]]

Attachment: Reduce frequency of data polling to lower load on json-pushes

See comment 0. I also filed bug 1079788 to figure out where most of the load on json-pushes is coming from (and as such, whether this is just a waste of time).

Comment 2

[Ed Morley [:emorley]]

remote:   https://hg.mozilla.org/webtools/tbpl/rev/cd2d355a207b

Comment 3

[Ed Morley [:emorley]]

Dropped the frequency by half again:
https://hg.mozilla.org/webtools/tbpl/rev/c036367d3309

Comment 4

[Ed Morley [:emorley]]

https://hg.mozilla.org/webtools/tbpl/rev/9e445e011814