526159 - Add a server-side pushlog cache or something else so that tbpl looks less like a DoS

Status: RESOLVED WORKSFORME

(Tree Management Graveyard :: TBPL, defect)

Description

[Phil Ringnalda (:philor)]

This afternoon, Aravind blocked tbpl from accessing hg.mozilla.org for a while, because someone was spidering and overloading hg, making it screw up tinderbox pulls, and when he looked at the logs, there appeared to be two people hitting it too hard, the other 'un and themasta.com.

From a possibly too-brief look at the code, it looks like every time someone's 120 second timeout fires, they'll hit http://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/file/74d47175b3e0/fetchraw.php which will fetch the pushlog and push a munged version on to the client, so while a dozen people evenly spaced would look like the "somebody's hitting it every 10s or so" that looked bad, two dozen would look twice as bad, and so on.

The first two things that occur to me are to have tbpl cache the pushlog on the server, so it could limit how often it hits hg.m.o no matter how many people are using it, or, to have hg.m.o provide something more like what tbpl wants to consume, so that like with tinderbox each client could fetch it directly from hg.m.o, fetching fewer bytes and letting Aravind figure out how often he wants to let the json or whatever update and hit hg itself.

Comment 1

[Arpad Borsos [:Swatinem]]

Prefiltering / Caching on the server side would also be very beneficial for the heavyweight Tinderbox JSON files.
I did propose a json format that combines the three datasources (pushlog, tinderbox, unittest) some time ago but there wasn't very much agreement to further pursue it.

Comment 2

[(not currently active) Ted Mielczarek]

One thing that'd help here would be to fix the pushlog JSON output, so TBPL can use that directly, and since we've added the cross-site headers, TBPL should be able to then fetch the JSON data client-side instead.

Comment 3

[Phil Ringnalda (:philor)]

Yeah, fixed by whichever bug switch to direct client-side fetching the pushlog JSON, but I'm too lazy to look up which that was.