682749 - With a slow and unreliable connection, usebuildbot frequently gets stuck, probably from too many connections

Status: RESOLVED WORKSFORME

(Tree Management Graveyard :: TBPL, defect)

Description

[Phil Ringnalda (:philor)]

I'm not entirely sure what's happening when this happens, but with my frequently slow and unreliable 3G connection, particularly if I have several tabs open to different trees, I get into a state that looks like we've maxed out the number of simultaneous connections to a server, and we aren't timing them out, so not only is the loading progress stuck spinning at a particular percent complete trying to load a bunch of versions of getRevisionBuilds.php, but also I can't even open a log (since that's now another connection to the tbpl server) until I refresh one tab (doesn't appear to matter which one) so that the other stalled ones can quickly grab its connections, get unstuck, and then let it connect.

Lots of indirect things, like only fetching results that have changed since the time of the last fetch so each request is smaller and faster, or letting caching work rather than fighting to avoid it at all costs so we aren't repeatedly fetching all of the exact same results for a push which has completely finished running everything, or making a single request for results for all the pushes we're showing instead of a request per push are likely to fix this without having to figure out what's stalling and not timing out quickly enough, but since it's my biggest annoyance with usebuildbot, I figured I'd file it anyway.

Comment 1

[Phil Ringnalda (:philor)]

While I'm not yet in the completely stuck state, I left the Web Console open on all three tabs I've got open now, and one of them is stuck at "Loading 89% ..." because despite having successfully loaded getRevisionBuilds.php for all five pushes nine times since, it's still trying to load two of the five from a refresh that's now almost twenty minutes old. Do we *ever* time out those requests?

Comment 2

[Ed Morley [:emorley]]

I haven't seen this for months/years :-)