977059 - What servers are the BZ-ETL processors accessing?

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: Infrastructure, defect)

Description

[Kyle Lahnakoski [:ekyle]]

I heard that replication can be several minutes behind depending on which database replication slave is being queried.  Furthermore, the routing is not sticky, so which slave you get is different for each connection (sometime getting a delayed slave, sometimes not).  

The ETL logic does not consider the possibility that different connections can have significantly different global state. 

I hope the ETL processes (both public and private) are 'stuck' to a single replication slave each; so even if there is replication delay, at least it is reasonably consistent.

Thanks

Comment 1

[Kendall Libby [:fubar] (he/him)]

They are configured to use one host: es1.bugs for public, and es4.bugs for private.

Comment 2

[Kendall Libby [:fubar] (he/him)]

glob++ for catching that I'd completely misread your query. :-\

mysql queries are pointed at db-bugs-ro, which is a load balanced address for all of the slaves.

Comment 3

[Kyle Lahnakoski [:ekyle]]

glob:

So, is that as bad as I imagine?  Maybe I should use just one connection for all my queries?  (right now I have a separate connection per thread, each responsible for a block of work)

Comment 4

[:glob ✱]

(In reply to Kyle Lahnakoski [:ekyle] from comment #3)
> So, is that as bad as I imagine?  Maybe I should use just one connection for
> all my queries?  (right now I have a separate connection per thread, each
> responsible for a block of work)

yes; that could get hairy when dealing with queries which are as time sensitive as yours.

using a single connection for all your queries makes sense, however i don't think you'd be able to do concurrent requests in that scenario.

Comment 5

[Kyle Lahnakoski [:ekyle]]

thanks.

the next version of the ETL will look back 5min since it's last successful run, rather than just one minute.