Closed
Bug 977059
Opened 10 years ago
Closed 10 years ago
What servers are the BZ-ETL processors accessing?
Categories
(bugzilla.mozilla.org :: Infrastructure, defect)
Tracking
()
RESOLVED
FIXED
People
(Reporter: ekyle, Assigned: fubar)
References
Details
I heard that replication can be several minutes behind depending on which database replication slave is being queried. Furthermore, the routing is not sticky, so which slave you get is different for each connection (sometime getting a delayed slave, sometimes not). The ETL logic does not consider the possibility that different connections can have significantly different global state. I hope the ETL processes (both public and private) are 'stuck' to a single replication slave each; so even if there is replication delay, at least it is reasonably consistent. Thanks
Assignee | ||
Comment 1•10 years ago
|
||
They are configured to use one host: es1.bugs for public, and es4.bugs for private.
Assignee: server-ops-webops → klibby
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Assignee | ||
Comment 2•10 years ago
|
||
glob++ for catching that I'd completely misread your query. :-\ mysql queries are pointed at db-bugs-ro, which is a load balanced address for all of the slaves.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Reporter | ||
Comment 3•10 years ago
|
||
glob: So, is that as bad as I imagine? Maybe I should use just one connection for all my queries? (right now I have a separate connection per thread, each responsible for a block of work)
(In reply to Kyle Lahnakoski [:ekyle] from comment #3) > So, is that as bad as I imagine? Maybe I should use just one connection for > all my queries? (right now I have a separate connection per thread, each > responsible for a block of work) yes; that could get hairy when dealing with queries which are as time sensitive as yours. using a single connection for all your queries makes sense, however i don't think you'd be able to do concurrent requests in that scenario.
Reporter | ||
Comment 5•10 years ago
|
||
thanks. the next version of the ETL will look back 5min since it's last successful run, rather than just one minute.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Updated•10 years ago
|
Component: WebOps: Bugzilla → Infrastructure
Product: Infrastructure & Operations → bugzilla.mozilla.org
You need to log in
before you can comment on or make changes to this bug.
Description
•