Closed
Bug 1125856
Opened 9 years ago
Closed 9 years ago
Use the master db to perform queries on data ingestion
Categories
(Tree Management :: Treeherder: Data Ingestion, defect, P1)
Tree Management
Treeherder: Data Ingestion
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: mdoglio, Assigned: mdoglio)
References
Details
Attachments
(1 file)
Now that bug 1125464 has landed, some new errors appear in newrelic. See https://rpm.newrelic.com/accounts/677903/applications/5585473/traced_errors/2973275709?original_error_id=2973401858 for an example. We should always use the master db when ingesting data to prevent errors due to delays in replication.
Assignee | ||
Updated•9 years ago
|
Assignee: nobody → mdoglio
Comment 1•9 years ago
|
||
There's also: https://rpm.newrelic.com/accounts/677903/applications/5585473/traced_errors/2973462084/similar_errors?original_error_id=2973462084 treeherder.etl.mixins:CollectionNotLoadedException Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/newrelic-2.40.0.34-py2.6-linux-x86_64.egg/newrelic/hooks/application_celery.py", line 66, in wrapper return wrapped(*args, **kwargs) File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__ return self.run(*args, **kwargs) File "/data/www/treeherder.allizom.org/treeherder-service/treeherder/etl/tasks/buildapi_tasks.py", line 32, in fetch_buildapi_running RunningJobsProcess().run() File "/data/www/treeherder.allizom.org/treeherder-service/treeherder/etl/buildapi.py", line 576, in run filter_to_project=filter_to_project) File "/data/www/treeherder.allizom.org/treeherder-service/treeherder/etl/mixins.py", line 167, in load raise CollectionNotLoadedException(errors) CollectionNotLoadedException: [mozilla-beta] Error posting data to jobs: {"detail": "u'2e4c23ca12cc67d7fdfcb826606da696f5d6eda6'"} [mozilla-inbound] Error posting data to jobs: {"detail": "u'c9d1a42d6a557343c8edfb3cd2b8a5f05232a8cb'"}
Assignee | ||
Comment 2•9 years ago
|
||
Attachment #8554659 -
Flags: review?(emorley)
Assignee | ||
Comment 3•9 years ago
|
||
:edmorley the error you posted seems to be the etl process counter part of the error I linked from newrelic.
Comment 4•9 years ago
|
||
Comment on attachment 8554659 [details] [review] Github PR #351 on treeherder-service These changes look good; though I haven't yet looked to see if we need any more than this. But let's land this on stage and see how the errors look :-)
Attachment #8554659 -
Flags: review?(emorley) → review+
Comment 5•9 years ago
|
||
Commit pushed to master at https://github.com/mozilla/treeherder-service https://github.com/mozilla/treeherder-service/commit/fa485b94e299ff847db0175b8642994df258111f Bug 1125856 - Use master db on data ingestion
Assignee | ||
Comment 6•9 years ago
|
||
The etl nodes on both staging and production are missing new flows to the readonly db.
Assignee | ||
Updated•9 years ago
|
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•