Closed
Bug 869648
Opened 11 years ago
Closed 11 years ago
[logparser] Support several destinations
Categories
(Testing :: General, defect)
Testing
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: mcote, Unassigned)
References
Details
Attachments
(1 file)
13.94 KB,
patch
|
jgriffin
:
review+
|
Details | Diff | Splinter Review |
The logparser will have to write to both the prod and dev ES instances. It will also have to write to the metrics instance until we are ready to switch away from it.
Assignee | ||
Comment 1•11 years ago
|
||
Should be applied on top of the patch in bug 869712.
Assignee | ||
Comment 2•11 years ago
|
||
The shell scripts will also have to be updated; however, since we will be changing the servers at least a couple times in the near future, I think we should modify the generated script (logparser/bin/savelogs-gen.sh) by hand for now and check in an updated version when we finally stop using the metrics cluster.
Comment 3•11 years ago
|
||
Comment on attachment 747475 [details] [diff] [review] Support multiple ES servers Review of attachment 747475 [details] [diff] [review]: ----------------------------------------------------------------- ::: logparser/logparser.py @@ +49,5 @@ > self.delete_file = delete_file > self.logger = logger > + if not es_servers: > + es_servers = DEFAULT_ES_SERVERS[:] > + self.es_servers = es_servers or more compactly: self.es_servers = es_servers or DEFAULT_ES_SERVERS[:] @@ +145,4 @@ > > if 'testsuites' in data: > for ts_index, ts in enumerate(data.get('testsuites', [])): > + for tg in testgroups: It seems like we're repeating 'for tg in testgroups' unnecessarily. Can't we just do: testgroups = [...] for tg in testgroups: # everything goes here # ... tg.submit() @@ +166,4 @@ > > for tp_index, tp in enumerate(ts.get('passes', [])): > + for tg in testgroups: > + tg.add_test_pass(doc_type='testpasses_' + testgroup.testsuites[-1].testsuite, s/testgroup/tg
Attachment #747475 -
Flags: review?(jgriffin) → review+
Assignee | ||
Comment 4•11 years ago
|
||
Fixed both. Also added "[default: localhost:9200]" to the --es-server help. We're repeating that default in a few places now, but meh for another day. https://hg.mozilla.org/automation/logparser/rev/abada2fd0fab We should deploy this as soon as possible to minimize the amount of data we need to copy over.
Assignee | ||
Updated•11 years ago
|
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Comment 5•11 years ago
|
||
So are we ready to deploy this and add an --es-server elasticsearch-zlb.webapp.scl3.mozilla.com:9200 argument to logparser's startup script?
Assignee | ||
Comment 6•11 years ago
|
||
I believe so. The necessary flow was created in bug 869627. We should probably wait on adding the dev es server until we figure out an expiration mechanism. Just keep an eye on it after deploying--or you can wait until tomorrow morning when we're both around.
Comment 7•11 years ago
|
||
I'll do it tomorrow AM, just in case!
Comment 8•11 years ago
|
||
Deployed, and verified that it's now writing data to both the metrics and the IT prod clusters.
You need to log in
before you can comment on or make changes to this bug.
Description
•