866768 - Move autolog to IT metrics cluster

Status: RESOLVED FIXED

(Testing Graveyard :: Autolog, defect)

Description

[Jonathan Griffin (:jgriffin)]

Metrics wants their ES cluster back, which means we need to migrate all a-team services that use it over to the IT metrics cluster instead.

edmorley is already doing this for OrangeFactor (bug 848834).

1] In order to transition autolog, we'll have to update it and mozautolog to use the new server address: elasticsearch-zlb.webapp.scl3.mozilla.com:9200.

2] If we want to retain access to data already present in autolog, we'll need to migrate it to the new server.

3] If we want to be able to run autolog locally in development mode, we may wan to initiate a periodic production->dev mirror, like edmorley is doing in bug 860256.  Depending on how that's implemented, we may just be able to piggyback on it without writing a separate script.

bc, mcote, I assume you'll want to retain historical data in autolog; how long in the past do you need the data preserved?

Comment 1

[Mark Côté [:mcote]]

We haven't been running unit tests in quite a while, so unless bc objects, I don't think we need any historical info.

Comment 2

[Bob Clary [:bc] (inactive)]

I agree with Mark. The work we did was preliminary and we don't need to save it.

Comment 3

[Jonathan Griffin (:jgriffin)]

Excellent, that makes my life easier :)

Comment 4

[Jonathan Griffin (:jgriffin)]

I landed a change to TPS (which uses autolog) to write results to both the old and new metrics clusters.  After we have some data in the new cluster, I'll change the autolog UI to use it, then modify TPS just to use the new instance.

https://github.com/jonallengriffin/coversheet/commit/782de5a821da67d761db089bab406229c8005ad0

Comment 5

[Jonathan Griffin (:jgriffin)]

Attachment: Make autolog use new ES cluster by default,

This change just switches the default server for autolog to the new prod IT cluster.  I won't deploy it until there's some data written to that cluster that autolog can use.

Comment 6

[Jonathan Griffin (:jgriffin)]

http://hg.mozilla.org/automation/autolog/rev/c5eec0d4a949

Comment 7

[Jonathan Griffin (:jgriffin)]

Deployed and working normally:  http://brasstacks.mozilla.com/autolog/

I've found there are some references to the metrics server in mozautoeslib which I will file a separate bug for, but it doesn't impact autolog, since autolog doesn't use the mozautoeslib defaults.