1163758 - Port the OF logparser to AWS

Status: RESOLVED WONTFIX

(Tree Management Graveyard :: OrangeFactor, defect)

Description

[Jonathan Griffin (:jgriffin)]

The OF logparser is currently housed in PHX1, but we need to move it out of there.  I'm going to port it to AWS.

Comment 1

[Ed Morley [:emorley]]

Seeing as the method of calculating the denominator for the orangefactor is already incorrect - I wonder if we could just use a treeherder API for "number of pushes" and use that for now instead?

Or alternatively, make the OF graph just use the absolute number of failures, rather than a rate?

It just seems pretty unfortunate to have to re-host the log parser, when we don't actually need the contents of the logs parsed, and will hopefully bin elasticsearch entirely soon.

Comment 2

[Jonathan Griffin (:jgriffin)]

Is there already an API in Treeherder that could be used for this?

Transplanting this to AWS is pretty trivial and doesn't incur long-term costs, so it's a backup strategy in case no one has the time to do the necessary refactoring in OF before we need to be out of PHX1.

Comment 3

[Ed Morley [:emorley]]

Re API, I guess we could just use the the pushlog directly? 

Though if it's not much hassle then perhaps AWS is the best solution short term..

Comment 4

[Jonathan Griffin (:jgriffin)]

I tried this out and it was easier than I thought; I'll file a separate bug rather than overloading this one.

Comment 5

[Jonathan Griffin (:jgriffin)]

We're just removing the dependency on logparsing instead in bug 1164260.