1208186 - SETA data should reside on a production machine

Status: RESOLVED WONTFIX

(Testing :: General, defect)

Description

[Kim Moir [:kmoir] ET]

Currently we consume SETA data from http://alertmanager.allizom.org which is not a production server that is monitored by MOC.  I didn't realize this until I opened bug 1200838 to monitor it and Amy pointed this out.  In any case, since this data is needed by our production CI farm to trim the number of tests we run it should be on a production server that is monitored by MOC, backed up etc.

Amy do you have suggestions where it should be moved to? Releng web cluster?

Comment 1

[Amy Rich [:arr] [:arich]]

Is it a webapp? What are it's requirements?

If it's a webapp should probably go on the relengweb cluster (and possibly be part of relengapi). I'll ask dustin to weigh in, since he's been working more with that sort of app deployment.  The alternative for a webapp is to put it in AWS or heroku or something like that.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Are you suggesting that this become a releng service?  If so, we'd need to look at the underlying code.  Ideally, yes, deployed as a Heroku app, but that requires 12-factor compliance.  Then again, deploying as part of relengapi does, too (plus rewriting as a blueprint).  We could probably host it as a different webapp on the releng web cluster (I assume it's on the webops staging cluster now?), but I don't see any great advantage over just hosting it on the webops generic cluster in that case.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

this is a database that is populated with some crontab scripts to suck data from treeherder, then we do analysis of it with other crontab jobs to generate the SETA data.  Likewise we could post this in-tree and update it every couple of days whenever there is a change.

It just depends on if we want something for the next year online or post updates in tree once or twice a week for the next year.

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

For access from the TaskCluster decision task, in-tree would be best: it eliminates a potential tree-closing dependency.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

in-tree might be the best solution then- now to figure out how to get the data for buildbot reconfigs and a simple process for updating the data in-tree.

Comment 6

[Kim Moir [:kmoir] ET]

Closing since we are going to save the SETA data in tree