1096877 - Increase the frequency of builds-2hr generation after bug 1088354

Status: RESOLVED WONTFIX

(Infrastructure & Operations :: RelOps: General, task)

Description

[Ed Morley [:emorley]]

As I understand it, builds-4hr is currently generated once a minute.

Once we switch to instead only generating builds-2hr in bug 1088354, we should see if we can increase the frequency, in order to help with bug 1096863. If we're only exporting the last 2 hours of jobs, we should be able to half the cron time - though I'll take anything lower than the current value :-)

Comment 1

[Ed Morley [:emorley]]

Increasing the frequency will likely cause bug 942616 to get worse, so we probably need to fix that first.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Cron doesn't run more than once per minute!  We could switch to another tool, but still, a script that runs every 30 seconds seems a little crazy.

More to the point, though, this defeats the purpose of switching to builds-2hr, which was to eliminate the needlessly repetitive process of generating the same data, writing it to disk, downloading it into tbpl/treeherder, and parsing it.  If we halve the data but double the frequency, we're right back where we were (or probably a little worse, due to various overheads).

I think the fix for bug 1096863 is probably to DTRT and consume from pulse (bug 1031238).  I understand that's a much harder problem, but one we have to solve at some point.  And I'm happy to have yet another desirable outcome waiting for that problem to be solved, as it makes closing bug 1026109 a little more likely!

So, motion to resolve WONTFIX?

Comment 3

[Ed Morley [:emorley]]

(In reply to Dustin J. Mitchell [:dustin] from comment #2)
> Cron doesn't run more than once per minute!

Ah that I did not know :-)

Thinking more after filing the deps of bug 1096863, I think this bug isn't the long pole, since unless bug 1096880 can go sub 1 minute (which I think we would struggle with), then increasing the frequency of generation of builds-2hr is pointless (even ignoring undesirable, given comment 2), since we wouldn't see the benefit.

Agree switching to Pulse is the right thing to do, I just think it will need a lot of coordination between various teams, and a chunk of their quarterly goal time, which is something that we've not really been able to achieve before. Hopefully we can all sit down at the coincidental work week and scope out the task, such that it's more of a known quantity and on future goal commitments :-)

Comment 4

[Ed Morley [:emorley]]

I've commented in bug 1031238 and filed bug 1097080 to hopefully make some progress :-)