792904 - DB changes to be more resilient (again) against late ADU pushes

Status: RESOLVED WONTFIX

(Socorro :: General, task, P1)

Description

[Robert Kaiser]

We once again had a late ADU push from metrics today, and post-mobeta code apparently has a hard time dealing with that.
Bug 792869 has been filed for backfilling this time, but we shouldn't need manual interaction every time this happens, esp. as we know this can happen all the time.

Pre-mobeta we did at least do TCBS and crash numbers and filled in ADU-dependent data based on an hourly job later if it wasn't present. We need to become more flexible like that again - not sure if crontabber or so helps there, I just want to have this tracked so we make sure it's being addressed.

Comment 1

[Robert Helmer [:rhelmer]]

Technically TCBS did run, it's just that now the graphs are built differently and depend on ADU.

crontabber is a better long-term solution for this (since it tracks and can retry dependencies like this).

Comment 2

[[:jberkus] Josh Berkus]

I'll go further and say that there is no way to fix this persistent issue without something which performs the same functions as crontabber is spec'd to do.

Comment 3

[Robert Kaiser]

I have no problem with the solution being crontabber if it comes rather soon, I just want the problem to be tracked and solved. ;-)

Comment 4

[Robert Kaiser]

BTW, I'm also pretty sure that not being able to calculate aggregates like TCBS without having ADU means that many third-party installations of Socorro probably are unusable, as I don't expect everyone to have that data about their users/installations.

Comment 5

[[:jberkus] Josh Berkus]

Just for the record:  tcbs is not dependant on ADU.  The views which are dependant on ADU are:

- home_page_graph
- crashes_by_user
- explosiveness

This is not a change from pre-mobeta.  The *way* in which they are dependant has changed (i.e. you need to rerun the matview generation, rather than just rerunning product_adu), but those screens were always missing data if ADU didn't run.

This is also not a change for the open source users.  My suggested solution would be an ADU generator cron job for the OSS users, which just pads out the ADU with predefined figures.

Comment 6

[Selena Deckelmann :selenamarie :selena]

r? https://github.com/mozilla/socorro/pull/923/

Comment 7

[Selena Deckelmann :selenamarie :selena]

Planning to chat with Peter this afternoon about this.

Comment 8

[Selena Deckelmann :selenamarie :selena]

This ticket was made obsolete by crontabber.