Last Comment Bug 792904 - DB changes to be more resilient (again) against late ADU pushes
: DB changes to be more resilient (again) against late ADU pushes
Product: Socorro
Classification: Server Software
Component: General (show other bugs)
: unspecified
: All All
P1 normal (vote)
: Future
Assigned To: Selena Deckelmann :selenamarie :selena use ni?
Depends on:
  Show dependency treegraph
Reported: 2012-09-20 10:39 PDT by Robert Kaiser
Modified: 2013-01-14 11:30 PST (History)
5 users (show)
See Also:
QA Whiteboard:
Iteration: ---
Points: ---


Description User image Robert Kaiser 2012-09-20 10:39:49 PDT
We once again had a late ADU push from metrics today, and post-mobeta code apparently has a hard time dealing with that.
Bug 792869 has been filed for backfilling this time, but we shouldn't need manual interaction every time this happens, esp. as we know this can happen all the time.

Pre-mobeta we did at least do TCBS and crash numbers and filled in ADU-dependent data based on an hourly job later if it wasn't present. We need to become more flexible like that again - not sure if crontabber or so helps there, I just want to have this tracked so we make sure it's being addressed.
Comment 1 User image Robert Helmer [:rhelmer] 2012-09-20 10:59:36 PDT
Technically TCBS did run, it's just that now the graphs are built differently and depend on ADU.

crontabber is a better long-term solution for this (since it tracks and can retry dependencies like this).
Comment 2 User image [:jberkus] Josh Berkus 2012-09-23 11:06:23 PDT
I'll go further and say that there is no way to fix this persistent issue without something which performs the same functions as crontabber is spec'd to do.
Comment 3 User image Robert Kaiser 2012-09-23 17:22:38 PDT
I have no problem with the solution being crontabber if it comes rather soon, I just want the problem to be tracked and solved. ;-)
Comment 4 User image Robert Kaiser 2012-10-17 06:01:13 PDT
BTW, I'm also pretty sure that not being able to calculate aggregates like TCBS without having ADU means that many third-party installations of Socorro probably are unusable, as I don't expect everyone to have that data about their users/installations.
Comment 5 User image [:jberkus] Josh Berkus 2012-10-17 09:58:08 PDT
Just for the record:  tcbs is not dependant on ADU.  The views which are dependant on ADU are:

- home_page_graph
- crashes_by_user
- explosiveness

This is not a change from pre-mobeta.  The *way* in which they are dependant has changed (i.e. you need to rerun the matview generation, rather than just rerunning product_adu), but those screens were always missing data if ADU didn't run.

This is also not a change for the open source users.  My suggested solution would be an ADU generator cron job for the OSS users, which just pads out the ADU with predefined figures.
Comment 6 User image Selena Deckelmann :selenamarie :selena use ni? 2012-11-07 10:39:36 PST
Comment 7 User image Selena Deckelmann :selenamarie :selena use ni? 2012-11-07 11:14:30 PST
Planning to chat with Peter this afternoon about this.
Comment 8 User image Selena Deckelmann :selenamarie :selena use ni? 2013-01-14 11:30:14 PST
This ticket was made obsolete by crontabber.

Note You need to log in before you can comment on or make changes to this bug.