Closed Bug 886296 Opened 11 years ago Closed 11 years ago

Run Socorro backfill for June 20 and June 21

Categories

(Data & BI Services Team :: DB: MySQL, task)

task
Not set
normal

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: laura, Assigned: mpressman)

Details

We got late ADU pushes from Metrics these days, and failed to auto-recover.  Please run backfill!
Assignee: server-ops-database → mpressman
Kicked it off, running backfill now
Backfill completed successfully:
breakpad=# select backfill_matviews('2013-06-21', '2013-06-22');
INFO:  backfilling reports_clean
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-21 00:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-21 06:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-21 12:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-21 18:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-22 00:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-22 06:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-22 12:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-22 18:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling other matviews for 2013-06-21
INFO:  adu
INFO:  signatures
INFO:  tcbs
NOTICE:  table "new_tcbs" does not exist, skipping
CONTEXT:  SQL statement "DROP TABLE IF EXISTS new_tcbs"
PL/pgSQL function backfill_matviews(date,date,boolean) line 67 at SQL statement
INFO:  crashes by user
INFO:  home page graph
INFO:  hang report (slow)
INFO:  nightly builds
INFO:  backfilling other matviews for 2013-06-22
INFO:  adu
INFO:  signatures
INFO:  tcbs
NOTICE:  table "new_tcbs" does not exist, skipping
CONTEXT:  SQL statement "DROP TABLE IF EXISTS new_tcbs"
PL/pgSQL function backfill_matviews(date,date,boolean) line 67 at SQL statement
INFO:  crashes by user
INFO:  home page graph
INFO:  hang report (slow)
INFO:  nightly builds
INFO:  rank_compare
INFO:  explosiveness (slow)
INFO:  correlations
 backfill_matviews
-------------------
 t
(1 row)

Time: 7934528.605 ms
breakpad=#
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Hmm, in a fast check, it looks like the 20th worked out but 21st may not have, I think we still don't have ADI for that and need metrics to push that first :(
Perhaps it is time to reconsider bug 793885....
Requested an ADU/ADI push from mreid and now am re-running 6/21
Now I'm confused, as my own reports had 6/20 display fine but https://crash-stats-django.mozilla.org/daily?p=Firefox is now showing 6/21 OK but 6/20 missing, what's up there?
(In reply to Matt Pressman [:mpressman] from comment #2)
> breakpad=# select backfill_matviews('2013-06-21', '2013-06-22');

Umm, that explains why 6/20 is still missing - it was 20th and 21st that should have been backfilled, not 21st and 22nd. So, 20th still needs to be backfilled.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #7)
> (In reply to Matt Pressman [:mpressman] from comment #2)
> > breakpad=# select backfill_matviews('2013-06-21', '2013-06-22');
> 
> Umm, that explains why 6/20 is still missing - it was 20th and 21st that
> should have been backfilled, not 21st and 22nd. So, 20th still needs to be
> backfilled.

Doh!  Running it now.
Started...
breakpad=# select backfill_matviews('2013-06-20', '2013-06-20');
Summary: Run Socorro backfill for June 21 and June 22 → Run Socorro backfill for June 20 and June 21
breakpad=# select backfill_matviews('2013-06-20', '2013-06-20');
INFO:  backfilling reports_clean
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-20 00:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-20 06:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-20 12:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-20 18:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling other matviews for 2013-06-20
INFO:  adu
INFO:  signatures
INFO:  tcbs
NOTICE:  table "new_tcbs" does not exist, skipping
CONTEXT:  SQL statement "DROP TABLE IF EXISTS new_tcbs"
PL/pgSQL function backfill_matviews(date,date,boolean) line 67 at SQL statement
INFO:  crashes by user
INFO:  home page graph
INFO:  hang report (slow)
INFO:  nightly builds
INFO:  rank_compare
INFO:  explosiveness (slow)
INFO:  correlations
 backfill_matviews 
-------------------
 t
(1 row)

Time: 4083964.683 ms
Status: REOPENED → RESOLVED
Closed: 11 years ago11 years ago
Resolution: --- → FIXED
(In reply to Selena Deckelmann :selenamarie :selena from comment #8)
> Doh!  Running it now.

Thanks, Laura mistakenly filed it for the dates on which the jobs were not running correctly, and not the dates for which the data would have been processed on those days.

That said, I now can confirm that the data is correctly available. Thanks a lot!
Status: RESOLVED → VERIFIED
Product: mozilla.org → Data & BI Services Team
You need to log in before you can comment on or make changes to this bug.