Run Socorro backfill for June 20 and June 21

VERIFIED FIXED

Status

VERIFIED FIXED
5 years ago
4 years ago

People

(Reporter: laura, Assigned: mpressman)

Tracking

Details

(Reporter)

Description

5 years ago
We got late ADU pushes from Metrics these days, and failed to auto-recover.  Please run backfill!
(Assignee)

Updated

5 years ago
Assignee: server-ops-database → mpressman
(Assignee)

Comment 1

5 years ago
Kicked it off, running backfill now
(Assignee)

Comment 2

5 years ago
Backfill completed successfully:
breakpad=# select backfill_matviews('2013-06-21', '2013-06-22');
INFO:  backfilling reports_clean
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-21 00:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-21 06:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-21 12:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-21 18:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-22 00:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-22 06:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-22 12:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-22 18:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling other matviews for 2013-06-21
INFO:  adu
INFO:  signatures
INFO:  tcbs
NOTICE:  table "new_tcbs" does not exist, skipping
CONTEXT:  SQL statement "DROP TABLE IF EXISTS new_tcbs"
PL/pgSQL function backfill_matviews(date,date,boolean) line 67 at SQL statement
INFO:  crashes by user
INFO:  home page graph
INFO:  hang report (slow)
INFO:  nightly builds
INFO:  backfilling other matviews for 2013-06-22
INFO:  adu
INFO:  signatures
INFO:  tcbs
NOTICE:  table "new_tcbs" does not exist, skipping
CONTEXT:  SQL statement "DROP TABLE IF EXISTS new_tcbs"
PL/pgSQL function backfill_matviews(date,date,boolean) line 67 at SQL statement
INFO:  crashes by user
INFO:  home page graph
INFO:  hang report (slow)
INFO:  nightly builds
INFO:  rank_compare
INFO:  explosiveness (slow)
INFO:  correlations
 backfill_matviews
-------------------
 t
(1 row)

Time: 7934528.605 ms
breakpad=#
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED

Comment 3

5 years ago
Hmm, in a fast check, it looks like the 20th worked out but 21st may not have, I think we still don't have ADI for that and need metrics to push that first :(
Perhaps it is time to reconsider bug 793885....
Requested an ADU/ADI push from mreid and now am re-running 6/21

Comment 6

5 years ago
Now I'm confused, as my own reports had 6/20 display fine but https://crash-stats-django.mozilla.org/daily?p=Firefox is now showing 6/21 OK but 6/20 missing, what's up there?

Comment 7

5 years ago
(In reply to Matt Pressman [:mpressman] from comment #2)
> breakpad=# select backfill_matviews('2013-06-21', '2013-06-22');

Umm, that explains why 6/20 is still missing - it was 20th and 21st that should have been backfilled, not 21st and 22nd. So, 20th still needs to be backfilled.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #7)
> (In reply to Matt Pressman [:mpressman] from comment #2)
> > breakpad=# select backfill_matviews('2013-06-21', '2013-06-22');
> 
> Umm, that explains why 6/20 is still missing - it was 20th and 21st that
> should have been backfilled, not 21st and 22nd. So, 20th still needs to be
> backfilled.

Doh!  Running it now.
Started...
breakpad=# select backfill_matviews('2013-06-20', '2013-06-20');
Summary: Run Socorro backfill for June 21 and June 22 → Run Socorro backfill for June 20 and June 21
breakpad=# select backfill_matviews('2013-06-20', '2013-06-20');
INFO:  backfilling reports_clean
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-20 00:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-20 06:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-20 12:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-06-20 18:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean) line 53 at PERFORM
INFO:  backfilling other matviews for 2013-06-20
INFO:  adu
INFO:  signatures
INFO:  tcbs
NOTICE:  table "new_tcbs" does not exist, skipping
CONTEXT:  SQL statement "DROP TABLE IF EXISTS new_tcbs"
PL/pgSQL function backfill_matviews(date,date,boolean) line 67 at SQL statement
INFO:  crashes by user
INFO:  home page graph
INFO:  hang report (slow)
INFO:  nightly builds
INFO:  rank_compare
INFO:  explosiveness (slow)
INFO:  correlations
 backfill_matviews 
-------------------
 t
(1 row)

Time: 4083964.683 ms
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → FIXED

Comment 11

5 years ago
(In reply to Selena Deckelmann :selenamarie :selena from comment #8)
> Doh!  Running it now.

Thanks, Laura mistakenly filed it for the dates on which the jobs were not running correctly, and not the dates for which the data would have been processed on those days.

That said, I now can confirm that the data is correctly available. Thanks a lot!
Status: RESOLVED → VERIFIED
Product: mozilla.org → Data & BI Services Team
You need to log in before you can comment on or make changes to this bug.