As a security precaution, we have turned on the setting "Require API key authentication for API requests" for everyone. If this has broken something, please contact bugzilla-admin@mozilla.org
Last Comment Bug 805402 - ADU and aggregation data missing for 2012-10-24
: ADU and aggregation data missing for 2012-10-24
Status: RESOLVED FIXED
:
Product: Data & BI Services Team
Classification: Other
Component: DB: MySQL (show other bugs)
: other
: All All
: -- normal
: ---
Assigned To: server-ops-database
: Corey Shields [:cshields]
:
Mentors:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-10-25 06:00 PDT by Robert Kaiser
Modified: 2014-10-17 12:46 PDT (History)
6 users (show)
See Also:
Due Date:
Mozilla Project: ---
QA Whiteboard:
Iteration: ---
Points: ---
Cab Review: ServiceNow Change Request (use flag)


Attachments

Description User image Robert Kaiser 2012-10-25 06:00:02 PDT
I saw today that once again we are missing aggregated (graph etc.) data for yesterday.

Before we can backfill, we need to get ADU data, though, as that seems to still be missing:

breakpad=> SELECT max(date) FROM raw_adu;
    max     
------------
 2012-10-23
(1 row)
Comment 1 User image Daniel Einspanjer [:dre] [:deinspanjer] 2012-10-25 06:52:44 PDT
There was an error with the EOD process.  We fixed it just a bit ago and the EOD succeeded and the exports are running now.  They should be done within 30 minutes.
Comment 2 User image Daniel Einspanjer [:dre] [:deinspanjer] 2012-10-25 07:30:18 PDT
The data was successfully exported to production.  Staging should be finished in a few moments as well.
Comment 3 User image Laura Thomson :laura 2012-10-25 07:31:26 PDT
These should be in now (ADUs were late form Metrics)

Bumping over to DB ops for backfill for yesterday.
Comment 4 User image Sheeri Cabral [:sheeri] 2012-10-25 07:34:58 PDT
backfill in progress:

breakpad=# select backfill_adu('2012-10-24'); backfill_adu 
--------------
 t
(1 row)

breakpad=# select backfill_matviews('2012-10-24');
INFO:  backfilling reports_clean
INFO:  backfilling 06:00:00 of reports_clean starting at 2012-10-24 00:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function "backfill_matviews" line 51 at PERFORM
Comment 5 User image Sheeri Cabral [:sheeri] 2012-10-25 08:30:14 PDT
Backfill complete.
Comment 6 User image Sheeri Cabral [:sheeri] 2012-10-25 09:18:52 PDT
forgot to restart the memcache webheads but at this point they should be good.
Comment 7 User image Matt Pressman [:mpressman] 2012-10-25 09:38:20 PDT
running backfill again to account for issue where two dates are now required, even for just one day's run
Comment 8 User image Robert Kaiser 2012-10-25 09:57:52 PDT
Reopening for comment #7 - Matt, can you close the bug again when that's finished and memcache has been restarted?
Comment 9 User image Robert Kaiser 2012-10-25 13:14:13 PDT
The backfill run by mpressman was successful, thanks.

Note You need to log in before you can comment on or make changes to this bug.