893728 - Backfill missing Socorro aggregation data for 2013-07-13

Status: RESOLVED FIXED

(Data & BI Services Team :: DB: MySQL, task)

Description

[Robert Kaiser]

I think bug 880378 didn't take effect yet with the late ADI push yesterday (I think we are running 52 now and that bug says 53 as the TM), so we need to backfill one more time.

We did get raw_adu for the 13th, but we received them late:

breakpad=> SELECT received_at FROM raw_adu WHERE date='2013-07-13' GROUP BY received_at;
         received_at          
------------------------------
 2013-07-14 18:47:59.73879+00
(1 row)


So, please backfill Socorro matviews for 2013-07-13.

Comment 1

[Sheeri Cabral [:sheeri]]

This is odd:

[[e-mail]1 ~]$ psql breakpad
Timing is on.
psql (9.2.4)
Type "help" for help.

breakpad=# select backfill_matviews('2013-07-13','2013-07-13');
ERROR:  function backfill_matviews(unknown, unknown) is not unique
LINE 1: select backfill_matviews('2013-07-13','2013-07-13');
               ^
HINT:  Could not choose a best candidate function. You might need to add explicit type casts.
Time: 1.154 ms

### OK, trying to explicitly cast things:

breakpad=# select backfill_matviews(to_date('2013-07-13','YYYY-MM-DD'),to_date('2013-07-13','YYYY-MM-DD'));
ERROR:  function backfill_matviews(date, date) is not unique
LINE 1: select backfill_matviews(to_date('2013-07-13','YYYY-MM-DD'),...
               ^
HINT:  Could not choose a best candidate function. You might need to add explicit type casts.
Time: 0.375 ms


Hrm, there *are* two different functions:

breakpad=#                 select proname,pronamespace,proowner,prolang,procost, prorows,  provariadic , protransform,  proisagg , proiswindow , prosecdef , proleakproof , proisstrict , proretset , provolatile , pronargs , pronargdefaults, prorettype , proargtypes  , proallargtypes , proargnames from pg_proc where proname='backfill_matviews';      proname      | pronamespace | proowner | prolang | procost | prorows | provariadic | protransform | proisagg | proiswindow | prosecdef | proleakproof | proisstrict | proretset | provolatile | pronargs | pronargdefaults | prorettype |    proargtypes    | proallargtypes |                 proargnames                  
-------------------+--------------+----------+---------+---------+---------+-------------+-------
-------+----------+-------------+-----------+--------------+-------------+-----------+-----------
--+----------+-----------------+------------+-------------------+----------------+---------------
-------------------------------
 backfill_matviews |         2200 |       10 |   12599 |     100 |       0 |           0 | -     
       | f        | f           | f         | f            | f           | f         | v         
  |        3 |               2 |         16 | 1082 1082 16      |                | {firstday,last
day,reportsclean}
 backfill_matviews |         2200 |       10 |   12599 |     100 |       0 |           0 | -     
       | f        | f           | f         | f            | f           | f         | v         
  |        4 |               3 |         16 | 1082 1082 16 1186 |                | {firstday,last
day,reportsclean,check_period}
(2 rows)

Hrm, it seems like there are 2 functions with the same name, with different numbers of options - either 2 or 3. Trying to use the 1st function with 3 arguments:

breakpad=# select backfill_matviews('2013-07-13','2013-07-13',false);
ERROR:  function backfill_matviews(unknown, unknown, boolean) is not unique
LINE 1: select backfill_matviews('2013-07-13','2013-07-13',false);
               ^HINT:  Could not choose a best candidate function. You might need to add explicit type casts.
Time: 0.435 
msbreakpad=# 

I'm stumped.

Still doesn't work.

Comment 2

[Ashish Vijayaram [:ashish]]

Assigning to :sheeri since she's working on this (and stop paging oncall).

Comment 3

[Sheeri Cabral [:sheeri]]

mpressman: got it - both the backfill_matviews argument forms are ambiguous because of the default value
[1:28pm] mpressman: if run explicitly using the argument form of firstday date, lastday date DEFAULT NULL::date, reportsclean boolean DEFAULT true, check_period interval DEFAULT '01:00:00'::interval it works
[1:28pm] sheeri: ah, so you have to change the default value in the script?
[1:28pm] sheeri: damn, I even tried casting strings explicitly as dates
[1:28pm] mpressman: basically put in the values that are defaulted to in the function argument list
[1:29pm] mpressman: I saw that and immediately thought that was the answer to until I did a \dC
[1:29pm] mpressman: *too
[1:29pm] mpressman: so, this is what finally is working: breakpad=# select backfill_matviews('2013-07-13', '2013-07-13', true, '01:00:00'::interval);

Comment 4

[Matt Pressman [:mpressman]]

the backfill has completed for 2013-07-13

breakpad=# select backfill_matviews('2013-07-13', '2013-07-13', true, '01:00:00'::interval);
INFO:  backfilling reports_clean
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-07-13 00:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean,interval) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-07-13 06:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean,interval) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-07-13 12:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean,interval) line 53 at PERFORM
INFO:  backfilling 06:00:00 of reports_clean starting at 2013-07-13 18:00:00+00
CONTEXT:  SQL statement "SELECT backfill_reports_clean( first_rc, last_rc )"
PL/pgSQL function backfill_matviews(date,date,boolean,interval) line 53 at PERFORM
INFO:  backfilling other matviews for 2013-07-13
INFO:  adu
INFO:  signatures
INFO:  tcbs
NOTICE:  table "new_tcbs" does not exist, skipping
CONTEXT:  SQL statement "DROP TABLE IF EXISTS new_tcbs"
PL/pgSQL function backfill_matviews(date,date,boolean,interval) line 67 at SQL statement
INFO:  crashes by user
INFO:  home page graph
INFO:  hang report (slow)
INFO:  nightly builds
INFO:  rank_compare
INFO:  explosiveness (slow)
INFO:  correlations
 backfill_matviews
-------------------
 t
(1 row)

Time: 4505093.454 ms