stage socorro new db table and cron job

RESOLVED FIXED

Status

mozilla.org Graveyard
Server Operations
--
major
RESOLVED FIXED
8 years ago
3 years ago

People

(Reporter: ozten, Assigned: aravind)

Tracking

Details

(Reporter)

Description

8 years ago
1) Please svn up scripts and socorro in stage.

2) Please create the new db table 'daily_crashes'. I think this is done via
scripts/setupDatabase.py

@aravind or @lars - please correct me

3) Run the new cron manually
scripts/startDailyCrash.py


Once this is finished, I will commit the frontend code.

I will also give you instructions for installing the cron later. We should run it once manually to backfill the tables, which may take several hours.
(Reporter)

Comment 1

8 years ago
(In reply to comment #0)
Instead of step #2, please execute this SQL:

CREATE TABLE daily_crashes
    (
        id serial NOT NULL PRIMARY KEY,
        COUNT INTEGER DEFAULT 0 NOT NULL,
        report_type CHAR(1) NOT NULL DEFAULT 'C',
        productdims_id INTEGER REFERENCES productdims(id),
        os_short_name CHAR(3),
        adu_day TIMESTAMP WITHOUT TIME ZONE NOT NULL,
        CONSTRAINT day_product_os_report_type_unique UNIQUE (adu_day, productdims_id, os_short_name, report_type)
    )

Updated

8 years ago
Assignee: server-ops → shyam
(In reply to comment #0)
> 1) Please svn up scripts and socorro in stage.

[root@mrapp-stage02 scripts]# svn up
A    startDailyCrash.py
U    config/processorconfig.py.dist
A    config/dailycrashconfig.py.dist
U    config/monitorconfig.py.dist
Updated to revision 2105.
[root@mrapp-stage02 scripts]# cd ../socorro/
[root@mrapp-stage02 socorro]# svn up
U    unittest/database/testSchema.py
U    unittest/collector/testCrashstorage.py
U    processor/processor.py
A    cron/daily_crash.py
U    hbase/hbaseClient.py
U    services/getCrash.py
U    services/aduByDay.py
U    database/schema.py
U    lib/psycopghelper.py
U    webapi/webapiService.py
U    monitor/monitor.py
Updated to revision 2105.


> 2) Please create the new db table 'daily_crashes'. I think this is done via

Don't think this really succeeded, suggestions? 

breakpad=# CREATE TABLE daily_crashes
breakpad-#     (
breakpad(#         id serial NOT NULL PRIMARY KEY,
breakpad(#         COUNT INTEGER DEFAULT 0 NOT NULL,
breakpad(#         report_type CHAR(1) NOT NULL DEFAULT 'C',
breakpad(#         productdims_id INTEGER REFERENCES productdims(id),
breakpad(#         os_short_name CHAR(3),
breakpad(#         adu_day TIMESTAMP WITHOUT TIME ZONE NOT NULL,
breakpad(#         CONSTRAINT day_product_os_report_type_unique UNIQUE (adu_day,
breakpad(# productdims_id, os_short_name, report_type)
breakpad(#     );
NOTICE:  CREATE TABLE will create implicit sequence "daily_crashes_id_seq" for serial column "daily_crashes.id"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "daily_crashes_pkey" for table "daily_crashes"
NOTICE:  CREATE TABLE / UNIQUE will create implicit index "day_product_os_report_type_unique" for table "daily_crashes"
CREATE TABLE
(Reporter)

Comment 3

8 years ago
Thanks fox2mike. Those notices look normal.
Ah okay, let me run the rest.
(Reporter)

Comment 5

8 years ago
Missed a step
3.1) cp config/dailycrashconfig.py.dist config/dailycrashconfig.py
3.2) Make sure you have the same env as a cron job (PYTHONPATH, etc)
I get this :

[processor@dm-breakpad-stage01 scripts]$ ./startDailyCrash.py 
Traceback (most recent call last):
  File "./startDailyCrash.py", line 10, in ?
    import dailycrashconfig as config
ImportError: No module named dailycrashconfig
(Assignee)

Comment 7

8 years ago
Ran the script and posted the output to khan.
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
(Reporter)

Comment 8

8 years ago
The log is filled with ERROR - permission denied for relation daily_crashes.

I'm guessing there are now rows in the new table.

aravind - Can you look at the username/password and grant permissions? thanks.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 9

8 years ago
Dropped and re-created the table.  Uploaded the log file to khan:/tmp/dailycrash.log
Status: REOPENED → RESOLVED
Last Resolved: 8 years ago8 years ago
Resolution: --- → FIXED
(Reporter)

Comment 10

8 years ago
Step 4 - restart the webservices.py code to pickup new AduByDay.py changes.
Step 5 - clear frontend web service call caches. This is probably memcached.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
aravind: I'm trying to work out why fox2mike had trouble staging this, sounds like something is missing from the docs.  Could you guys please work together to sort it out?
(Assignee)

Comment 12

8 years ago
(In reply to comment #11)
> aravind: I'm trying to work out why fox2mike had trouble staging this, sounds
> like something is missing from the docs.  Could you guys please work together
> to sort it out?

My guess is that (from reading comment #1) is that only the scripts and processor layer were updated.  I guess the fix is to specifically say that you want all layers updated.

@ozten: Working on pushing the code out to the middleware layer.
(Assignee)

Comment 13

8 years ago
middleware code updated.
Status: REOPENED → RESOLVED
Last Resolved: 8 years ago8 years ago
Resolution: --- → FIXED
(Reporter)

Comment 14

8 years ago
Step 3.3) Please update the webapiconfig.py and add

adubd.AduByDay200912 to the servicesList.default 

So it will look something like:
servicesList.default = [tcbst.TopCrashBySignatureTrends, sighist.SignatureHistory, adubd.AduByDay, adubd.AduByDay200912, crash.GetCrash]
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 15

8 years ago
Updated the config file - getting this error now http://breakpad.pastebin.mozilla.org/728540
Assignee: shyam → aravind
(Reporter)

Comment 16

8 years ago
(In reply to comment #15)
Filed Bug#568322.
(Assignee)

Comment 17

8 years ago
We know now that the middle-ware layer is functioning now, closing the bug.
Status: REOPENED → RESOLVED
Last Resolved: 8 years ago8 years ago
Resolution: --- → FIXED
(Reporter)

Comment 18

8 years ago
@aravind - Since this is not installed as a cron yet, can you run this script again:
./startDailyCrash.py

This will put data in for the last couple of days.

Thanks
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 19

8 years ago
Done.
Status: REOPENED → RESOLVED
Last Resolved: 8 years ago8 years ago
Resolution: --- → FIXED
(Reporter)

Comment 20

8 years ago
(In reply to comment #0)
This is looking good. Please re-run 
./startDailyCrash.py

Then also please do
Step 6) Install ./startDailyCrash.py as a daily cron job.
It should run after midnight, once per day.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 21

8 years ago
Done, set cron job to run nightly at 00:05
Status: REOPENED → RESOLVED
Last Resolved: 8 years ago8 years ago
Resolution: --- → FIXED
(Reporter)

Comment 22

8 years ago
Notes for prod deployment:

Step 7) Ensure that webservices.py has read access to the daily_crashes table.
7.1) Restart apache/webservices if needed to pick up new source code (needed if mod_python)
7.2) curl "http://localhost:8085/201005/adu/byday/p/Firefox/v/3.6.4/rt/any/os/Windows;Mac;Linux/start/2010-05-22/end/2010-06-06"
Grep webapi.log for fatal, error, or warnings
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.