551703 - addon compat checking info missing from csv files after 2/25

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[chris hofmann]

looks like around 2/25 we stopped getting addon compat checking info in the csv files.  can someone have a look and fix this?


20100224-crashdata.csv
156967 count of reports with unknown
114440 count of reports with checked
4736   count of reports with no info
 431   count of reports with not checked
   1   header info in report -> addons_checked

20100225-crashdata.csv
255374 
   1 addons_checked

20100226-crashdata.csv
145923 
   1 addons_checked

...

Comment 1

[Frank Griswold [:griswolf] [:fgriswold]]

Aravind:
Can you please tell me what revision of dailyUrl.py is used by the startDailyUrl.py script on production? Are you supplying appropriate rawFileMountPoints parameter?

Comment 2

[Aravind Gottipati [:aravind]]

[processor@cm-breakpad02 scripts]$ svn info startDailyUrl.py
Path: startDailyUrl.py
Name: startDailyUrl.py
URL: http://socorro.googlecode.com/svn/trunk/scripts/startDailyUrl.py
Repository Root: http://socorro.googlecode.com/svn
Repository UUID: efcc4bdc-162a-0410-a8af-5b4b46c5376d
Revision: 1733
Node Kind: file
Schedule: normal
Last Changed Author: twobraids
Last Changed Rev: 1363
Last Changed Date: 2009-10-05 10:39:13 -0700 (Mon, 05 Oct 2009)
Text Last Updated: 2009-10-08 22:35:22 -0700 (Thu, 08 Oct 2009)
Checksum: 47cd9c87e00650610d624f54fb420499

[processor@cm-breakpad02 scripts]$


This coincides with the day we switched the nfs stores.  I forgot to mount the necessary filesystems for this script.

Can we run the script for these days again?

Comment 3

[Frank Griswold [:griswolf] [:fgriswold]]

Not appropriate to work prior dates for two reasons:
 1: The script strongly believes it is handling 'today' (yesterday, really).
 2: Betcha the json files are already gone, so a re-run would yield the same empty data (if they are still present, this would be feasible with some effort)

(and of course, one more)
 3: There is a later 'pure database' version waiting in the wings which would in theory be able to go back... if re-written to accept date range: About a man-day of work to write, test. However it depends on a different processor.py to populate the new columns in the database table... and that processor wasn't running then in any case.

Comment 4

[chris hofmann]

not following all this, but I think we could get by without rerunning the script to correct the old files.  this will mostly be of use to diagnose future crashes, so if we just get it working again thats good enough for me.  hope this helps.

Comment 5

[Aravind Gottipati [:aravind]]

I fixed the script that adds these mount points.  Tonights run should work correctly.

Comment 6

[Aravind Gottipati [:aravind]]

Can anyone confirm that the data is all showing up in the reports?

Comment 7

[chris hofmann]

yeah, 3/18 looks good again

20100318-crashdata.csv
238057 checked
89189 unknown
5362 "blank"
 440 not checked

thanks aravind.