740114 - Error in logs from Fix Broken Dumps

Status: RESOLVED WONTFIX

(Socorro :: Backend, task)

Description

[[DEACTIVATED] Adrian Gaudebert]

Looking at syslog logs on sp-admin01 after the 3 release today, I found this:

Mar 28 12:22:03 Socorro Fix Broken Dumps (pid 22032): 2012-03-28 12:22:03,575 ERROR - MainThread - File "/data/socorro/application/socorro/storage/hbaseClient.py", line 180, in guid_to_timestamped_row_id#012    if timestamp[-6] in "-+":
Mar 28 12:22:03 Socorro Fix Broken Dumps (pid 22032): 2012-03-28 12:22:03,575 ERROR - MainThread - BadOoidException: Bad OOID: <type 'exceptions.TypeError'>-'datetime.datetime' object is unsubscriptable

It seems this was happening before. Here is the full stack, dated 3 days before the release:

Mar 25 04:22:01 Socorro Fix Broken Dumps (pid 16098): 2012-03-25 04:22:01,822 ERROR - MainThread - Caught Error: <class 'socorro.storage.hbaseClient.BadOoidException'>
Mar 25 04:22:01 Socorro Fix Broken Dumps (pid 16098): 2012-03-25 04:22:01,822 ERROR - MainThread - Bad OOID: <type 'exceptions.TypeError'>-'datetime.datetime' object is unsubscriptable
Mar 25 04:22:01 Socorro Fix Broken Dumps (pid 16098): 2012-03-25 04:22:01,823 ERROR - MainThread - trace back follows:
Mar 25 04:22:01 Socorro Fix Broken Dumps (pid 16098): 2012-03-25 04:22:01,823 ERROR - MainThread - Traceback (most recent call last):
Mar 25 04:22:01 Socorro Fix Broken Dumps (pid 16098): 2012-03-25 04:22:01,824 ERROR - MainThread - File "/data/socorro/application/socorro/cron/fixBrokenDumps.py", line 58, in fix#012    hbc.put_fixed_dump(ooid, fixed_dump, add_to_unprocessed_queue = True, submitted_timestamp = utc_now())
Mar 25 04:22:01 Socorro Fix Broken Dumps (pid 16098): 2012-03-25 04:22:01,824 ERROR - MainThread - File "/data/socorro/application/socorro/storage/hbaseClient.py", line 144, in f#012    result = fn(self, *args, **kwargs)
Mar 25 04:22:01 Socorro Fix Broken Dumps (pid 16098): 2012-03-25 04:22:01,824 ERROR - MainThread - File "/data/socorro/application/socorro/storage/hbaseClient.py", line 733, in put_fixed_dump#012    self.put_crash_report_indices(ooid,submitted_timestamp,indices)
Mar 25 04:22:01 Socorro Fix Broken Dumps (pid 16098): 2012-03-25 04:22:01,824 ERROR - MainThread - File "/data/socorro/application/socorro/storage/hbaseClient.py", line 144, in f#012    result = fn(self, *args, **kwargs)
Mar 25 04:22:01 Socorro Fix Broken Dumps (pid 16098): 2012-03-25 04:22:01,824 ERROR - MainThread - File "/data/socorro/application/socorro/storage/hbaseClient.py", line 606, in put_crash_report_indices#012    row_id = guid_to_timestamped_row_id(ooid,timestamp)
Mar 25 04:22:01 Socorro Fix Broken Dumps (pid 16098): 2012-03-25 04:22:01,824 ERROR - MainThread - File "/data/socorro/application/socorro/storage/hbaseClient.py", line 68, in f#012    result = fn(*args, **kwargs)
Mar 25 04:22:01 Socorro Fix Broken Dumps (pid 16098): 2012-03-25 04:22:01,825 ERROR - MainThread - File "/data/socorro/application/socorro/storage/hbaseClient.py", line 180, in guid_to_timestamped_row_id#012    if timestamp[-6] in "-+":
Mar 25 04:22:01 Socorro Fix Broken Dumps (pid 16098): 2012-03-25 04:22:01,825 ERROR - MainThread - BadOoidException: Bad OOID: <type 'exceptions.TypeError'>-'datetime.datetime' object is unsubscriptable

Comment 1

[Robert Helmer [:rhelmer]]

The broken dump problem is for such an old version of Fennec we should probably just disable it at this point. Nothing changed in the code directly so I'd guess it's one of the dependent socorro libs.