Closed
Bug 1040365
Opened 11 years ago
Closed 11 years ago
ceph processor failing ("this crash cannot be found in raw crash storage")
Categories
(Socorro :: Backend, task)
Socorro
Backend
Tracking
(Not tracked)
RESOLVED
FIXED
95
People
(Reporter: rhelmer, Unassigned)
References
Details
Ceph crash storage is failing in processor with:
2014-07-17 17:50:04,020 WARNING - Thread-1 - 35deb152-04af-4b57-ba4f-8233a2140717 rejected: this crash cannot be found in raw crash storage
If I dig into this, I see the problem is when get_raw_crash()->do_get_raw_crash()->boto_s3_store._fetch_from_boto_s3(), if I put debugging in https://github.com/mozilla/socorro/blob/master/socorro/external/ceph/crashstorage.py#L360 like:
self.logger.debug('%s %s %s' % (the_day_bucket_name, bucket, key))
The result is:
2014-07-17 17:54:51,418 DEBUG - Thread-2 - 140717 <Bucket: 140717> e391429c-9be0-4927-a6f3-d25002140717.raw_crash
2014-07-17 17:54:51,933 DEBUG - Thread-2 - 140717 <Bucket: 140717> e391429c-9be0-4927-a6f3-d25002140717.dump_names
2014-07-17 17:54:51,955 DEBUG - Thread-2 - 140717 <Bucket: 140717> e391429c-9be0-4927-a6f3-d25002140717.upload_file_minidump
The last one is where it is failing. When I look in S3, I see that collector has stored using these filenames:
e391429c-9be0-4927-a6f3-d25002140717.raw_crash
e391429c-9be0-4927-a6f3-d25002140717.dump_names
e391429c-9be0-4927-a6f3-d25002140717.dump
If I look in the .dump_names file, I see:
["upload_file_minidump"]
So - there's a disagreement somewhere about what the dump file should be named.
Comment 1•11 years ago
|
||
this is one of those the-name-changes-as-it-moves-through-the-system because of historical glitches. see https://github.com/mozilla/socorro/blob/master/socorro/external/hb/crashstorage.py#L167 and https://github.com/mozilla/socorro/blob/master/socorro/external/hb/crashstorage.py#L367
BotoS3 needs the same translation logic that HBase is using... PR on the way...
Comment 2•11 years ago
|
||
Commit pushed to master at https://github.com/mozilla/socorro
https://github.com/mozilla/socorro/commit/8be62766433e85b34fbc0a3b32abfed2c5907689
Merge pull request #2222 from twobraids/dump-name
Fixes Bug 1040365 - added dump name mangling & test
Updated•11 years ago
|
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Updated•11 years ago
|
Target Milestone: --- → 95
You need to log in
before you can comment on or make changes to this bug.
Description
•