FakeData incorrectly sets a date for processed

RESOLVED FIXED in 57

Status

Socorro
General
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: brandon, Assigned: rhelmer)

Tracking

unspecified
x86
Mac OS X

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [qa-])

(Reporter)

Description

5 years ago
I had a crash report that was dated August 21, 2013 but was "processed" on August 14, 2013. This created a problem where the crashes/ API didn't load the crash. We should make the fakedata always have the processed date after the crash date (since it would be in real life).

Comment 1

5 years ago
which is the other date than "processed"? If it's the "crash date" we send with the crash report, things ought to work when it's in the future, as it's taken from the local computer clock, and there's definitely users out there how have skewed clocks.
+1 to what KaiRo says, we have no control over the "client_crash_date", so having weird dates should be expected and should not break Socorro. Afaik, we set "date_processed", "completeddatetime" and "startedDateTime" in the processor, and:

date_processed < startedDateTime < completeddatetime
(Reporter)

Comment 3

5 years ago
Okay, so I explained the problem poorly. Here's what I meant.

The UUID for the crash has a date (last six characters of the OOID) of 130821. However, our internal processing date is 2013-08-14 - supposedly more than a week before the crash occurred.

This only affects fakedata as far as I can tell. But the crash/ API expects that the OOID date is correct, and uses that with the utc_day_is() function to determine if it should pull the crash or not.

It's possible the crashes/ API is broken or has wrong assumptions. I think it's more likely that fakedata just has a bug in it.
(Assignee)

Comment 4

5 years ago
(In reply to Brandon Savage [:brandon] from comment #3)
> Okay, so I explained the problem poorly. Here's what I meant.
> 
> The UUID for the crash has a date (last six characters of the OOID) of
> 130821. However, our internal processing date is 2013-08-14 - supposedly
> more than a week before the crash occurred.
> 
> This only affects fakedata as far as I can tell. But the crash/ API expects
> that the OOID date is correct, and uses that with the utc_day_is() function
> to determine if it should pull the crash or not.
> 
> It's possible the crashes/ API is broken or has wrong assumptions. I think
> it's more likely that fakedata just has a bug in it.

Hrm so looking at the code it looks like both date_processed and the crashid generation code uses the same input (a datetime object named "timestamp").

I'll build a local fakedata db and take a peek.
(Assignee)

Comment 5

5 years ago
(In reply to Brandon Savage [:brandon] from comment #0)
> I had a crash report that was dated August 21, 2013 but was "processed" on
> August 14, 2013. This created a problem where the crashes/ API didn't load
> the crash. We should make the fakedata always have the processed date after
> the crash date (since it would be in real life).

So to clarify, are you talking about the "date_processed" column in the reports table? It's actually a misnomer, that's the collection time (the collectors set it)

Comment 6

5 years ago
(In reply to Brandon Savage [:brandon] from comment #3)
> The UUID for the crash has a date (last six characters of the OOID) of
> 130821. However, our internal processing date is 2013-08-14 - supposedly
> more than a week before the crash occurred.

OK, that's surely an error. "date_processed" is the collection time, as rhelmer says, and the date part of the crash ID (can we please get rid of the "OOID" moniker?) is taken from the collection date as well. Those never should differ. Sounds definitely like a bug in fakedata.
(Assignee)

Comment 7

5 years ago
Yeah I think this is a bug in fakedata.py, I suspect it's using the very last timestamp (derived from the time the script was started) for the crash id (uuid column) instead of the same that is used for the date_processed on that row:

fakedata=# select uuid,date_processed from reports limit 10;
                 uuid                 |        date_processed         
--------------------------------------+-------------------------------
 60597bdc-5dbe-4409-6b38-4309c0130828 | 2013-08-13 06:16:04.630337+00
 d1ca4dc4-edfd-e416-3eff-8b3fc0130828 | 2013-08-13 06:16:04.630337+00
 5e0730b3-cc17-0c31-c7ee-c61bb0130828 | 2013-08-13 06:16:04.630337+00
 b3de08f9-ec98-3704-4692-ba0350130828 | 2013-08-13 06:16:40.630337+00
 7bd9e8a1-ff29-7d0e-4f2e-84fcb0130828 | 2013-08-13 06:16:40.630337+00
 400db00d-d388-1a50-5805-6ed090130828 | 2013-08-13 06:16:40.630337+00
 21fdac3a-f325-ffbe-9065-6c8fb0130828 | 2013-08-13 06:17:16.630337+00
 ec08693c-7401-f5ce-2425-d75a40130828 | 2013-08-13 06:17:16.630337+00
 78573797-4a80-7546-4627-314170130828 | 2013-08-13 06:17:16.630337+00
 7deaab64-16d1-b5ac-2e1c-5d3a50130828 | 2013-08-13 06:17:52.630337+00
(10 rows)
Status: NEW → ASSIGNED
(Assignee)

Comment 8

5 years ago
r? https://github.com/mozilla/socorro/pull/1448
Flags: needinfo?(bsavage)

Comment 9

5 years ago
Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/9cfe503c2cc1d9fd54c37dabf6fba947d9b0eada
fixes bug 909469 - date_processed and crash id (uuid) timestamps should match

https://github.com/mozilla/socorro/commit/afe16d2919f1533b0198a3a08670aee3a3069dea
Merge pull request #1448 from rhelmer/bug909469-fakedata-uuid-date_processed-discrepency

fixes bug 909469 - date_processed and crash id (uuid) timestamps should ...

Updated

5 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Whiteboard: [qa-]
Target Milestone: --- → 57
(Reporter)

Updated

5 years ago
Flags: needinfo?(bsavage)
You need to log in before you can comment on or make changes to this bug.