Closed Bug 1403339 Opened 8 years ago Closed 6 years ago

add data caching to signature generation cli

Categories

(Socorro :: General, task, P3)

Product:
Socorro
Component:
General
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: willkg, Unassigned)

Details

When working on bug #1402037, I was doing signature generation analysis on 16,000 crashes. That's fine except that it takes *HOURS* to run and any changes in signature generation take *HOURS* to test. The bulk of the time being used is the HTTP requests to fetch the raw and processed crash data. This data wasn't changing at all between my signature generation analysis attempts. I claim that any signature generation project would benefit from caching this data to make iterations *MUCH* faster. I don't think we want caching for all signature generation cli usage--just in certain scenarios. I think it should be disabled by default. This bug covers adding opt-in caching to the signature generation cli.
It'd be nice to specify the place to cache the data. That way you can associate the cache with some project and delete it when you're done. Maybe something like this: python -m socorro.signature --use-cache ./1402037-cache That would enable caching for the HTTP responses and store the cached versions in that specific directory. You could delete the directory when you're done or when you need updated data. Pretty sure this would cover my needs. Maybe since this is just caching HTTP responses, it should be --use-http-cache ?
Making this a P2. This would have sped up work on bug #1402037 significantly and I see more signature generation analysis in my future.
Priority: -- → P2
Changing this bug to be a P3. It's interesting, but probably not something we have time for. If there's a compelling reason why this has urgency and should get done now, please let us know in a comment!
Priority: P2 → P3
This idea is super, but I've lived without it for a while and I'm not going to do it.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.