Closed
Bug 1439019
Opened 7 years ago
Closed 7 years ago
write script to compare data in two environments
Categories
(Socorro :: General, task, P1)
Socorro
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: willkg, Assigned: willkg)
References
Details
We're building a new infrastructure and Socorro is very complex with non-trivial and very stateful data flows. One of the things we ask ourselves is, "Is the new infrastructure producing the same data as the old one?"
This question is hard to ask specifically since there's a lot of data involved and it has shapes and timeliness factors. It's also really hard to answer for exactly the same reasons.
This bug covers writing a script that uses existing APIs in the webapp to compare data generated by two environments and show where they differ.
Assignee | ||
Comment 1•7 years ago
|
||
Making this a P1. I mentioned this to Miles and he said he'd like something like this yesterday. This is a pretty critical time to have something like this and it's probably the case that anything we put together will help.
Grabbing this to work on next week.
Assignee: nobody → willkg
Status: NEW → ASSIGNED
Priority: -- → P1
Assignee | ||
Comment 2•7 years ago
|
||
Oops--blocked the wrong tracker. SO MANY TRACKERS!
Assignee | ||
Comment 3•7 years ago
|
||
I'm (ab)using pytest's diff things and wrote it as pytest tests:
https://github.com/willkg/socorro-compare
That has some tests done. It doesn't have tests for raw and processed crashes which I think we want. It probably needs to be less strict, too. But it's a good start.
Assignee | ||
Comment 4•7 years ago
|
||
I finished a first pass on all the things I was thinking we should compare:
* adi -- tests build_adu data flow
* product versions -- tests ftpscraper based data flow
* supersearch -- tests that the processor is processing the same crashes between two environments
* raw_crash -- tests that raw crashes look the same
* processed_crash -- tests that processed crashes look the same which also tests mdsw configuration and a bunch of other things
I think it's good enough for now. We can hone it going forward.
Marking as FIXED.
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•