Closed Bug 1357246 Opened 5 years ago Closed 3 years ago

move de-nulling to processor

Categories

(Socorro :: Processor, task, P2)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: willkg, Assigned: willkg)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

Antenna, like the Socorro collector, removes null bytes from keys and values in the raw crash data. It's been doing it for a while, so it might be "safe", but my basic problem with this is that I don't think the collector should be modifying the crash at all.

Seems like it makes more sense for the processor to do a de-nulling pass. Then if we discover bugs or oddities, we can fix them in the processor and re-process the affected crashes. Further, the processor already has a notes field where it can log how it changed the crash and what problems it encountered.

This bug covers looking into moving the de-nulling code. Maybe it's something we wait on thinking about until after we've rewritten the processor.
Switching bugs to the Processor component.
Component: Antenna → Processor
I want to do this because it simplifies Antenna a bit.

Making this a P2.
Priority: -- → P2
There are two things that need to be done here:

1. Add a de-nulling transform rule to the processor that takes the raw crash and de-nulls all the things in it.

If we do this now, the processor will save the de-nulled raw crash over the original. I'm not thrilled about that. If I had my druthers, we'd sort out the situation we have now where the processor stomps on raw crash values. It should never touch the raw crash and never save it. So maybe we shouldn't work on this until after we've sorted out the processor stomping on original data.

2. Anyhow, after those code changes land and are pushed to prod, then we can remove the de-nulling step from the collector.

Comment #3 suggests that the processor stomps on the raw crash. The code isn't wildly clear, but I'm pretty sure that's actually not the case.

I'll verify this and then write a processor rule.

Assignee: nobody → willkg
Status: NEW → ASSIGNED

I added metrics for de-nulling and they've been out for a little less than a day. In all that time, de-nulling hasn't had any effect.

So, let's move forward with this and remove de-nulling from Antenna. If there are problems, we have the crash report so we can fix the processor and then reprocess the crash.

This went out just now in Antenna 27. We're no longer de-nulling in the collector. Yay!

Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.