729097 - Rewrite versions that are on the esr channel to have esr in the version number

Status: VERIFIED FIXED

(Socorro :: Backend, task)

Description

[Laura Thomson :laura]

For version n channel esr, rewrite version to nesr.

Example: receive raw crash for Fennec, version '10.0.3', channel 'esr'.
In the processed crash and into the database, write the version string as '10.0.3esr'

This should be a configurable rule (loaded from the database?) much like rewriting Fennec to FennecAndroid.

Comment 1

[K Lars Lohn [:lars] [:klohn]]

questions:

in the example, 'esr' as the channel is the sole trigger, correct?  No matter what the product, if the channel is 'esr', the version gets rewritten as version + 'esr'

in making this rule configurable, should it be dedicated to just rewriting version numbers based on channel or do you want a more flexible system?  Do you want the ability change one or more arbitrary json fields based on some predicate defined in terms of the json fields?

If the system is to be dedicated to changing the version string based on just the channel, it hardly seems worth encoding into a database table.  How often are we going to add new channels that need to be encoded into the version?

If the system needs to do arbitrarily complex transforms, rather than inventing a new language to express them, I would (somewhat hesitatingly) suggest putting python code into a database table.

  if j.Channel == 'esr':
      j.Version = j.Version + 'esr'

these rules could be loaded from the database, eval'd in an appropriate context and then executed when needed.  This allows the greatest flexibility as rules could be added/modified at run time without restarting the processor.  Great care would have to be taken to make sure this doesn't become an attack vector.

Alternatively, a safer but less flexible method would be to encode the rules in a python module and have the database table just name the rules to be used.  New rules would require recoding and restarting the processor.  

Both of these proposals could encompass the esr rule as well as the existing ProductName rewrite.

Suggestions as to direction?

Comment 2

[Laura Thomson :laura]

To answer your first question, yes, this is correct:
"in the example, 'esr' as the channel is the sole trigger, correct?  No matter what the product, if the channel is 'esr', the version gets rewritten as version + 'esr'"

Re the implementation, here are a couple of considerations I think are important:
1.  Whatever we do should cause the least amount of pain for non-Mozilla users
2. I really don't want to put Python code in the database. That way leads to pain.

The only reason I suggested the database was for consistency with the way Fennec/FennecAndroid is handled.

I don't think the system has to be amazingly flexible and arbitrary, but I'd prefer it be somewhere in the middle ground between "completely flexible and horrifyingly complicated" and "a one-time hack".

Comment 3

[K Lars Lohn [:lars] [:klohn]]

see http://www.twobraids.com/2012/02/socorro-rule-system.html for a missive on how I'm implementing the solution to this problem.

Comment 4

[K Lars Lohn [:lars] [:klohn]]

Attachment: tarball containing a test crash that exercises the esr and product rename change

When processed by Socorro, this crash should display a version of '8.0a1esr' and have a Product of 'FennecAndroid'.

Comment 5

[K Lars Lohn [:lars] [:klohn]]

Pull Req 390 submitted: Implemented using a rule based system outlined in this blog post: http://www.twobraids.com/2012/02/socorro-rule-system.html Reviewers ought to read that first to understand what's going on.

Part of the objective was to make it so that the rules could be loaded from the database. While this is implemented, if something were to go wrong (such as Bug 731000 not getting implemented for this deadline), the rule system will fall back to hardcoded rules that accomplish the 'ESR' and 'ProductName' json rewrites.

Comment 6

[K Lars Lohn [:lars] [:klohn]]

Pull Req 390 was withdrawn due to the inadvertent inclusion of some unrelated changes.  The new pull req is 393.

Comment 7

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/bec9210587005e9067345c303d98f86057b1ec06
Merge pull request #393 from twobraids/bug-729097-esr-version

Fixes bug 729097 - Rewrite versions that are on the esr channel to have esr in the version number

Comment 8

[K Lars Lohn [:lars] [:klohn]]

why is this [qa-]?  This should be tested by QA.  Comment #4 outlines how to test.

Comment 9

[K Lars Lohn [:lars] [:klohn]]

these ooids were posted to staging at 10:45 (or so) on 2012-03-07

CrashID=bp-efd650f6-240d-43b8-8fcb-47acf2120307
CrashID=bp-02a8133e-1fd7-40e5-99cf-b08792120307
CrashID=bp-5fa919e6-42a4-4c30-b406-0261f2120307
CrashID=bp-985f00d4-c2f5-4ffe-b711-546fa2120307
CrashID=bp-fca6280d-83c9-4800-8798-40f1a2120307
CrashID=bp-2909cb60-1207-44cf-bf28-d82e92120307
CrashID=bp-e1e21eb7-29d7-4401-aee9-c2aa82120307
CrashID=bp-edef29bb-703f-45a7-8bd7-e32602120307
CrashID=bp-865835c5-f226-4ba4-8d6d-a87d12120307
CrashID=bp-95b199b6-4aba-4898-bec1-d6bf62120307

Comment 10

[Matt Brandt [:mbrandt]]

Attachment: qa - verified

QA verified on stage - :lars thank you for getting the crashes submitted to stage - comment 4 and comment 9 -

Crashes that were submitted displayed the expected version of '8.0a1esr' and have a Product of 'FennecAndroid'.