619825 - Add retry for Postgres connections to processors

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Laura Thomson :laura]

Reuse the progressive back-off code from HBase.

Comment 1

[K Lars Lohn [:lars] [:klohn]]

this was done in r2940.

added backing off retry to all database transactions.
  This ended up being far more involved than originally thought
  and touched way more than just the processor code.  I had to
  entirely rewrite the processor unit tests because retrofitting
  would have taken more than a week.  At the same time, I
  cleaned up processor logging to be more minimal.
  This also included a minor enhancement to the unit test
  'expectation' class.
  Also added some routines to the database utility class to
  implement the backing off retry in a generic manner.
  Another side effect was to rewrite the plugin processing
  code.

Comment 2

[Robert Helmer [:rhelmer]]

These changes make us explicitly depend on python2.6, which is fine but our older staging environment (currently being replaced) still defaults to python2.4 in a few places.

We're going to go ahead and set python2.6 in cron jobs, init scripts and move collector from mod_python to mod_wsgi (since it'll be ~1 week until new staging is up).

Filed bug 635393 to followup on that.