1304160 - Upgrading Crypt::OpenPGP exposed the integer overflow error causing mail to queue for certain bmo accounts

Status: NEW

(bugzilla.mozilla.org :: Email Notifications, defect)

Description

[Dylan Hardison [:dylan] (he/him)]

There's a bug on file with Crypt::OpenPGP:

"ElGamal public key encryption can loop forever for too larges values of 'p'"

This was probably patched in the RPM deployed to BMO's infrastructure,
but it was at least more recently reported in https://github.com/btrott/Crypt-OpenPGP/issues/30.

As we know use vendor bundles to supply CPAN modules, the bug began to impact BMO on Tuesday, Sep 20 after the production push. Only a few people with keys with large values of 'p' were impacted, but this was enough to alert that the job queues were full.

A number of actions can and should be taken to prevent this from happening again:

1) We should test the sending of email in CI
2) If possible, we should test the encryption routines on every public key in BMO. For privacy reasons we may not be able to do this in CI, but we could have a script that we run on the admin node every once and a while to ensure encryption works.
3) Efforts should be taken to get the fix in Crypt::OpenPGP commited and released. 
4) While infinite loop bugs are unlikely to happen again, there is room for error reporting improvements to be made to the jobqueue system.

Comment 1

[Dylan Hardison [:dylan] (he/him)]

Note that the email system still performed well -- only two emails were delayed (for some hours) but the rest went out smoothly.

Comment 2

[Dylan Hardison [:dylan] (he/him)]

Status:

carton bundles updated in
To github.com:bugzilla/carton-bundles.git
   45991c7..f11be8b  master -> master

bmo vendor tarball updated (the rest will follow)

New pull request for Crypt::OpenPGP.

https://github.com/btrott/Crypt-OpenPGP/pull/33