661534 - Spark EOL stage: Error while auto-updating

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Franck Lecollinet]

I'm currently receiving this error every 5 minutes from the stage auto-update script:

Cron <root@mrapp-stage04> cd /data/www/spark.allizom.org/spark-eol && /usr/bin/python2.6 bin/update_site.py -e stage > /dev/null && /bin/touch wsgi/spark-eol.wsgi

error: SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed while accessing https://github.com/mozilla/spark-eol/info/refs

fatal: HTTP request failed
There was an error while updating. Please try again later. Aborting.

Comment 1

[Franck Lecollinet]

It appears that Github has renewed their SSL certificate, resulting in this issue. Here is a fix for CentOS, just in case it might be helpful:

http://eric.lubow.org/2011/security/fixing-centos-root-certificate-authority-issues/

Comment 2

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

There are many other Github users that are complaining about this issue as well.

This issue blocks the retirement of the Spark campaign in bug 661519.

Comment 3

[Jake Maul [:jakem]]

The issue is that RHEL5 / CentOS5 ship an old ca-bundle.crt built from Mozilla NSS:

http://bonsai.mozilla.org/cvslog.cgi?file=mozilla/security/nss/lib/ckfw/builtins/certdata.txt&rev=HEAD&mark=1.74

RHEL5 ships with version 1.39 of that. The new github.com cert is signed by DigiCert, which was added in version 1.42.

RHEL6 ships with version 1.63, so those servers are unaffected by this.

====================

We are working on a global config update to push out a current version of the ca-bundle.crt across all of our infrastructure.

====================

In the meantime, you can use this as a workaround:

GIT_SSL_NO_VERIFY=true git pull (or whatever you need)

====================

I have run the job from comment 0 this way:

[root@mrapp-stage04 spark-eol]# GIT_SSL_NO_VERIFY=true python26 bin/update_site.py -e stage
At revision 89826.

====================

I'll update this bug again when the new bundle is pushed out everywhere. Thanks!

Comment 4

[Jake Maul [:jakem]]

This has been pushed out via puppet, and should be live on the majority of our infrastructure by now (not everything is puppetized, but most things are). If anyone is still having trouble, please re-open and note this bug with what server is having the problem and we'll fix it manually.

Thanks!