1501484 - build a custom python 3.7.1 rpm for releng puppet

Status: RESOLVED WONTFIX

(Release Engineering :: Release Automation, defect)

Description

[Aki Sasaki (not active)]

For our CentOS puppet infrastructure, we build custom RPMs to install packages.
We've done this for the past 3 versions of python 3 (3.5.2, 3.6.5-1, 3.6.5-2 (lzma));
we need to do the same for 3.7.1.

Documentation for how to build an RPM is here [1]. I already created a spec file
and landed that here [2]. I also deployed that and started testing, and realized
we need a new build, because of this error message during building:

    Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with
    X509_VERIFY_PARAM_set1_host(). LibreSSL 2.6.4 and earlier do not provide the
    necessary APIs, https://github.com/libressl-portable/portable/issues/381

This breaks the _ssl package, which breaks downloading packages from puppet,
plus it would break all of our scriptworkers.

Unfortunately, we currently have OpenSSL 1.0.1e deployed in our CentOS mirrors.
We therefore need to either build or mirror OpenSSL 1.0.2 or 1.1.x .
Mirroring may be easier, but we'd need to make sure it doesn't auto-roll-out
to machines before we're ready (make sure we have our OpenSSL version pinned
everywhere).

Once we build a version of python 3.7.1 with a working ssl that's ready for
testing, we can add that to the mozilla-python37 repo using these docs [3],
and we should be able to test in puppet.

- update openssl, either by mirroring or building a custom package
  - if mirroring, make sure we have the current version pinned everywhere before deploying
  - if building custom package, we need to write a puppet patch that points at the custom repo. Here is the puppet patch that would point us at the mozilla-python37 repo: [4].
- rebuild the python 3.7.1 rpm
  - update rpmpackager1 with the new openssl first?
  - rebuild per [1]
  - download the built rpms
- upload the built rpms to releng-puppet2.srv.releng.mdc1.mozilla.com:/data/repos/yum/custom/mozilla-python37/x86_64/
  - run `createrepo --update .` and `puppetmaster-fixperms` per [3]
- proceed to testing via puppet environment

[1] https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/HowTo/Build_RPMs
[2] https://github.com/escapewindow/build-puppet/commit/b38e3a3e99308bd3c8809b2575ae4df0c4df48e4
[3] https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/Packages#CentOS:_Adding_New_Packages
[4] https://github.com/escapewindow/build-puppet/commit/8d09a6617cc021f1f622ba2313812d5051d73b84

Comment 1

[Aki Sasaki (not active)]

13:47 <catlee> dividehex: how hard would it be to update our openssl, so we can build a newer python? https://bugzilla.mozilla.org/show_bug.cgi?id=1501484#c0
13:47 <firebot> Bug 1501484 — NEW, [e-mail] — build a custom python 3.7.1 rpm for releng puppet
13:54 <dividehex> catlee: aki: i don't think it should be that hard.  We've updated it a few times and it is definitely pinned so it should be safe to update the custom repo and then test the new version on a testing host

Comment 2

[Kendall Libby [:fubar] (he/him)]

Dragos, could you have a go at building a new openssl rpm for CentOS?

Comment 3

[Dragos Crisan [:dragrom]]

Created https://bugzilla.mozilla.org/show_bug.cgi?id=1505293 for this

Comment 4

[Aki Sasaki (not active)]

-> docker