1190516 - pash should grab variables from files, not use Puppet or Ansible templates

Status: RESOLVED FIXED

(Developer Services :: Mercurial: hg.mozilla.org, defect)

Description

[Gregory Szorc [:gps]]

Currently pash is using Puppet/Ansible templates to insert things like credentials directly into the source code. This makes deploying and testing pash a bit painful. Let's have it grab credentials from files instead.

Comment 1

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pash: remove execute bit from ldap_helper.py (bug 1190516); r=fubar

pash: remove execute bit from ldap_helper.py (bug 1190516); r?fubar

The __main__ routine in this file was not very useful. It doesn't need
to be executable. Remove the executable bit and the shebang.

Comment 2

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pash: add MPL license header (bug 1190516); r=fubar

pash: add MPL license header (bug 1190516); r?fubar

Files should have license info.

These files all originated in an internal Mozilla repository. The files
were all written by people who were Mozilla employees at the time. The
rights to the content is therefore Mozilla's. We typically put MPL 2.0
on things, so that's what I am doing here.

Comment 3

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pash: reformat ldap_helper.py (bug 1190516); r=fubar

pash: reformat ldap_helper.py (bug 1190516); r?fubar

We were using 2 space indent before. I ran autopep8 on this to clean up
the whitespace. I manually rolled back some changed to make the diff
more comprehensible. You may want to ignore whitespace when looking at
this diff, as 95% of the changes are intra-line whitespace only.

Comment 4

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pash: remove verbose users support (bug 1190516); r=fubar

pash: remove verbose users support (bug 1190516); r?fubar

It was hard coded to bkero. It is obviously not used any more. And it
likely had marginal utility to him. Remove the code.

Comment 5

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pash: remove executability of hg_helper.py (bug 1190516); r=fubar

pash: remove executability of hg_helper.py (bug 1190516); r?fubar

Another file that has a __main__ providing little to no value. Kill the
__main__ function and remove executability from the file.

Comment 6

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pash: grab LDAP credentials from a file (bug 1190516); r?fubar

pash: grab LDAP credentials from a file (bug 1190516); r?fubar

Previously, LDAP credentials were inserted directly into ldap_helper.py
via Puppet template magic. This is somewhat hacky and difficult to
integrate into deployment tools. Change to reading credentials from a
file instead.

With this change, ldap_helper.py no longer needs to be preprocessed
before running.

Comment 7

[Gregory Szorc [:gps]]

Comment on attachment 8642627 [details]
MozReview Request: pash: grab LDAP credentials from a file (bug 1190516); r?fubar

I'm going to change this one around a bit. Cancelling review on just this one commit for right now.

Comment 8

[Kendall Libby [:fubar] (he/him)]

Comment on attachment 8642622 [details]
MozReview Request: pash: remove execute bit from ldap_helper.py (bug 1190516); r=fubar

https://reviewboard.mozilla.org/r/14747/#review13339

Ship It!

Comment 9

[Kendall Libby [:fubar] (he/him)]

Comment on attachment 8642623 [details]
MozReview Request: pash: add MPL license header (bug 1190516); r=fubar

https://reviewboard.mozilla.org/r/14749/#review13341

Ship It!

Comment 10

[Kendall Libby [:fubar] (he/him)]

Comment on attachment 8642624 [details]
MozReview Request: pash: reformat ldap_helper.py (bug 1190516); r=fubar

https://reviewboard.mozilla.org/r/14751/#review13343

Ship It!

Comment 11

[Kendall Libby [:fubar] (he/him)]

Comment on attachment 8642625 [details]
MozReview Request: pash: remove verbose users support (bug 1190516); r=fubar

https://reviewboard.mozilla.org/r/14753/#review13345

Ship It!

Comment 12

[Kendall Libby [:fubar] (he/him)]

Comment on attachment 8642626 [details]
MozReview Request: pash: remove executability of hg_helper.py (bug 1190516); r=fubar

https://reviewboard.mozilla.org/r/14755/#review13347

Ship It!

Comment 13

[Kendall Libby [:fubar] (he/him)]

Comment on attachment 8642627 [details]
MozReview Request: pash: grab LDAP credentials from a file (bug 1190516); r?fubar

https://reviewboard.mozilla.org/r/14757/#review13349

Ship It!

Comment 14

[Gregory Szorc [:gps]]

Comment on attachment 8642627 [details]
MozReview Request: pash: grab LDAP credentials from a file (bug 1190516); r?fubar

pash: grab LDAP credentials from a file (bug 1190516); r?fubar

Previously, LDAP credentials were inserted directly into ldap_helper.py
via Puppet template magic. This is somewhat hacky and difficult to
integrate into deployment tools. Change to reading credentials from a
file instead.

With this change, ldap_helper.py no longer needs to be preprocessed
before running. Hacky code in docker-hg-ssh's entrypoint script has been
removed. The role now also installs an appropriate JSON file providing
the needed info.

Comment 15

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pash: reformat imports in hg_helper.py (bug 1190516); r?fubar

pash: reformat imports in hg_helper.py (bug 1190516); r?fubar

Switching to a more standard formatting. Alphabetized with standard
library imports grouped first.

Comment 16

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pash: grab LDAP URI from settings file (bug 1190516); r?fubar

pash: grab LDAP URI from settings file (bug 1190516); r?fubar

In the same spirit as the previous commit, we stop preprocessing Python
files by extracting variables to a standalone file. We do still
preprocess a file in entrypoint.py, but it is a data file, which is
preferred to a .py file.

Comment 17

[Gregory Szorc [:gps]]

Attachment: MozReview Request: pash: grab URLs from config file (bug 1190516); r?fubar

pash: grab URLs from config file (bug 1190516); r?fubar

We complete our refactoring of pash to grab configuration from an
external file by updating pash.py to obtain LDAP URLs from the just
established LDAP config file.

Comment 18

[Gregory Szorc [:gps]]

Comment on attachment 8642622 [details]
MozReview Request: pash: remove execute bit from ldap_helper.py (bug 1190516); r=fubar

pash: remove execute bit from ldap_helper.py (bug 1190516); r=fubar

The __main__ routine in this file was not very useful. It doesn't need
to be executable. Remove the executable bit and the shebang.

Comment 19

[Gregory Szorc [:gps]]

Comment on attachment 8642623 [details]
MozReview Request: pash: add MPL license header (bug 1190516); r=fubar

pash: add MPL license header (bug 1190516); r=fubar

Files should have license info.

These files all originated in an internal Mozilla repository. The files
were all written by people who were Mozilla employees at the time. The
rights to the content is therefore Mozilla's. We typically put MPL 2.0
on things, so that's what I am doing here.

Comment 20

[Gregory Szorc [:gps]]

Comment on attachment 8642624 [details]
MozReview Request: pash: reformat ldap_helper.py (bug 1190516); r=fubar

pash: reformat ldap_helper.py (bug 1190516); r=fubar

We were using 2 space indent before. I ran autopep8 on this to clean up
the whitespace. I manually rolled back some changed to make the diff
more comprehensible. You may want to ignore whitespace when looking at
this diff, as 95% of the changes are intra-line whitespace only.

Comment 21

[Gregory Szorc [:gps]]

Comment on attachment 8642625 [details]
MozReview Request: pash: remove verbose users support (bug 1190516); r=fubar

pash: remove verbose users support (bug 1190516); r=fubar

It was hard coded to bkero. It is obviously not used any more. And it
likely had marginal utility to him. Remove the code.

Comment 22

[Gregory Szorc [:gps]]

Comment on attachment 8642626 [details]
MozReview Request: pash: remove executability of hg_helper.py (bug 1190516); r=fubar

pash: remove executability of hg_helper.py (bug 1190516); r=fubar

Another file that has a __main__ providing little to no value. Kill the
__main__ function and remove executability from the file.

Comment 23

[Gregory Szorc [:gps]]

Comment on attachment 8642627 [details]
MozReview Request: pash: grab LDAP credentials from a file (bug 1190516); r?fubar

pash: grab LDAP credentials from a file (bug 1190516); r?fubar

Previously, LDAP credentials were inserted directly into ldap_helper.py
via Puppet template magic. This is somewhat hacky and difficult to
integrate into deployment tools. Change to reading credentials from a
file instead.

With this change, ldap_helper.py no longer needs to be preprocessed
before running. Hacky code in docker-hg-ssh's entrypoint script has been
removed. The role now also installs an appropriate JSON file providing
the needed info.

Comment 24

[Gregory Szorc [:gps]]

Comment on attachment 8642648 [details]
MozReview Request: pash: reformat imports in hg_helper.py (bug 1190516); r?fubar

pash: reformat imports in hg_helper.py (bug 1190516); r?fubar

Switching to a more standard formatting. Alphabetized with standard
library imports grouped first.

Comment 25

[Gregory Szorc [:gps]]

Comment on attachment 8642649 [details]
MozReview Request: pash: grab LDAP URI from settings file (bug 1190516); r?fubar

pash: grab LDAP URI from settings file (bug 1190516); r?fubar

In the same spirit as the previous commit, we stop preprocessing Python
files by extracting variables to a standalone file. We do still
preprocess a file in entrypoint.py, but it is a data file, which is
preferred to a .py file.

Comment 26

[Gregory Szorc [:gps]]

Comment on attachment 8642650 [details]
MozReview Request: pash: grab URLs from config file (bug 1190516); r?fubar

pash: grab URLs from config file (bug 1190516); r?fubar

We complete our refactoring of pash to grab configuration from an
external file by updating pash.py to obtain LDAP URLs from the just
established LDAP config file.

Comment 27

[Gregory Szorc [:gps]]

Comment on attachment 8642649 [details]
MozReview Request: pash: grab LDAP URI from settings file (bug 1190516); r?fubar

pash: grab LDAP URI from settings file (bug 1190516); r?fubar

In the same spirit as the previous commit, we stop preprocessing Python
files by extracting variables to a standalone file. We do still
preprocess a file in entrypoint.py, but it is a data file, which is
preferred to a .py file.

Comment 28

[Gregory Szorc [:gps]]

Comment on attachment 8642650 [details]
MozReview Request: pash: grab URLs from config file (bug 1190516); r?fubar

pash: grab URLs from config file (bug 1190516); r?fubar

We complete our refactoring of pash to grab configuration from an
external file by updating pash.py to obtain LDAP URLs from the just
established LDAP config file.

Comment 29

[Kendall Libby [:fubar] (he/him)]

https://reviewboard.mozilla.org/r/14757/#review13399

Ship It!

Comment 30

[Kendall Libby [:fubar] (he/him)]

Comment on attachment 8642648 [details]
MozReview Request: pash: reformat imports in hg_helper.py (bug 1190516); r?fubar

https://reviewboard.mozilla.org/r/14759/#review13403

Ship It!

Comment 31

[Kendall Libby [:fubar] (he/him)]

Comment on attachment 8642649 [details]
MozReview Request: pash: grab LDAP URI from settings file (bug 1190516); r?fubar

https://reviewboard.mozilla.org/r/14761/#review13405

Ship It!

Comment 32

[Kendall Libby [:fubar] (he/him)]

Comment on attachment 8642650 [details]
MozReview Request: pash: grab URLs from config file (bug 1190516); r?fubar

https://reviewboard.mozilla.org/r/14763/#review13407

Ship It!

Comment 33

[Gregory Szorc [:gps]]

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/48b1c62b2e01385eb0b35aa4d37d0fc2a599f923
changeset:  48b1c62b2e01385eb0b35aa4d37d0fc2a599f923
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 03 14:50:23 2015 -0700
description:
pash: remove execute bit from ldap_helper.py (bug 1190516); r=fubar

The __main__ routine in this file was not very useful. It doesn't need
to be executable. Remove the executable bit and the shebang.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/4c86dfa216787caed08c6f4daa2f62d084932e75
changeset:  4c86dfa216787caed08c6f4daa2f62d084932e75
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 03 14:50:37 2015 -0700
description:
pash: add MPL license header (bug 1190516); r=fubar

Files should have license info.

These files all originated in an internal Mozilla repository. The files
were all written by people who were Mozilla employees at the time. The
rights to the content is therefore Mozilla's. We typically put MPL 2.0
on things, so that's what I am doing here.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/5e4eff738e1f70024191bffdbb524a57c6f973be
changeset:  5e4eff738e1f70024191bffdbb524a57c6f973be
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 03 14:50:50 2015 -0700
description:
pash: reformat ldap_helper.py (bug 1190516); r=fubar

We were using 2 space indent before. I ran autopep8 on this to clean up
the whitespace. I manually rolled back some changed to make the diff
more comprehensible. You may want to ignore whitespace when looking at
this diff, as 95% of the changes are intra-line whitespace only.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/bb8550e36ca3d8a0193a44f902f228caf87fb3d1
changeset:  bb8550e36ca3d8a0193a44f902f228caf87fb3d1
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 03 14:51:13 2015 -0700
description:
pash: remove verbose users support (bug 1190516); r=fubar

It was hard coded to bkero. It is obviously not used any more. And it
likely had marginal utility to him. Remove the code.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/4bddc6bde009d850afd9810f9cce2b6dd23ceb4b
changeset:  4bddc6bde009d850afd9810f9cce2b6dd23ceb4b
user:       Gregory Szorc <[e-mail]>
date:       Mon Aug 03 14:51:25 2015 -0700
description:
pash: remove executability of hg_helper.py (bug 1190516); r=fubar

Another file that has a __main__ providing little to no value. Kill the
__main__ function and remove executability from the file.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/7e95ce41fc207f37b5350f72c3642933c3ee8412
changeset:  7e95ce41fc207f37b5350f72c3642933c3ee8412
user:       Gregory Szorc <[e-mail]>
date:       Tue Aug 04 11:12:06 2015 -0700
description:
pash: grab LDAP credentials from a file (bug 1190516); r=fubar

Previously, LDAP credentials were inserted directly into ldap_helper.py
via Puppet template magic. This is somewhat hacky and difficult to
integrate into deployment tools. Change to reading credentials from a
file instead.

With this change, ldap_helper.py no longer needs to be preprocessed
before running. Hacky code in docker-hg-ssh's entrypoint script has been
removed. The role now also installs an appropriate JSON file providing
the needed info.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/3b0e729bc3d9f04e5c1bb70c5eba71cd5077a3ec
changeset:  3b0e729bc3d9f04e5c1bb70c5eba71cd5077a3ec
user:       Gregory Szorc <[e-mail]>
date:       Tue Aug 04 11:12:18 2015 -0700
description:
pash: reformat imports in hg_helper.py (bug 1190516); r=fubar

Switching to a more standard formatting. Alphabetized with standard
library imports grouped first.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/f1df217276bf72db3558427c3a706f9b079c0676
changeset:  f1df217276bf72db3558427c3a706f9b079c0676
user:       Gregory Szorc <[e-mail]>
date:       Tue Aug 04 11:12:29 2015 -0700
description:
pash: grab LDAP URI from settings file (bug 1190516); r=fubar

In the same spirit as the previous commit, we stop preprocessing Python
files by extracting variables to a standalone file. We do still
preprocess a file in entrypoint.py, but it is a data file, which is
preferred to a .py file.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/2b7276ed597e636b2676f10b079f2509953abdaf
changeset:  2b7276ed597e636b2676f10b079f2509953abdaf
user:       Gregory Szorc <[e-mail]>
date:       Tue Aug 04 11:12:41 2015 -0700
description:
pash: grab URLs from config file (bug 1190516); r=fubar

We complete our refactoring of pash to grab configuration from an
external file by updating pash.py to obtain LDAP URLs from the just
established LDAP config file.

Comment 34

[Gregory Szorc [:gps]]

There is nothing to deploy (yet) because pash is currently managed by Puppet, not Ansible. This should change soon though.