Closed Bug 1190516 Opened 6 years ago Closed 6 years ago

pash should grab variables from files, not use Puppet or Ansible templates

Categories

(Developer Services :: Mercurial: hg.mozilla.org, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: gps, Assigned: gps)

References

Details

Attachments

(9 files)

40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
Currently pash is using Puppet/Ansible templates to insert things like credentials directly into the source code. This makes deploying and testing pash a bit painful. Let's have it grab credentials from files instead.
pash: remove execute bit from ldap_helper.py (bug 1190516); r?fubar

The __main__ routine in this file was not very useful. It doesn't need
to be executable. Remove the executable bit and the shebang.
Attachment #8642622 - Flags: review?(klibby)
pash: add MPL license header (bug 1190516); r?fubar

Files should have license info.

These files all originated in an internal Mozilla repository. The files
were all written by people who were Mozilla employees at the time. The
rights to the content is therefore Mozilla's. We typically put MPL 2.0
on things, so that's what I am doing here.
Attachment #8642623 - Flags: review?(klibby)
pash: reformat ldap_helper.py (bug 1190516); r?fubar

We were using 2 space indent before. I ran autopep8 on this to clean up
the whitespace. I manually rolled back some changed to make the diff
more comprehensible. You may want to ignore whitespace when looking at
this diff, as 95% of the changes are intra-line whitespace only.
Attachment #8642624 - Flags: review?(klibby)
pash: remove verbose users support (bug 1190516); r?fubar

It was hard coded to bkero. It is obviously not used any more. And it
likely had marginal utility to him. Remove the code.
Attachment #8642625 - Flags: review?(klibby)
pash: remove executability of hg_helper.py (bug 1190516); r?fubar

Another file that has a __main__ providing little to no value. Kill the
__main__ function and remove executability from the file.
Attachment #8642626 - Flags: review?(klibby)
pash: grab LDAP credentials from a file (bug 1190516); r?fubar

Previously, LDAP credentials were inserted directly into ldap_helper.py
via Puppet template magic. This is somewhat hacky and difficult to
integrate into deployment tools. Change to reading credentials from a
file instead.

With this change, ldap_helper.py no longer needs to be preprocessed
before running.
Attachment #8642627 - Flags: review?(klibby)
Comment on attachment 8642627 [details]
MozReview Request: pash: grab LDAP credentials from a file (bug 1190516); r?fubar

I'm going to change this one around a bit. Cancelling review on just this one commit for right now.
Attachment #8642627 - Flags: review?(klibby)
Comment on attachment 8642622 [details]
MozReview Request: pash: remove execute bit from ldap_helper.py (bug 1190516); r=fubar

https://reviewboard.mozilla.org/r/14747/#review13339

Ship It!
Attachment #8642622 - Flags: review?(klibby) → review+
Comment on attachment 8642623 [details]
MozReview Request: pash: add MPL license header (bug 1190516); r=fubar

https://reviewboard.mozilla.org/r/14749/#review13341

Ship It!
Attachment #8642623 - Flags: review?(klibby) → review+
Attachment #8642624 - Flags: review?(klibby) → review+
Comment on attachment 8642624 [details]
MozReview Request: pash: reformat ldap_helper.py (bug 1190516); r=fubar

https://reviewboard.mozilla.org/r/14751/#review13343

Ship It!
Comment on attachment 8642625 [details]
MozReview Request: pash: remove verbose users support (bug 1190516); r=fubar

https://reviewboard.mozilla.org/r/14753/#review13345

Ship It!
Attachment #8642625 - Flags: review?(klibby) → review+
Comment on attachment 8642626 [details]
MozReview Request: pash: remove executability of hg_helper.py (bug 1190516); r=fubar

https://reviewboard.mozilla.org/r/14755/#review13347

Ship It!
Attachment #8642626 - Flags: review?(klibby) → review+
Comment on attachment 8642627 [details]
MozReview Request: pash: grab LDAP credentials from a file (bug 1190516); r?fubar

https://reviewboard.mozilla.org/r/14757/#review13349

Ship It!
Comment on attachment 8642627 [details]
MozReview Request: pash: grab LDAP credentials from a file (bug 1190516); r?fubar

pash: grab LDAP credentials from a file (bug 1190516); r?fubar

Previously, LDAP credentials were inserted directly into ldap_helper.py
via Puppet template magic. This is somewhat hacky and difficult to
integrate into deployment tools. Change to reading credentials from a
file instead.

With this change, ldap_helper.py no longer needs to be preprocessed
before running. Hacky code in docker-hg-ssh's entrypoint script has been
removed. The role now also installs an appropriate JSON file providing
the needed info.
pash: reformat imports in hg_helper.py (bug 1190516); r?fubar

Switching to a more standard formatting. Alphabetized with standard
library imports grouped first.
Attachment #8642648 - Flags: review?(klibby)
pash: grab LDAP URI from settings file (bug 1190516); r?fubar

In the same spirit as the previous commit, we stop preprocessing Python
files by extracting variables to a standalone file. We do still
preprocess a file in entrypoint.py, but it is a data file, which is
preferred to a .py file.
Attachment #8642649 - Flags: review?(klibby)
pash: grab URLs from config file (bug 1190516); r?fubar

We complete our refactoring of pash to grab configuration from an
external file by updating pash.py to obtain LDAP URLs from the just
established LDAP config file.
Attachment #8642650 - Flags: review?(klibby)
Comment on attachment 8642622 [details]
MozReview Request: pash: remove execute bit from ldap_helper.py (bug 1190516); r=fubar

pash: remove execute bit from ldap_helper.py (bug 1190516); r=fubar

The __main__ routine in this file was not very useful. It doesn't need
to be executable. Remove the executable bit and the shebang.
Attachment #8642622 - Attachment description: MozReview Request: pash: remove execute bit from ldap_helper.py (bug 1190516); r?fubar → MozReview Request: pash: remove execute bit from ldap_helper.py (bug 1190516); r=fubar
Comment on attachment 8642623 [details]
MozReview Request: pash: add MPL license header (bug 1190516); r=fubar

pash: add MPL license header (bug 1190516); r=fubar

Files should have license info.

These files all originated in an internal Mozilla repository. The files
were all written by people who were Mozilla employees at the time. The
rights to the content is therefore Mozilla's. We typically put MPL 2.0
on things, so that's what I am doing here.
Attachment #8642623 - Attachment description: MozReview Request: pash: add MPL license header (bug 1190516); r?fubar → MozReview Request: pash: add MPL license header (bug 1190516); r=fubar
Comment on attachment 8642624 [details]
MozReview Request: pash: reformat ldap_helper.py (bug 1190516); r=fubar

pash: reformat ldap_helper.py (bug 1190516); r=fubar

We were using 2 space indent before. I ran autopep8 on this to clean up
the whitespace. I manually rolled back some changed to make the diff
more comprehensible. You may want to ignore whitespace when looking at
this diff, as 95% of the changes are intra-line whitespace only.
Attachment #8642624 - Attachment description: MozReview Request: pash: reformat ldap_helper.py (bug 1190516); r?fubar → MozReview Request: pash: reformat ldap_helper.py (bug 1190516); r=fubar
Attachment #8642625 - Attachment description: MozReview Request: pash: remove verbose users support (bug 1190516); r?fubar → MozReview Request: pash: remove verbose users support (bug 1190516); r=fubar
Comment on attachment 8642625 [details]
MozReview Request: pash: remove verbose users support (bug 1190516); r=fubar

pash: remove verbose users support (bug 1190516); r=fubar

It was hard coded to bkero. It is obviously not used any more. And it
likely had marginal utility to him. Remove the code.
Comment on attachment 8642626 [details]
MozReview Request: pash: remove executability of hg_helper.py (bug 1190516); r=fubar

pash: remove executability of hg_helper.py (bug 1190516); r=fubar

Another file that has a __main__ providing little to no value. Kill the
__main__ function and remove executability from the file.
Attachment #8642626 - Attachment description: MozReview Request: pash: remove executability of hg_helper.py (bug 1190516); r?fubar → MozReview Request: pash: remove executability of hg_helper.py (bug 1190516); r=fubar
Comment on attachment 8642627 [details]
MozReview Request: pash: grab LDAP credentials from a file (bug 1190516); r?fubar

pash: grab LDAP credentials from a file (bug 1190516); r?fubar

Previously, LDAP credentials were inserted directly into ldap_helper.py
via Puppet template magic. This is somewhat hacky and difficult to
integrate into deployment tools. Change to reading credentials from a
file instead.

With this change, ldap_helper.py no longer needs to be preprocessed
before running. Hacky code in docker-hg-ssh's entrypoint script has been
removed. The role now also installs an appropriate JSON file providing
the needed info.
Comment on attachment 8642648 [details]
MozReview Request: pash: reformat imports in hg_helper.py (bug 1190516); r?fubar

pash: reformat imports in hg_helper.py (bug 1190516); r?fubar

Switching to a more standard formatting. Alphabetized with standard
library imports grouped first.
Comment on attachment 8642649 [details]
MozReview Request: pash: grab LDAP URI from settings file (bug 1190516); r?fubar

pash: grab LDAP URI from settings file (bug 1190516); r?fubar

In the same spirit as the previous commit, we stop preprocessing Python
files by extracting variables to a standalone file. We do still
preprocess a file in entrypoint.py, but it is a data file, which is
preferred to a .py file.
Comment on attachment 8642650 [details]
MozReview Request: pash: grab URLs from config file (bug 1190516); r?fubar

pash: grab URLs from config file (bug 1190516); r?fubar

We complete our refactoring of pash to grab configuration from an
external file by updating pash.py to obtain LDAP URLs from the just
established LDAP config file.
Comment on attachment 8642649 [details]
MozReview Request: pash: grab LDAP URI from settings file (bug 1190516); r?fubar

pash: grab LDAP URI from settings file (bug 1190516); r?fubar

In the same spirit as the previous commit, we stop preprocessing Python
files by extracting variables to a standalone file. We do still
preprocess a file in entrypoint.py, but it is a data file, which is
preferred to a .py file.
Comment on attachment 8642650 [details]
MozReview Request: pash: grab URLs from config file (bug 1190516); r?fubar

pash: grab URLs from config file (bug 1190516); r?fubar

We complete our refactoring of pash to grab configuration from an
external file by updating pash.py to obtain LDAP URLs from the just
established LDAP config file.
Comment on attachment 8642648 [details]
MozReview Request: pash: reformat imports in hg_helper.py (bug 1190516); r?fubar

https://reviewboard.mozilla.org/r/14759/#review13403

Ship It!
Attachment #8642648 - Flags: review?(klibby) → review+
Attachment #8642649 - Flags: review?(klibby) → review+
Comment on attachment 8642649 [details]
MozReview Request: pash: grab LDAP URI from settings file (bug 1190516); r?fubar

https://reviewboard.mozilla.org/r/14761/#review13405

Ship It!
Comment on attachment 8642650 [details]
MozReview Request: pash: grab URLs from config file (bug 1190516); r?fubar

https://reviewboard.mozilla.org/r/14763/#review13407

Ship It!
Attachment #8642650 - Flags: review?(klibby) → review+
url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/48b1c62b2e01385eb0b35aa4d37d0fc2a599f923
changeset:  48b1c62b2e01385eb0b35aa4d37d0fc2a599f923
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 03 14:50:23 2015 -0700
description:
pash: remove execute bit from ldap_helper.py (bug 1190516); r=fubar

The __main__ routine in this file was not very useful. It doesn't need
to be executable. Remove the executable bit and the shebang.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/4c86dfa216787caed08c6f4daa2f62d084932e75
changeset:  4c86dfa216787caed08c6f4daa2f62d084932e75
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 03 14:50:37 2015 -0700
description:
pash: add MPL license header (bug 1190516); r=fubar

Files should have license info.

These files all originated in an internal Mozilla repository. The files
were all written by people who were Mozilla employees at the time. The
rights to the content is therefore Mozilla's. We typically put MPL 2.0
on things, so that's what I am doing here.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/5e4eff738e1f70024191bffdbb524a57c6f973be
changeset:  5e4eff738e1f70024191bffdbb524a57c6f973be
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 03 14:50:50 2015 -0700
description:
pash: reformat ldap_helper.py (bug 1190516); r=fubar

We were using 2 space indent before. I ran autopep8 on this to clean up
the whitespace. I manually rolled back some changed to make the diff
more comprehensible. You may want to ignore whitespace when looking at
this diff, as 95% of the changes are intra-line whitespace only.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/bb8550e36ca3d8a0193a44f902f228caf87fb3d1
changeset:  bb8550e36ca3d8a0193a44f902f228caf87fb3d1
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 03 14:51:13 2015 -0700
description:
pash: remove verbose users support (bug 1190516); r=fubar

It was hard coded to bkero. It is obviously not used any more. And it
likely had marginal utility to him. Remove the code.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/4bddc6bde009d850afd9810f9cce2b6dd23ceb4b
changeset:  4bddc6bde009d850afd9810f9cce2b6dd23ceb4b
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 03 14:51:25 2015 -0700
description:
pash: remove executability of hg_helper.py (bug 1190516); r=fubar

Another file that has a __main__ providing little to no value. Kill the
__main__ function and remove executability from the file.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/7e95ce41fc207f37b5350f72c3642933c3ee8412
changeset:  7e95ce41fc207f37b5350f72c3642933c3ee8412
user:       Gregory Szorc <gps@mozilla.com>
date:       Tue Aug 04 11:12:06 2015 -0700
description:
pash: grab LDAP credentials from a file (bug 1190516); r=fubar

Previously, LDAP credentials were inserted directly into ldap_helper.py
via Puppet template magic. This is somewhat hacky and difficult to
integrate into deployment tools. Change to reading credentials from a
file instead.

With this change, ldap_helper.py no longer needs to be preprocessed
before running. Hacky code in docker-hg-ssh's entrypoint script has been
removed. The role now also installs an appropriate JSON file providing
the needed info.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/3b0e729bc3d9f04e5c1bb70c5eba71cd5077a3ec
changeset:  3b0e729bc3d9f04e5c1bb70c5eba71cd5077a3ec
user:       Gregory Szorc <gps@mozilla.com>
date:       Tue Aug 04 11:12:18 2015 -0700
description:
pash: reformat imports in hg_helper.py (bug 1190516); r=fubar

Switching to a more standard formatting. Alphabetized with standard
library imports grouped first.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/f1df217276bf72db3558427c3a706f9b079c0676
changeset:  f1df217276bf72db3558427c3a706f9b079c0676
user:       Gregory Szorc <gps@mozilla.com>
date:       Tue Aug 04 11:12:29 2015 -0700
description:
pash: grab LDAP URI from settings file (bug 1190516); r=fubar

In the same spirit as the previous commit, we stop preprocessing Python
files by extracting variables to a standalone file. We do still
preprocess a file in entrypoint.py, but it is a data file, which is
preferred to a .py file.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/2b7276ed597e636b2676f10b079f2509953abdaf
changeset:  2b7276ed597e636b2676f10b079f2509953abdaf
user:       Gregory Szorc <gps@mozilla.com>
date:       Tue Aug 04 11:12:41 2015 -0700
description:
pash: grab URLs from config file (bug 1190516); r=fubar

We complete our refactoring of pash to grab configuration from an
external file by updating pash.py to obtain LDAP URLs from the just
established LDAP config file.
There is nothing to deploy (yet) because pash is currently managed by Puppet, not Ansible. This should change soon though.
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.