Closed Bug 1192338 Opened 6 years ago Closed 6 years ago

Ansibilize reviewboard-hg

Categories

(MozReview Graveyard :: General, defect, P1)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: gps, Assigned: gps)

Details

Attachments

(15 files)

40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
40 bytes, text/x-review-board-request
fubar
: review+
Details
Following work in hg.mozilla.org land, we should get reviewboard-hg using Ansible for as much as possible. This shouldn't be too invasive, as the existing hg-reviewboard Ansible role was based on the Puppet module and we have thorough tests of MozReview (unlike hg.mozilla.org's hosting) to validate any changes we make.
ansible/hg-reviewboard: configure IUS repo (bug 1192338); r?fubar

We'll eventually want to install Python 2.7. Configure the IUS repo to
enable this.
Attachment #8645101 - Flags: review?(klibby)
ansible/hg-reviewboard: install v-c-t from public server (bug 1192338); r?fubar

Previously, we assumed v-c-t existed in a staging directory on the local
filesystem (from Docker). This assumption does not hold true on servers.
Install v-c-t from a public server when not running in Docker.
Attachment #8645102 - Flags: review?(klibby)
ansible/hg-reviewboard: drop maximum requests in WSGI processes (bug 1192388); r?fubar

This was recently changed in production as a way to try to keep memory
leaks in check.
Attachment #8645103 - Flags: review?(klibby)
ansible/hg-reviewboard: remove unused pash features (bug 1192338); r?fubar

We have 2 versions of pash floating around. Over time, I'd like to
consolidate them. But that's a large task. This commit starts small
by removing unused features from reviewboard's copy of pash.

Support for the "pushlog" and "repo-group" commands have been removed.
These are or were used for replication and are not relevant to
reviewboard's Mercurial server.

Since we removed the "repo-group" command, repo_group.py no longer
serves a purpose and has been removed as well.
Attachment #8645104 - Flags: review?(klibby)
ansible/hg-reviewboard: clean up pash.py (bug 1192338); r?fubar

I'm going to try to have this file match the one used by hg.mozilla.org
as much as possible. Start by performing some basic whitespace cleanup
and by consolidating use of the USER environment variable.
Attachment #8645105 - Flags: review?(klibby)
ansible/hg-reviewboard: move QuoteForPOSIX into pash.py (bug 1192338); r?fubar

We don't want pash.py importing non-system modules because failure to
import could lead to inability to SSH into the machine! We move
QuoteForPOSIX into pash.py to avoid a module import.
Attachment #8645106 - Flags: review?(klibby)
ansible/hg-reviewboard: remove server port handling (bug 1192338); r?fubar

We are only ever listening on port 22. This code is not needed.
Attachment #8645107 - Flags: review?(klibby)
ansible/hg-reviewboard: move non-root handling to own function (bug 1192338); r?fubar

We move the processing of non-root logins to its own function so we can
delay import modules that may fail to import. This ensures that root
logins have minimal dependencies and are more robust against failures.

We also clean up some unused imports.
Attachment #8645108 - Flags: review?(klibby)
pash: remove unused variable; r?fubar

I found this as part of comparing the 2 versions of pash.py.
Attachment #8645109 - Flags: review?(klibby)
ansible/hg-reviewboard: use standard environment variable fetching (bug 1192338); r?fubar

os.environ is preferred over the low-level os.getenv() function.
Attachment #8645110 - Flags: review?(klibby)
ansible/hg-reviewboard: fix many style nits (bug 1192338); r?fubar

There should be no significant code changes in this commit. Just a lot
of style refactoring to act on warnings issued by flake8.
Attachment #8645111 - Flags: review?(klibby)
ansible/hg-reviewboard: review executability of pash files (bug 1192338); r?fubar

These files are Python support modules, not scripts. Remove the shebang,
any __main__ routines, and executable bit from files.
Attachment #8645112 - Flags: review?(klibby)
ansible/hg-reviewboard: more minor formatting changes (bug 1192388); r?fubar
Attachment #8645113 - Flags: review?(klibby)
Comment on attachment 8645101 [details]
MozReview Request: ansible/hg-reviewboard: configure IUS repo (bug 1192338); r=fubar

https://reviewboard.mozilla.org/r/15389/#review13769

Ship It!
Attachment #8645101 - Flags: review?(klibby) → review+
Comment on attachment 8645102 [details]
MozReview Request: ansible/hg-reviewboard: install v-c-t from public server (bug 1192338); r=fubar

https://reviewboard.mozilla.org/r/15391/#review13773

Ship It!
Attachment #8645102 - Flags: review?(klibby) → review+
Comment on attachment 8645103 [details]
MozReview Request: ansible/hg-reviewboard: drop maximum requests in WSGI processes (bug 1192388); r=fubar

https://reviewboard.mozilla.org/r/15393/#review13775

Ship It!
Attachment #8645103 - Flags: review?(klibby) → review+
Comment on attachment 8645104 [details]
MozReview Request: ansible/hg-reviewboard: remove unused pash features (bug 1192338); r=fubar

https://reviewboard.mozilla.org/r/15395/#review13777

Ship It!
Attachment #8645104 - Flags: review?(klibby) → review+
Comment on attachment 8645105 [details]
MozReview Request: ansible/hg-reviewboard: clean up pash.py (bug 1192338); r=fubar

https://reviewboard.mozilla.org/r/15397/#review13779

Ship It!
Attachment #8645105 - Flags: review?(klibby) → review+
Attachment #8645106 - Flags: review?(klibby) → review+
Comment on attachment 8645106 [details]
MozReview Request: ansible/hg-reviewboard: move QuoteForPOSIX into pash.py (bug 1192338); r=fubar

https://reviewboard.mozilla.org/r/15399/#review13781

Ship It!
Comment on attachment 8645107 [details]
MozReview Request: ansible/hg-reviewboard: remove server port handling (bug 1192338); r=fubar

https://reviewboard.mozilla.org/r/15401/#review13783

Ship It!
Attachment #8645107 - Flags: review?(klibby) → review+
Attachment #8645108 - Flags: review?(klibby) → review+
Comment on attachment 8645108 [details]
MozReview Request: ansible/hg-reviewboard: move non-root handling to own function (bug 1192338); r=fubar

https://reviewboard.mozilla.org/r/15403/#review13789

Ship It!
Attachment #8645109 - Flags: review?(klibby) → review+
Comment on attachment 8645109 [details]
MozReview Request: pash: remove unused variable; r=fubar

https://reviewboard.mozilla.org/r/15405/#review13791

Ship It!
Comment on attachment 8645110 [details]
MozReview Request: ansible/hg-reviewboard: use standard environment variable fetching (bug 1192338); r=fubar

https://reviewboard.mozilla.org/r/15407/#review13793

Ship It!
Attachment #8645110 - Flags: review?(klibby) → review+
Comment on attachment 8645112 [details]
MozReview Request: ansible/hg-reviewboard: remove executability of pash files (bug 1192338); r=fubar

https://reviewboard.mozilla.org/r/15411/#review13797

Ship It!
Attachment #8645112 - Flags: review?(klibby) → review+
Attachment #8645113 - Flags: review?(klibby) → review+
Comment on attachment 8645113 [details]
MozReview Request: ansible/hg-reviewboard: more minor formatting changes (bug 1192388); r=fubar

https://reviewboard.mozilla.org/r/15413/#review13799

Ship It!
Comment on attachment 8645111 [details]
MozReview Request: ansible/hg-reviewboard: fix many style nits (bug 1192338); r=fubar

https://reviewboard.mozilla.org/r/15409/#review13801

Ship It!
Attachment #8645111 - Flags: review?(klibby) → review+
Comment on attachment 8645101 [details]
MozReview Request: ansible/hg-reviewboard: configure IUS repo (bug 1192338); r=fubar

ansible/hg-reviewboard: configure IUS repo (bug 1192338); r=fubar

We'll eventually want to install Python 2.7. Configure the IUS repo to
enable this.
Attachment #8645101 - Attachment description: MozReview Request: ansible/hg-reviewboard: configure IUS repo (bug 1192338); r?fubar → MozReview Request: ansible/hg-reviewboard: configure IUS repo (bug 1192338); r=fubar
Comment on attachment 8645102 [details]
MozReview Request: ansible/hg-reviewboard: install v-c-t from public server (bug 1192338); r=fubar

ansible/hg-reviewboard: install v-c-t from public server (bug 1192338); r=fubar

Previously, we assumed v-c-t existed in a staging directory on the local
filesystem (from Docker). This assumption does not hold true on servers.
Install v-c-t from a public server when not running in Docker.
Attachment #8645102 - Attachment description: MozReview Request: ansible/hg-reviewboard: install v-c-t from public server (bug 1192338); r?fubar → MozReview Request: ansible/hg-reviewboard: install v-c-t from public server (bug 1192338); r=fubar
Comment on attachment 8645103 [details]
MozReview Request: ansible/hg-reviewboard: drop maximum requests in WSGI processes (bug 1192388); r=fubar

ansible/hg-reviewboard: drop maximum requests in WSGI processes (bug 1192388); r=fubar

This was recently changed in production as a way to try to keep memory
leaks in check.
Attachment #8645103 - Attachment description: MozReview Request: ansible/hg-reviewboard: drop maximum requests in WSGI processes (bug 1192388); r?fubar → MozReview Request: ansible/hg-reviewboard: drop maximum requests in WSGI processes (bug 1192388); r=fubar
Comment on attachment 8645104 [details]
MozReview Request: ansible/hg-reviewboard: remove unused pash features (bug 1192338); r=fubar

ansible/hg-reviewboard: remove unused pash features (bug 1192338); r=fubar

We have 2 versions of pash floating around. Over time, I'd like to
consolidate them. But that's a large task. This commit starts small
by removing unused features from reviewboard's copy of pash.

Support for the "pushlog" and "repo-group" commands have been removed.
These are or were used for replication and are not relevant to
reviewboard's Mercurial server.

Since we removed the "repo-group" command, repo_group.py no longer
serves a purpose and has been removed as well.
Attachment #8645104 - Attachment description: MozReview Request: ansible/hg-reviewboard: remove unused pash features (bug 1192338); r?fubar → MozReview Request: ansible/hg-reviewboard: remove unused pash features (bug 1192338); r=fubar
Comment on attachment 8645105 [details]
MozReview Request: ansible/hg-reviewboard: clean up pash.py (bug 1192338); r=fubar

ansible/hg-reviewboard: clean up pash.py (bug 1192338); r=fubar

I'm going to try to have this file match the one used by hg.mozilla.org
as much as possible. Start by performing some basic whitespace cleanup
and by consolidating use of the USER environment variable.
Attachment #8645105 - Attachment description: MozReview Request: ansible/hg-reviewboard: clean up pash.py (bug 1192338); r?fubar → MozReview Request: ansible/hg-reviewboard: clean up pash.py (bug 1192338); r=fubar
Comment on attachment 8645106 [details]
MozReview Request: ansible/hg-reviewboard: move QuoteForPOSIX into pash.py (bug 1192338); r=fubar

ansible/hg-reviewboard: move QuoteForPOSIX into pash.py (bug 1192338); r=fubar

We don't want pash.py importing non-system modules because failure to
import could lead to inability to SSH into the machine! We move
QuoteForPOSIX into pash.py to avoid a module import.
Attachment #8645106 - Attachment description: MozReview Request: ansible/hg-reviewboard: move QuoteForPOSIX into pash.py (bug 1192338); r?fubar → MozReview Request: ansible/hg-reviewboard: move QuoteForPOSIX into pash.py (bug 1192338); r=fubar
Attachment #8645107 - Attachment description: MozReview Request: ansible/hg-reviewboard: remove server port handling (bug 1192338); r?fubar → MozReview Request: ansible/hg-reviewboard: remove server port handling (bug 1192338); r=fubar
Comment on attachment 8645107 [details]
MozReview Request: ansible/hg-reviewboard: remove server port handling (bug 1192338); r=fubar

ansible/hg-reviewboard: remove server port handling (bug 1192338); r=fubar

We are only ever listening on port 22. This code is not needed.
Comment on attachment 8645108 [details]
MozReview Request: ansible/hg-reviewboard: move non-root handling to own function (bug 1192338); r=fubar

ansible/hg-reviewboard: move non-root handling to own function (bug 1192338); r=fubar

We move the processing of non-root logins to its own function so we can
delay import modules that may fail to import. This ensures that root
logins have minimal dependencies and are more robust against failures.

We also clean up some unused imports.
Attachment #8645108 - Attachment description: MozReview Request: ansible/hg-reviewboard: move non-root handling to own function (bug 1192338); r?fubar → MozReview Request: ansible/hg-reviewboard: move non-root handling to own function (bug 1192338); r=fubar
Comment on attachment 8645109 [details]
MozReview Request: pash: remove unused variable; r=fubar

pash: remove unused variable; r=fubar

I found this as part of comparing the 2 versions of pash.py.
Attachment #8645109 - Attachment description: MozReview Request: pash: remove unused variable; r?fubar → MozReview Request: pash: remove unused variable; r=fubar
Attachment #8645110 - Attachment description: MozReview Request: ansible/hg-reviewboard: use standard environment variable fetching (bug 1192338); r?fubar → MozReview Request: ansible/hg-reviewboard: use standard environment variable fetching (bug 1192338); r=fubar
Comment on attachment 8645110 [details]
MozReview Request: ansible/hg-reviewboard: use standard environment variable fetching (bug 1192338); r=fubar

ansible/hg-reviewboard: use standard environment variable fetching (bug 1192338); r=fubar

os.environ is preferred over the low-level os.getenv() function.
Comment on attachment 8645111 [details]
MozReview Request: ansible/hg-reviewboard: fix many style nits (bug 1192338); r=fubar

ansible/hg-reviewboard: fix many style nits (bug 1192338); r=fubar

There should be no significant code changes in this commit. Just a lot
of style refactoring to act on warnings issued by flake8.
Attachment #8645111 - Attachment description: MozReview Request: ansible/hg-reviewboard: fix many style nits (bug 1192338); r?fubar → MozReview Request: ansible/hg-reviewboard: fix many style nits (bug 1192338); r=fubar
Comment on attachment 8645112 [details]
MozReview Request: ansible/hg-reviewboard: remove executability of pash files (bug 1192338); r=fubar

ansible/hg-reviewboard: remove executability of pash files (bug 1192338); r=fubar

These files are Python support modules, not scripts. Remove the shebang,
any __main__ routines, and executable bit from files.
Attachment #8645112 - Attachment description: MozReview Request: ansible/hg-reviewboard: review executability of pash files (bug 1192338); r?fubar → MozReview Request: ansible/hg-reviewboard: remove executability of pash files (bug 1192338); r=fubar
Comment on attachment 8645113 [details]
MozReview Request: ansible/hg-reviewboard: more minor formatting changes (bug 1192388); r=fubar

ansible/hg-reviewboard: more minor formatting changes (bug 1192388); r=fubar
Attachment #8645113 - Attachment description: MozReview Request: ansible/hg-reviewboard: more minor formatting changes (bug 1192388); r?fubar → MozReview Request: ansible/hg-reviewboard: more minor formatting changes (bug 1192388); r=fubar
ansible/hg-reviewboard: grab LDAP settings from a file (bug 1192388); r?fubar

This is similar to what we do on hg.mozilla.org. This will eventually
enable the pash files to not be templates.
Attachment #8645848 - Flags: review?(klibby)
ansible/hg-reviewboard: convert pash from templates to regular files (bug 1192388); r?fubar

With LDAP credentials coming from a standalone file, there are no longer
any substitutions in these files and they can be converted to regular
files.

We drop the docker-startup tag because there are no changes at container
start time to make, so this isn't necessary.
Attachment #8645849 - Flags: review?(klibby)
Comment on attachment 8645848 [details]
MozReview Request: ansible/hg-reviewboard: grab LDAP settings from a file (bug 1192388); r?fubar

https://reviewboard.mozilla.org/r/15557/#review13991

Ship It!
Attachment #8645848 - Flags: review?(klibby) → review+
Comment on attachment 8645849 [details]
MozReview Request: ansible/hg-reviewboard: convert pash from templates to regular files (bug 1192388); r?fubar

https://reviewboard.mozilla.org/r/15559/#review13993

Ship It!
Attachment #8645849 - Flags: review?(klibby) → review+
url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/50ed4a74bfb5a112df4a7fd43d8b36e990000ac3
changeset:  50ed4a74bfb5a112df4a7fd43d8b36e990000ac3
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 10 11:39:31 2015 -0700
description:
ansible/hg-reviewboard: configure IUS repo (bug 1192338); r=fubar

We'll eventually want to install Python 2.7. Configure the IUS repo to
enable this.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/5240becf3191df26b2e53f79d2661e441f2cee30
changeset:  5240becf3191df26b2e53f79d2661e441f2cee30
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 10 11:39:45 2015 -0700
description:
ansible/hg-reviewboard: install v-c-t from public server (bug 1192338); r=fubar

Previously, we assumed v-c-t existed in a staging directory on the local
filesystem (from Docker). This assumption does not hold true on servers.
Install v-c-t from a public server when not running in Docker.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/196fc327b33c7e9c9bc59a39e1115e9e447870ee
changeset:  196fc327b33c7e9c9bc59a39e1115e9e447870ee
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 10 11:40:12 2015 -0700
description:
ansible/hg-reviewboard: remove unused pash features (bug 1192338); r=fubar

We have 2 versions of pash floating around. Over time, I'd like to
consolidate them. But that's a large task. This commit starts small
by removing unused features from reviewboard's copy of pash.

Support for the "pushlog" and "repo-group" commands have been removed.
These are or were used for replication and are not relevant to
reviewboard's Mercurial server.

Since we removed the "repo-group" command, repo_group.py no longer
serves a purpose and has been removed as well.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/26f3a48c663ca5b2e1cbf0a4e7c1d8628a5e0413
changeset:  26f3a48c663ca5b2e1cbf0a4e7c1d8628a5e0413
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 10 11:40:25 2015 -0700
description:
ansible/hg-reviewboard: clean up pash.py (bug 1192338); r=fubar

I'm going to try to have this file match the one used by hg.mozilla.org
as much as possible. Start by performing some basic whitespace cleanup
and by consolidating use of the USER environment variable.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/de1d6f6647703398750f901cec96678bae376da9
changeset:  de1d6f6647703398750f901cec96678bae376da9
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 10 11:40:38 2015 -0700
description:
ansible/hg-reviewboard: move QuoteForPOSIX into pash.py (bug 1192338); r=fubar

We don't want pash.py importing non-system modules because failure to
import could lead to inability to SSH into the machine! We move
QuoteForPOSIX into pash.py to avoid a module import.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/ae60916d30dc095c0ba4a3416c7656578dc0be97
changeset:  ae60916d30dc095c0ba4a3416c7656578dc0be97
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 10 11:40:55 2015 -0700
description:
ansible/hg-reviewboard: remove server port handling (bug 1192338); r=fubar

We are only ever listening on port 22. This code is not needed.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/eff01ca68c17fc6ddc0e4834ee4a558849051520
changeset:  eff01ca68c17fc6ddc0e4834ee4a558849051520
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 10 11:41:07 2015 -0700
description:
ansible/hg-reviewboard: move non-root handling to own function (bug 1192338); r=fubar

We move the processing of non-root logins to its own function so we can
delay import modules that may fail to import. This ensures that root
logins have minimal dependencies and are more robust against failures.

We also clean up some unused imports.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/657bb56cac549a8a9af03683f0c752cb168ee9db
changeset:  657bb56cac549a8a9af03683f0c752cb168ee9db
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 10 11:41:34 2015 -0700
description:
ansible/hg-reviewboard: use standard environment variable fetching (bug 1192338); r=fubar

os.environ is preferred over the low-level os.getenv() function.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/931e452672790ffc34a692581f4faddf29caa5f2
changeset:  931e452672790ffc34a692581f4faddf29caa5f2
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 10 11:41:47 2015 -0700
description:
ansible/hg-reviewboard: fix many style nits (bug 1192338); r=fubar

There should be no significant code changes in this commit. Just a lot
of style refactoring to act on warnings issued by flake8.

url:        https://hg.mozilla.org/hgcustom/version-control-tools/rev/4fb6656f1d9d25a21284bd6ab2fcb70a0872aaae
changeset:  4fb6656f1d9d25a21284bd6ab2fcb70a0872aaae
user:       Gregory Szorc <gps@mozilla.com>
date:       Mon Aug 10 11:42:02 2015 -0700
description:
ansible/hg-reviewboard: remove executability of pash files (bug 1192338); r=fubar

These files are Python support modules, not scripts. Remove the shebang,
any __main__ routines, and executable bit from files.
Priority: -- → P1
fubar removed most of the puppet bits for reviewboard-hg1 this morning. I performed a deploy and confirmed Ansible is able to manage the host. Huzzah!
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Product: Developer Services → MozReview
You need to log in before you can comment on or make changes to this bug.