Clean up the Vagrant puppet scripts



2 years ago
2 years ago


(Reporter: emorley, Assigned: emorley)




(1 attachment)



2 years ago
Whilst working on bug 1289156 I spotted a few things that could do with fixing:
* there are a number of tasks performed that really aren't necessary and just add to the provision time / puppet script complexity unnecessarily
* the environment variables / bashrc additions are hardcoded in the configs as numerous puppet steps, when having them in an external file would be clearer
* the environment variables / profile scripts are copied in rather than symlinked, so don't get updated until someone re-runs provision manually (this would be a good start towards fixing bug 1196725)
* the entire default mysql config is overwritten rather than just adding a separate config that adjusts the settings we use that are non-default

...and so on.
Created attachment 8782133 [details] [review]
[treeherder] mozilla:puppet-cleanup > mozilla:master


2 years ago
Attachment #8782133 - Flags: review?(cdawson)
Comment on attachment 8782133 [details] [review]
[treeherder] mozilla:puppet-cleanup > mozilla:master

Whew!  That was a lot of work.  thanks for doing this.  :)
Attachment #8782133 - Flags: review?(cdawson) → review+

Comment 3

2 years ago
Commits pushed to master at
Bug 1296047 - Vagrant: Stop creating a 'puppet' group during provision

Since it's unused.
Bug 1296047 - Vagrant: Use the default rabbitmq guest user

A default user exists with username 'guest' and password 'guest', which
we might as well use to save having to create another and set up grants.

Bug 1296047 - Vagrant: Avoid repetition in DATABASE_URL_RO

By using django-environ's proxied environment variable feature, like is
already being used in Heroku's value for `DATABASE_URL_RO`. See:
Bug 1296047 - Vagrant: Use the default root mysql user

A default user exists with username 'root' and blank password, which we
might as well use to save having to create another. We still have to add
a grant to allow root to connect from outside the VM, since the default
grant of `root@localhost` only allows connections via the loopback

The dependency on the `create-db` task has also been removed, since the
grant uses a wildcard, so doesn't refer to the `treeherder` DB directly.
Bug 1296047 - Vagrant: Move environment variables to their own file

Having them in a separate file is cleaner, makes the discovering where
the environment variables are set easier, plus means we can symlink the
file, so future variable changes will take effect immediately, rather
than needing a re-provision after pulling latest master.
Bug 1296047 - Vagrant: Move helper functions to a symlinked .profile

This renames the existing `.bash_aliases` file to `.profile`, since
we're soon going to use it for more than just aliases. It overwrites the
default `.profile` file in the VM, so we need to source `.bashrc` as the
original did.

In addition, rather than copying the file we now symlink it, so that
future changes don't require a re-provision after pulling latest master
to take effect.
Bug 1296047 - Vagrant: Move .bashrc additions to .profile

Previously provision would append entries to .bashrc in multiple steps,
whereas now these are just included in the `.profile` that is symlinked
from the environment. As such, future changes will no longer need a
re-provision after pulling latest master, to take effect.
Bug 1296047 - Vagrant: Merge the two sample django settings files

Previously there were two sample Django config files, and confusingly
the one that would be used in the Vagrant environment wasn't the one
that was the most visible.

In addition, we're not performing any kind of variable substitution, so
don't need to use `content => template()`.
Bug 1296047 - Vagrant: Combine the directory creation steps
Bug 1296047 - Vagrant: Combine the two pip install steps

Since the dev.txt packages no longer depend on system packages installed
in mysql.pp, so don't need to be installed separately. The working
directory has also been adjusted to avoid the need to specify the full
path to the requirements files.
Bug 1296047 - Vagrant: Remove redundant fully qualified paths

The path used for `exec{}` commands (defined in `vagrant.pp`) includes
`${VENV_DIR}/bin` so we don't need to specify the full path for pip
and python invocations that are meant to use the virtualenv binaries.
Bug 1296047 - Vagrant: Only override specific MySQL server preferences

Rather than overwriting the default MySQL 5.6 config file, the specific
changes we wish to make are now made via a file in the `mysql/conf.d/`
include directory. This makes it easier to see where we differ from the
defaults, as well as preventing us from inadvertently overriding any
new defaults in `/etc/mysql/my.cnf` when we update to new MySQL major
versions in the future.

The contents of this new file were determined by diffing against the
untouched `/etc/mysql/my.cnf` file, and removing anything that was still
set to defaults.
Bug 1296047 - Travis: Use the same MySQL config file as Vagrant

To reduce duplication and ensure the configurations remain in sync.

Note: The Vagrant config does set `bind-address` to allow non-localhost
connections, which isn't necessary on Travis, however this is fine since
(a) it's only Travis, (b) the user grants on Travis won't actually allow
non-localhost connections anyway, even if Travis' network settings
allowed connections between test nodes.


2 years ago
Last Resolved: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.