1205316 - Update the mysql used on Travis to v5.6

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P2)

Description

[Ed Morley [:emorley]]

Bug 1204918 updated the version of mysql used in our Vagrant environment to v5.6, for parity with stage/prod, but also so we could start using the fulltext search capabilities of mysql 5.6+.

Unfortunately Travis currently only natively supports mysql 5.5.

In order for us to install it manually ourselves, we either have to:
a) Use sudo mode, which means we have to switch back to the legacy (slower) infrastructure & also lose caching mode (negating the benefits of bug 1137278).
b) use their "APT" addon feature, which allows for installation of whitelisted packages using the steps at http://docs.travis-ci.com/user/apt/

The latter option is much preferred, and requires us to submit a whitelist request, following the steps at:
https://github.com/travis-ci/apt-package-whitelist#package-approval-process

Hopefully this should be no problem, since I already see other mysql versions listed in:
https://github.com/travis-ci/apt-package-whitelist/blob/master/ubuntu-precise

I've opened the request:
https://github.com/travis-ci/apt-package-whitelist/issues/1206

And if/when it's approved, I'll update travis.yml appropriately.

Comment 1

[William Lachance (:wlach)]

Apparently because some of the packages we want have a suid bit, we're in some kind of queue to get whitelisted:

https://github.com/travis-ci/apt-package-whitelist/pull/1234

Looking at other PR's (https://github.com/travis-ci/apt-package-whitelist/pulls), some have been waiting for quite some time.

Comment 2

[Ed Morley [:emorley]]

Mauro can work around this using RunSQL (I think we mentioned in the meeting last week we'd wait a few days to see if Travis responded, then fall back to using RunSQL).

Comment 3

[William Lachance (:wlach)]

Still no response from travis, can we do the runsql solution please? The lack of a dedicated testing database is a major roadblock for me.

Comment 4

[Mauro Doglio [:mdoglio]]

I tried to make fix it with a workaround a couple of weeks ago but I was not able to make green run on travis. Here is the pull request I was working on: https://github.com/mozilla/treeherder/pull/968
I'll see if I can spend some time on it later this week. Feel free to pick it up if you have some free cycles.

Comment 5

[David Burns :automatedtester]

I have spoken to Josh (founder at Travis) and he will get someone on to this.

Comment 6

[Ed Morley [:emorley]]

So the APT package whitelist PR has been merged, however it doesn't work due to a package conflict:
https://github.com/travis-ci/apt-package-whitelist/issues/1206#issuecomment-149884653

The way forwards are:
1) Wait until that conflict is resolved (unlikely)
2) Use the beta trusty image (http://docs.travis-ci.com/user/trusty-ci-environment/)
3) Use the legacy infra

Both #2 and #3 don't support caching, and require us to still install packages manually, which adds quite a bit to the CI runtime.

I have a WIP for #2 in progress - since at least that feels like we're moving forwards (and caching might be added in the future, whereas the legacy infra is EOL) - so we can see what the full end-to-end time impact is, and thus whether we go with that or the RunSQL approach to manually add the MyIASM table.

Comment 7

[Ed Morley [:emorley]]

Attachment: Use the beta Trusty infra & update to mysql 5.6

So this ended up being a bit of a PITA, since there are some strange quirks with the beta Trusty infra (it was only announced two days ago, so there are missing features and the docs are suboptimal), but it works now at least.

This causes an approx 3 minute setup time penalty before anything (including flake8 etc) runs, which is approx an increase of 30% of a typical test run (ignoring the recently regression which I need to file a bug for).

Example run:
https://travis-ci.org/mozilla/treeherder/builds/86681543

Comment 8

[William Lachance (:wlach)]

Comment on attachment 8677026 [details] [review]
Use the beta Trusty infra & update to mysql 5.6

Patch looks fine, I'll leave the decision to merge up to you. I don't think slightly longer wait times are that big a deal, personally.

Comment 9

[Ed Morley [:emorley]]

I meant to add:

Given that stage/prod/Vagrant now all use InnoDB for that table (reminds me: we must check Heroku) I'd prefer to keep parity with it if possible, so I think the 3 minute penalty is worth it.

Comment 10

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/f0aac8884aff63a387840ad3e71107aeb81a7721
Bug 1205316 - Travis: Use the beta Trusty infra & update to mysql 5.6

Stage/production/Vagrant/Heroku's RDS all use mysql 5.6, however Travis
is currently running v5.5. Installing mysql 5.6 manually on the Travis
container infra is currently broken:
https://github.com/travis-ci/apt-package-whitelist/issues/1206#issuecomment-149884653

To use sudo (for apt-get) we either have to fall back to the legacy
non-container infra, or else use the new Trusty beta infra:
http://blog.travis-ci.com/2015-10-14-opening-up-ubuntu-trusty-beta/
http://docs.travis-ci.com/user/trusty-ci-environment/#Runtimes

The Trusty beta infra is also non-container, but at least isn't EOL.
Unfortunately similar to the legacy non-container infra, it doesn't
offer caching, so incurs a setup time penalty of approx 3 minutes
(including the mysql 5.6 install, npm install and peep install). See:
https://github.com/travis-ci/travis-ci/issues/4997

If/when the container infra uses mysql 5.6 by default, or the bug
preventing installing it using the apt travis.yml option is fixed, we
should switch back to the container infra, to speed up the Travis run.

In this commit, we use `--user` with the peep install, since the
non-container infra doesn't set up a virtualenv, and we cannot use sudo
due to:
https://github.com/travis-ci/travis-ci/issues/4989

In addition, the current user ("travis") now doesn't have permissions to
create the Treeherder DB, so we have to use `-u root` (the password is
blank).

The new infra is running Python 2.7.10 (rather than the v2.7.9 of the
container infra) which now matches what runs in the Vagrant environment.
In future bugs we should update stage/prod and Heroku to 2.7.10 too.