1148966 - Use container based travis CI to improve test wait times

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Rail Aliiev [:rail]]

Sometimes it takes a lot of time to get Travis test coverage. 

Per http://docs.travis-ci.com/user/workers/container-based-infrastructure/:

* start up faster
* allow the use of caches for public repositories

Let's optimize a bit more!

Comment 1

[Rail Aliiev [:rail]]

Attachment: custom.diff

Comment 2

[Rail Aliiev [:rail]]

Attachment: faster_tests-tools.diff

Comment 3

[Rail Aliiev [:rail]]

Attachment: faster_tests-buildbot-configs.diff

* 3 chunks
* s/pep8/flake8/

Comment 4

[Rail Aliiev [:rail]]

All 3 patches are tested in my user repos

Comment 5

[Justin Wood (:Callek)]

Comment on attachment 8585230 [details] [diff] [review]
faster_tests-buildbot-configs.diff

Review of attachment 8585230 [details] [diff] [review]:
-----------------------------------------------------------------

::: .travis.yml
@@ +5,5 @@
> +sudo: false
> +
> +env:
> +  global:
> +    - PIP_FIND_LINKS=file://$HOME/.cache/pip

not sure I get why PIP_FIND_LINKS was needed... (even when looking at your git commit history and related travis jobs)

::: test-masters.sh
@@ +27,5 @@
> +    exit 1
> +  fi
> +}
> +
> +if [ "$1" == "--unittests-only" ]; then

please allow (with deprecation warning) "run-tests" as the trigger for this at least in the short term. (I think some of the stuff armen was working on was using it)

(I'm happy to be talked out of this opinion)

@@ +63,5 @@
>  FAILFILE=$(mktemp $WORK/tmp.failfile.XXXXXXXXXX)
>  atexit+=("rm $FAILFILE")
>  
>  # Construct the set of masters that we will test.
> +MASTERS=($(./setup-master.py -l -j "$MASTERS_JSON" --tested-only --error-logs))

"$@" was used to allow us to specify additional input, I don't think anyone actively used it, but mentioning incase it will matter.

::: tox.ini
@@ +45,5 @@
> +
> +[flake8]
> +exclude = .ropeproject,.tox
> +show-source = True
> +max-line-length=159

So I've been pretty much against flake8 (instead of seperate pyflakes and pep8 calls) until I read this hunk; including it in the tox.ini is *awesome* thanks.

Comment 6

[Justin Wood (:Callek)]

Comment on attachment 8585229 [details] [diff] [review]
faster_tests-tools.diff

Review of attachment 8585229 [details] [diff] [review]:
-----------------------------------------------------------------

r+ if you undo the comment-out of irc notifs

::: .travis.yml
@@ +7,3 @@
>  env:
> +  global:
> +    - PIP_FIND_LINKS=file://$HOME/.cache/pip

same find_links confusion

@@ +46,5 @@
> +    #template:
> +      #- "%{repository}#%{build_number} (%{branch} - %{commit} : %{author}): %{message}"
> +      #- "Change view : %{compare_url}"
> +      #- "Build details : %{build_url}"
> +      #- "Commit message : %{commit_message}"

undo this hunk

Comment 7

[Justin Wood (:Callek)]

Comment on attachment 8585228 [details] [diff] [review]
custom.diff

Review of attachment 8585228 [details] [diff] [review]:
-----------------------------------------------------------------

::: .travis.yml
@@ +5,5 @@
> +sudo: false
> +
> +env:
> +  global:
> +    - PIP_FIND_LINKS=file://$HOME/.cache/pip

again .. ;-)

Comment 8

[Rail Aliiev [:rail]]

(In reply to Justin Wood (:Callek) from comment #5)
> Comment on attachment 8585230 [details] [diff] [review]
> faster_tests-buildbot-configs.diff
> 
> Review of attachment 8585230 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: .travis.yml
> @@ +5,5 @@
> > +sudo: false
> > +
> > +env:
> > +  global:
> > +    - PIP_FIND_LINKS=file://$HOME/.cache/pip
> 
> not sure I get why PIP_FIND_LINKS was needed... (even when looking at your
> git commit history and related travis jobs)

This is used by "pip install tox" inside .travis.yml. Tox ignores environment variables AFAIK.

 
> ::: test-masters.sh
> @@ +27,5 @@
> > +    exit 1
> > +  fi
> > +}
> > +
> > +if [ "$1" == "--unittests-only" ]; then
> 
> please allow (with deprecation warning) "run-tests" as the trigger for this
> at least in the short term. (I think some of the stuff armen was working on
> was using it)

Let's ask Armen about this.

I'd prefer to kill the old option without any deprecation warning (written in bash, eeew).

> @@ +63,5 @@
> >  FAILFILE=$(mktemp $WORK/tmp.failfile.XXXXXXXXXX)
> >  atexit+=("rm $FAILFILE")
> >  
> >  # Construct the set of masters that we will test.
> > +MASTERS=($(./setup-master.py -l -j "$MASTERS_JSON" --tested-only --error-logs))
> 
> "$@" was used to allow us to specify additional input, I don't think anyone
> actively used it, but mentioning incase it will matter.

I didn't want to increase complexity of this script by separating its own options and options passed to to setup-master.py. This script should be used by CI only! :P 
> ::: tox.ini
> @@ +45,5 @@
> > +
> > +[flake8]
> > +exclude = .ropeproject,.tox
> > +show-source = True
> > +max-line-length=159
> 
> So I've been pretty much against flake8 (instead of seperate pyflakes and
> pep8 calls) until I read this hunk; including it in the tox.ini is *awesome*
> thanks.

Sweet! :)

Comment 9

[Rail Aliiev [:rail]]

Comment on attachment 8585228 [details] [diff] [review]
custom.diff

https://hg.mozilla.org/build/buildbotcustom/rev/6c2b489f2866

Comment 10

[Rail Aliiev [:rail]]

Comment on attachment 8585229 [details] [diff] [review]
faster_tests-tools.diff

https://hg.mozilla.org/build/tools/rev/e4be714c4944

Comment 11

[Rail Aliiev [:rail]]

Agreed on IRC.

Comment 12

[Rail Aliiev [:rail]]

Comment on attachment 8585230 [details] [diff] [review]
faster_tests-buildbot-configs.diff

https://hg.mozilla.org/build/buildbot-configs/rev/df468d1dc88d

Comment 13

[Jordan Lund (:jlund)]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/df468d1dc88d

Comment 14

[Jordan Lund (:jlund)]

In production: https://hg.mozilla.org/build/buildbotcustom/rev/6c2b489f2866