1148528 - Add buildbot-configs test to prevent removing build builders without removing their associated testers

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Armen [:armenzg]]

Attachment: test.diff

In this pull request [1] you can see a run before my two buildbot-configs changes [2].
In [2] you will see that 402 builders are without their much needed build builders.
In [3] you can see the test pass after my buildbot-configs which fixes the situation.

[1] https://github.com/mozilla/build-buildbot-configs/pull/1
[2] https://travis-ci.org/mozilla/build-buildbot-configs/builds/56120390
[3] https://travis-ci.org/mozilla/build-buildbot-configs/builds/56132383

Comment 1

[Rail Aliiev [:rail]]

Comment on attachment 8584714 [details] [diff] [review]
test.diff

Review of attachment 8584714 [details] [diff] [review]:
-----------------------------------------------------------------

::: test/test_no_testers_without_builders.py
@@ +13,5 @@
> +    # before running this test rather than fetching the one from the server
> +    # This is important as we want to test against the latest list of builders
> +    assert os.path.exists("allthethings.json")
> +
> +    list = []

This masks the built-in list(). Can you changes it to something else? builder_list maybe

@@ +24,5 @@
> +        if determine_upstream_builder(builder) is None:
> +            list.append(builder)
> +
> +    assert len(list) == 0, \
> +        "There are downstream builders without upstream builders to trigger them."

Can you dump the list of builders, something like:

"There are downstream builders without upstream builders to trigger them.\n%s" % "\n".join(builder_list)

It would be easier to find offenders without looking at the code.

Comment 2

[Armen [:armenzg]]

> > +
> > +    assert len(list) == 0, \
> > +        "There are downstream builders without upstream builders to trigger them."
> 
> Can you dump the list of builders, something like:
> 
> "There are downstream builders without upstream builders to trigger
> them.\n%s" % "\n".join(builder_list)
> 
> It would be easier to find offenders without looking at the code.

If you look at the Travis log you can actually see the offenders:
>       assert len(list) == 0, \
            "There are downstream builders without upstream builders to trigger them."
E       AssertionError: There are downstream builders without upstream builders to trigger them.
E       assert 402 == 0
E        +  where 402 = len(['Ubuntu Code Coverage VM 12.04 x64 try debug test cppunit', 'Ubuntu Code Coverage VM 12.04 x64 try debug test crashte...ode Coverage VM 12.04 x64 try debug test jittest-2', 'Ubuntu Code Coverage VM 12.04 x64 try debug test jsreftest', ...])
test/test_no_testers_without_builders.py:27: AssertionError
=========================== 1 failed in 1.03 seconds ===========================
ERROR: InvocationError: '/home/travis/build/mozilla/build-buildbot-configs/.tox/py27/bin/py.test test'
___________________________________ summary ____________________________________
ERROR:   py27: commands failed
The command "tox" exited with 1.

Comment 4

[Armen [:armenzg]]

Attachment: test.diff

I dealt with the variable name.

Comment 5

[Armen [:armenzg]]

http://hg.mozilla.org/build/buildbot-configs/rev/05c0f42e3e67

Comment 6

[Armen [:armenzg]]

Closing. If it fails we will re-open:
https://travis-ci.org/mozilla/build-buildbot-configs/builds/56405588

Thanks rail!

Comment 7

[Jordan Lund (:jlund)]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/05c0f42e3e67