Closed Bug 419071 Opened 13 years ago Closed 12 years ago

update buildbot masters to use build slaves randomly

Categories

(Release Engineering :: General, defect, P2)

x86
Linux
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: anodelman, Assigned: rcampbell)

References

Details

Currently, builds are distributed to talos boxes sequentially (box 1 is always preferred, if it isn't available then box 2 is selected, etc).  This can mean that box 3, 4, etc are never touched for tests that run quicker than builds are generated.  We can also end up in bad states where box 1 is failing out quickly but then immediately served another build - consuming all available builds without managing to test anything.

I'd like to more equitably distribute builds to test boxes using this patch:

http://buildbot.net/trac/ticket/36
Assignee: nobody → anodelman
note that this will affect all other instances of buildbot on qm-rhel02, namely the unittest master and talos staging master (for now). This shouldn't be a problem.

Are we going to apply this to the production branch?
OS: Mac OS X → Linux
Blocks: 396574
I'm taking this.

I need to run this on the unittest staging environment for a day or so to make sure nothing breaks there.
Assignee: anodelman → rcampbell
Priority: -- → P2
Component: Testing → Release Engineering
Product: Core → mozilla.org
Whiteboard: running on qm-unittest02, apply to rhel02 on monday
Version: unspecified → other
Component: Release Engineering → Release Engineering: Maintenance
QA Contact: testing → release
somehow I missed installing this on qm-buildbot01. Below is the console output from a successful install.

[root@qm-buildbot01 buildbot]# patch -p0 <random.patch 
patching file buildbot/process/builder.py
Hunk #1 succeeded at 1 with fuzz 2.
Hunk #2 succeeded at 227 (offset 2 lines).
[root@qm-buildbot01 buildbot]# python setup.py build
running build
running build_py
copying buildbot/process/builder.py -> build/lib/buildbot/process
running build_scripts
[root@qm-buildbot01 buildbot]# python setup.py install
running install
running build
running build_py
running build_scripts
running install_lib
copying build/lib/buildbot/process/builder.py -> /tools/python/lib/python2.5/site-packages/buildbot/process
byte-compiling /tools/python/lib/python2.5/site-packages/buildbot/process/builder.py to builder.pyc
/tools/python/lib/python2.5/site-packages/buildbot/process/builder.py:386: Warning: 'with' will become a reserved keyword in Python 2.6
/tools/python/lib/python2.5/site-packages/buildbot/process/builder.py:387: Warning: 'with' will become a reserved keyword in Python 2.6
/tools/python/lib/python2.5/site-packages/buildbot/process/builder.py:388: Warning: 'with' will become a reserved keyword in Python 2.6
/tools/python/lib/python2.5/site-packages/buildbot/process/builder.py:392: Warning: 'with' will become a reserved keyword in Python 2.6
running install_scripts
changing mode of /tools/python/bin/buildbot to 775
running install_data
running install_egg_info
Removing /tools/python/lib/python2.5/site-packages/buildbot-0.7.5-py2.5.egg-info
Writing /tools/python/lib/python2.5/site-packages/buildbot-0.7.5-py2.5.egg-info
[root@qm-buildbot01 buildbot]# 


restarting the buildbot master. Should see builds coming up on random machines now. Hopefully.

Can we get this installed into BUILDBOT_MOZILLA_PRODUCTION?
Whiteboard: running on qm-unittest02, apply to rhel02 on monday → running on qm-unittest02, apply to rhel02 ASAP
Blocks: 422328
This seems like it would be useful for all the masters used by build also. Can we also get this installed on staging-1.8-master, staging-1.9-master, production-1.8-master, production-1.9-master? 
Summary: update buildbot on qm-rhel02 to use build slaves randomly → update buildbot masters to use build slaves randomly
(In reply to comment #5)
> This seems like it would be useful for all the masters used by build also. Can
> we also get this installed on staging-1.8-master, staging-1.9-master,
> production-1.8-master, production-1.9-master? 
> 

AFAIK all of these only have one slave right now. Even with two, dep builds would still only alternate between the builders (because they build continuously). It's a bit of a PITA to upgrade Buildbot right now so we may as well wait until we upgrade Buildbot on these to pick this up.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Whiteboard: running on qm-unittest02, apply to rhel02 ASAP
Component: Release Engineering: Maintenance → Release Engineering
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.