Closed
Bug 816237
Opened 12 years ago
Closed 12 years ago
Set up staging mozpool of pandas
Categories
(Infrastructure & Operations Graveyard :: CIDuty, task, P1)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: armenzg, Assigned: armenzg)
References
Details
Attachments
(5 files, 6 obsolete files)
10.31 KB,
patch
|
armenzg
:
review+
armenzg
:
checked-in+
|
Details | Diff | Splinter Review |
3.62 KB,
patch
|
mozilla
:
review+
armenzg
:
checked-in+
|
Details | Diff | Splinter Review |
1.15 KB,
patch
|
mozilla
:
review+
armenzg
:
checked-in+
|
Details | Diff | Splinter Review |
1.20 KB,
patch
|
Callek
:
review+
armenzg
:
checked-in+
|
Details | Diff | Splinter Review |
10.52 KB,
patch
|
Callek
:
review+
armenzg
:
checked-in+
|
Details | Diff | Splinter Review |
foopy[39-45] panda-0[082-169] FTR we're aiming to move away from clientproxy.py if possible and just start buildbot. If any imaging fails then buildbot should be shutdown gracefully.
Assignee | ||
Updated•12 years ago
|
Assignee: nobody → armenzg
Priority: -- → P1
Assignee | ||
Comment 1•12 years ago
|
||
Here's my current distribution: foopy39.p1.releng.scl1.mozilla.com - panda-00[82-94] foopy40.p1.releng.scl1.mozilla.com - panda-0[095-107] foopy41.p1.releng.scl1.mozilla.com - panda-0[108-120] foopy42.p1.releng.scl1.mozilla.com - panda-0[121-133] foopy43.p1.releng.scl1.mozilla.com - panda-0[134-146] foopy44.p1.releng.scl1.mozilla.com - panda-0[147-159] foopy45.p1.releng.scl1.mozilla.com - panda-0[160-169] I will try tomorrow to see if I can make use of this version of devices.json that does not keep track of the relayhost and relayid: http://hg.mozilla.org/users/armenzg_mozilla.com/tools/rev/8f6df37b8282
Assignee | ||
Comment 2•12 years ago
|
||
Attachment #686305 -
Flags: review?(jhopkins)
Assignee | ||
Comment 3•12 years ago
|
||
I have one of the new pandas connected: http://dev-master01.build.scl1.mozilla.com:8042/buildslaves/panda-0082 # as root mkdir /builds/panda-0082 chown cltbld /builds/panda-0082 # as cltbld wget -O/builds/panda-0082/buildbot.tac http://slavealloc.build.mozilla.org/gettac/panda-0082 /tools/buildbot-0.8.4-pre-moz2/bin/buildslave start /builds/panda-0082 I wanted to use this twistd command but it was failing [1]: /tools/buildbot-0.8.4-pre-moz2/bin/python2.7 /tools/buildbot-0.8.4-pre-moz2/bin/twistd --no_save --rundir=/builds/panda-0082 --pidfile=/builds/panda-0082/twistd.pid --python=/builds/panda-0082/buildbot.tac I don't know why it would want to put the twistd.log under /builds rather than /builds/panda-0082. [1] [cltbld@foopy39 builds]$ /tools/buildbot-0.8.4-pre-moz2/bin/python2.7 /tools/buildbot-0.8.4-pre-moz2/bin/twistd --no_save --rundir=/builds/panda-0082 --pidfile=/builds/panda-0082/twistd.pid --python=/builds/panda-0082/buildbot.tac Traceback (most recent call last): File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/application/app.py", line 631, in run runApp(config) File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/scripts/twistd.py", line 23, in runApp _SomeApplicationRunner(config).run() File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/application/app.py", line 374, in run self.application = self.createOrGetApplication() File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/application/app.py", line 439, in createOrGetApplication application = getApplication(self.config, passphrase) --- <exception caught here> --- File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/application/app.py", line 450, in getApplication application = service.loadApplication(filename, style, passphrase) File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/application/service.py", line 400, in loadApplication application = sob.loadValueFromFile(filename, 'application', passphrase) File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/persisted/sob.py", line 210, in loadValueFromFile exec fileObj in d, d File "/builds/panda-0082/buildbot.tac", line 27, in <module> maxRotatedFiles=maxRotatedFiles) File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/python/logfile.py", line 49, in fromFullPath os.path.dirname(logPath), *args, **kwargs) File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/python/logfile.py", line 162, in __init__ BaseLogFile.__init__(self, name, directory, defaultMode) File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/python/logfile.py", line 41, in __init__ self._openFile() File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/python/logfile.py", line 167, in _openFile BaseLogFile._openFile(self) File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/python/logfile.py", line 76, in _openFile self._file = file(self.path, "w+", 1) exceptions.IOError: [Errno 13] Permission denied: '/builds/twistd.log' Failed to load application: [Errno 13] Permission denied: '/builds/twistd.log'
Comment 4•12 years ago
|
||
(In reply to Armen Zambrano G. [:armenzg] from comment #3) > I wanted to use this twistd command but it was failing [1]: > /tools/buildbot-0.8.4-pre-moz2/bin/python2.7 > /tools/buildbot-0.8.4-pre-moz2/bin/twistd --no_save > --rundir=/builds/panda-0082 --pidfile=/builds/panda-0082/twistd.pid > --python=/builds/panda-0082/buildbot.tac > > I don't know why it would want to put the twistd.log under /builds rather > than /builds/panda-0082. My guess was basedir in the .tac being /builds but currently its not, have you tried any of the following: * having `pwd` be /builds/panda-0082 when you launch the twistd command * adding a / to the basedir/rundir param/options * doing a more permissive chmod on /builds/panda-0082 (rather than just chown to cltbld) * doing a chgrp on /builds/panda-0082
Assignee | ||
Comment 5•12 years ago
|
||
From inside of /builds/panda-0082 seems to do the job (thanks!): /tools/buildbot-0.8.4-pre-moz2/bin/python2.7 /tools/buildbot-0.8.4-pre-moz2/bin/twistd --no_save --rundir=/builds/panda-0082 --pidfile=/builds/panda-0082/twistd.pid --python=/builds/panda-0082/buildbot.tac
Comment 6•12 years ago
|
||
Comment on attachment 686305 [details] [diff] [review] attach staging pandas to armenzg's master armenzg says: 133 is my test master #2 and poolid=16 is "preprod-tests" Looks good to me.
Attachment #686305 -
Flags: review?(jhopkins) → review+
Assignee | ||
Comment 7•12 years ago
|
||
This does what I need. jhopkins suggested to get a way to check that the bmm is up before starting buildbot.
Assignee | ||
Comment 8•12 years ago
|
||
After lunch I will automate the creation of the /builds/panda-####.
Assignee | ||
Comment 9•12 years ago
|
||
It is not complete yet but it shows what way I'm heading with.
Updated•12 years ago
|
Attachment #686796 -
Attachment mime type: text/x-python → text/plain
Assignee | ||
Comment 10•12 years ago
|
||
I used a bash script because make was not available on the machine. We can store the bmm value as an environment variable if prefered (in ~/.bash_profile or so). These are the steps to reproduce and below the output to show what gets created. # as root runuser -l cltbld -c "hg pull -u /builds/tools" cd /builds/tools/buildfarm/mobile/ python setup_foopy.py ## Output to show what gets created ##################################### [root@foopy39 mobile]# diff /builds/panda-0082/buildbot.tac /builds/panda-0083/buildbot.tac 14c14 < basedir = '/builds/panda-0082' --- > basedir = '/builds/panda-0083' 16c16 < slavename = 'panda-0082' --- > slavename = 'panda-0083' [root@foopy39 mobile]# find /builds/panda-00* /builds/panda-0082 /builds/panda-0082/make.sh /builds/panda-0082/buildbot.tac /builds/panda-0083 /builds/panda-0083/make.sh /builds/panda-0083/buildbot.tac /builds/panda-0084 /builds/panda-0084/make.sh /builds/panda-0084/buildbot.tac /builds/panda-0085 /builds/panda-0085/make.sh /builds/panda-0085/buildbot.tac /builds/panda-0086 /builds/panda-0086/make.sh /builds/panda-0086/buildbot.tac /builds/panda-0087 /builds/panda-0087/make.sh /builds/panda-0087/buildbot.tac /builds/panda-0088 /builds/panda-0088/make.sh /builds/panda-0088/buildbot.tac /builds/panda-0089 /builds/panda-0089/make.sh /builds/panda-0089/buildbot.tac /builds/panda-0090 /builds/panda-0090/make.sh /builds/panda-0090/buildbot.tac /builds/panda-0091 /builds/panda-0091/make.sh /builds/panda-0091/buildbot.tac /builds/panda-0092 /builds/panda-0092/make.sh /builds/panda-0092/buildbot.tac /builds/panda-0093 /builds/panda-0093/make.sh /builds/panda-0093/buildbot.tac /builds/panda-0094 /builds/panda-0094/make.sh /builds/panda-0094/buildbot.tac [root@foopy39 mobile]# cat /builds/bmm.hostname.txt mobile-imaging-001.p1.releng.scl1.mozilla.com
Attachment #686631 -
Attachment is obsolete: true
Attachment #686796 -
Attachment is obsolete: true
Attachment #687097 -
Flags: review?(bugspam.Callek)
Attachment #687097 -
Flags: review?(aki)
Assignee | ||
Comment 11•12 years ago
|
||
I modified the sql statement a bit and landed it. For every line I added this: ", basedir='/builds/panda-00##' " (e.g. panda-0082) I can now start up all the buildbot instances of foopy39 ( I will skip the panda-00[82-89] since jhopkins holds them).
Attachment #686305 -
Attachment is obsolete: true
Attachment #687105 -
Flags: review+
Attachment #687105 -
Flags: checked-in+
Assignee | ||
Comment 12•12 years ago
|
||
This file deals with proper ownership of the files and directories. The following output shows that from a known state we can get to a working state. [root@foopy39 mobile]# rm -rf /builds/panda-00* [root@foopy39 mobile]# python setup_foopy.py [cltbld@foopy39 builds]$ for i in {90..94}; do cd /builds/panda-00$i; ./make.sh start; done 2012-11-30 08:26:08-0800 [-] Log opened. 2012-11-30 08:26:08-0800 [-] twistd 10.2.0 (/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 2.7.2) starting up. 2012-11-30 08:26:08-0800 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2012-11-30 08:26:08-0800 [-] Log opened. 2012-11-30 08:26:08-0800 [-] twistd 10.2.0 (/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 2.7.2) starting up. 2012-11-30 08:26:08-0800 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2012-11-30 08:26:08-0800 [-] Log opened. 2012-11-30 08:26:08-0800 [-] twistd 10.2.0 (/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 2.7.2) starting up. 2012-11-30 08:26:08-0800 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2012-11-30 08:26:08-0800 [-] Log opened. 2012-11-30 08:26:08-0800 [-] twistd 10.2.0 (/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 2.7.2) starting up. 2012-11-30 08:26:08-0800 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2012-11-30 08:26:08-0800 [-] Log opened. 2012-11-30 08:26:08-0800 [-] twistd 10.2.0 (/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 2.7.2) starting up. 2012-11-30 08:26:08-0800 [-] reactor class: twisted.internet.selectreactor.SelectReactor. [cltbld@foopy39 panda-0094]$ for i in {90..94}; do cd /builds/panda-00$i; ./make.sh stop; done buildslave process 23286 is dead buildslave process 23293 is dead buildslave process 23302 is dead buildslave process 23309 is dead buildslave process 23318 is dead
Attachment #687097 -
Attachment is obsolete: true
Attachment #687097 -
Flags: review?(bugspam.Callek)
Attachment #687097 -
Flags: review?(aki)
Attachment #687113 -
Flags: review?(bugspam.Callek)
Attachment #687113 -
Flags: review?(aki)
Assignee | ||
Comment 13•12 years ago
|
||
I have setup foopies 40, 43, 44 & 45 like this: # remove previous existing directories owned by root rm -rf /builds/panda-0* runuser -l cltbld -c "wget -q https://bugzilla.mozilla.org/attachment.cgi?id=687113" # otherwise the files will be owned by root runuser -l cltbld -c "cd /builds/tools && patch -p1 < ~/attachment.cgi?id=687113" cd /builds/tools/buildfarm/mobile/ # odd - the patch says "new file mode 100755" chmod 755 make.sh runuser -l cltbld -c "hg pull -u /builds/tools" python setup_foopy.py su cltbld for i in `ls /builds | grep panda-`; do cd /builds/$i; ./make.sh start; done
Assignee | ||
Comment 14•12 years ago
|
||
jhopkins is going to try to image and run the cat command from his repo manually. We have to figure out how to execute a cat command remotely (we're hoping Callek can lead us in the right direction). armenzg will be polishing his buildbot-configs patches and BuildSlaves templates.
Comment 15•12 years ago
|
||
Comment on attachment 687113 [details] [diff] [review] setup_foopy.py, updated devices.json and make.sh I'm not sure if we want a different target in make.sh to somehow mark the buildslave/device as bad (stop-and-email or stop-and-something-else) or if mozharness should do that and just run make.sh stop. Other than that I'm mostly concerned about, "does this work?" and by your testing it looks like it does.
Attachment #687113 -
Flags: review?(aki) → review+
Assignee | ||
Comment 16•12 years ago
|
||
Locally I'm running it with this: MOZHARNESS_REPO = "http://hg.mozilla.org/users/armenzg_mozilla.com/mozharness" This cannot land without mozpool and b2g_panda.py changes landing on mozharness first. BTW I have made --mozpool-device not mandatory. Unfortunately I had to use "print response" rather than "self.log(response)" since it said something about splitlines. You can check results in: http://dev-master01.build.scl1.mozilla.com:8042/builders/b2g_b2g_panda%20cedar%20opt%20test%20gaia-mochitest
Attachment #687239 -
Flags: review?(aki)
Assignee | ||
Comment 17•12 years ago
|
||
I know that the naming does not follow the example of panda_android (arch_os) but whatever.
Attachment #687245 -
Flags: review?(aki)
Assignee | ||
Comment 18•12 years ago
|
||
Attachment #687247 -
Flags: review?(aki)
Assignee | ||
Comment 19•12 years ago
|
||
Interestingly enough my scripts shows some weirdness about pings: http://mobile-imaging-001.p1.releng.scl1.mozilla.com/ui/log.html?device=panda-0100 I don't know when the board is done rebooting and meanwhile there is inconsistency with pings. http://mobile-imaging-001.p1.releng.scl1.mozilla.com/ui/log.html?device=panda-0100 FTR, if anyone wants to trigger more jobs while I'm away you can do the following: * land a change to the b2gpanda branch on my user repo * rebuild the previous job OR force build with "script_repo_revision":"b2gpanda"
Assignee | ||
Comment 20•12 years ago
|
||
FTR my patches only add the test builders but there are no sendchanges to trigger them: https://tbpl.mozilla.org/php/getParsedLog.php?id=17492455&tree=Firefox&full=1 I will be working on that side of it on Monday. All the patches can land even though without the mozharness patches the manually triggered jobs would fail (until we land our changes on mozharness).
Updated•12 years ago
|
Attachment #687247 -
Flags: review?(aki) → review+
Updated•12 years ago
|
Attachment #687245 -
Flags: review?(aki) → review+
Comment 21•12 years ago
|
||
Comment on attachment 687245 [details] [diff] [review] BuildSlaves Templates in buildbot-configs > 'mock': 'pass' >+ 'b2g_panda': 'pass' You'll need a comma after the mock entry in mozilla/BuildSlaves.py.template. Putting a comma after both lines may help avoid this in the future.
Comment 22•12 years ago
|
||
(In reply to Armen Zambrano G. [:armenzg] from comment #16) > Unfortunately I had to use "print response" rather than "self.log(response)" > since it said something about splitlines. Try self.info(str(response)) to both make the log level INFO, and ensure that response is in string format.
Comment 23•12 years ago
|
||
Comment on attachment 687239 [details] [diff] [review] buildbot-config changes to add b2g dummy job +PLATFORMS['b2g_panda']['slave_platforms'] = ['b2g_panda'] +PLATFORMS['b2g_panda']['env_name'] = None Nit: Could you remove the trailing whitespace here? +BRANCHES['cedar']['platforms']['b2g_panda']['b2g_panda']['opt_unittest_suites'] = [ + ('gaia-mochitest', {'suite': 'gaia-mochitest', + 'mozharness_repo': MOZHARNESS_REPO, Also trailing whitespace here.
Attachment #687239 -
Flags: review?(aki) → review+
Comment 24•12 years ago
|
||
Armen, I got this failure when running test-masters.sh: test_stag_not_in_prod ... [FAIL] =============================================================================== [FAIL] Traceback (most recent call last): File "test/test_slave_allocation.py", line 33, in test_stag_not_in_prod 'declared as staging-only:\n%s' % '\n'.join(sorted(common_slaves)) File "/Users/asasaki/wrk/virtualenv/bb08/lib/python2.6/site-packages/Twisted-12.1.0-py2.6-macosx-10.6-x86_64.egg/twisted/trial/unittest.py", line 270, in assertEqual % (msg, pformat(first), pformat(second))) twisted.trial.unittest.FailTest: Staging-only slaves should not be declared as production and vice versa. However, the following production slaves declared as staging-only: I think we might be adding production slaves to staging automatically. Just wanted to highlight this since preprod will start whining if you land as-is.
Comment 25•12 years ago
|
||
Comment on attachment 687247 [details] [diff] [review] BuildSlaves-tests.py.erb - puppet changes for the masters Review of attachment 687247 [details] [diff] [review]: ----------------------------------------------------------------- ::: modules/buildmaster/templates/BuildSlaves-tests.py.erb @@ +21,5 @@ > 'tegra_android-armv6': '<%=scope.lookupvar('secrets::tegra_android_tests_password')%>', > 'tegra_android-noion': '<%=scope.lookupvar('secrets::tegra_android_tests_password')%>', > 'fedora-b2g': '<%=scope.lookupvar('secrets::linux_tests_password')%>', > 'panda_android': '<%=scope.lookupvar('secrets::panda_android_tests_password')%>', > + 'b2g_panda': '<%=scope.lookupvar('secrets::panda_android_tests_password')%>', please can we use a new secret here? I don't like the idea of reusing "panda_*android*...." for b2g pandas.
Attachment #687247 -
Flags: feedback-
Comment 26•12 years ago
|
||
(In reply to Armen Zambrano G. [:armenzg] from comment #10) > I used a bash script because make was not available on the machine. Not needed to worry about, but its simple to add make to Linux foopies if we want :-) > We can store the bmm value as an environment variable if prefered (in > ~/.bash_profile or so). IMO its far harder to change (for the system) the bmm value if we store it in .bash_profile since foopies don't reboot often, but easier to discover in running jobs (since it will show in buildbot output) end-result I'm indifferent on that (In reply to Armen Zambrano G. [:armenzg] from comment #14) > jhopkins is going to try to image and run the cat command from his repo > manually. > We have to figure out how to execute a cat command remotely (we're hoping > Callek can lead us in the right direction). cat of the remote devices filesystem, or do a cat of a file on the foopy remotely? from wher and why? (I'm not sure what you're trying to achieve, so advising is hard) (In reply to Aki Sasaki [:aki] from comment #15) > Comment on attachment 687113 [details] [diff] [review] > > I'm not sure if we want a different target in make.sh to somehow mark the > buildslave/device as bad (stop-and-email or stop-and-something-else) or if > mozharness should do that and just run make.sh stop. IMHO mozharness marking the device bad in whatever way is accurate is probably best, bmm itself in some way should probably be responsible for the notification sendout somehow (as in, *IT* can try to recover in the way[s] we teach it, if it fails then it notifies us, mozharness/foopies should just be responsible in telling bmm/lifeguard/whatever that the device can't swim) - The "lifeguard" can then call the "ambulance" (us/relops/dcops) if it needs help.
Comment 27•12 years ago
|
||
Comment on attachment 687113 [details] [diff] [review] setup_foopy.py, updated devices.json and make.sh Review of attachment 687113 [details] [diff] [review]: ----------------------------------------------------------------- All the below is nits/questions, and overall this looks great! I marked it r- since I need the questions answered to give it a proper final review (and want them before we land), so when you answer them please re-request or flag me on a new patch, I'll grab it right away. ::: buildfarm/mobile/make.sh @@ +1,1 @@ > +#!/bin/bash I'm not a huge fan of the naming of this file (I'd like something that makes sense at a glance, and makes us know what it is used for easily. |make.sh| feels so generic, and almost makes me feel I should type "make" to run it (even with the .sh extension). -- Sadly I have no better suggestions atm, so unless you can think of one, I won't block/bikeshed on it, but I just wanted to voice my sadness at it. ;-) @@ +4,5 @@ > +DEVICE=`basename $PWD` > +opt="$1" > + > +if [ "$opt" = "buildslave" ]; then > + if [ "$2" = "start" ]; then Nit: rather than sprinkle $2 throughout this file, why not do a test if $2 exists (if not exit 1) and set a var like $buildslave_cmd for it, or something @@ +16,5 @@ > + rm buildbot.tac > + wget -q -O/builds/$DEVICE/buildbot.tac http://slavealloc.build.mozilla.org/gettac/$DEVICE > +elif [ "$opt" = "start" ]; then > + /tools/buildbot-0.8.4-pre-moz2/bin/python2.7 \ > + /tools/buildbot-0.8.4-pre-moz2/bin/twistd --no_save \ Nit I'd love these paths to be constants at the top of this .sh script, such as $BUILDBOT_PATH $BB_PYTHON $BB_TWISTD etc. Makes it easier if we ever do change the versions/details. ::: buildfarm/mobile/setup_foopy.py @@ +1,1 @@ > +import urllib2 ..is the idea that this file is to be used for all panda foopies, all b2g foopies, or *all* foopies [eventually, not necessarily at landing]. If *all* this is fine, if b2g only, perhaps rename to setup_b2g_foopy.py or something? [I'm open to arguments why that is a bad idea] @@ +9,5 @@ > +toolsdir = "/builds/tools" > +path_to_bmm_host = "/builds/bmm.hostname.txt" > +# cltbld's uid and gid > +const_uid=500 > +const_gid=500 Do we really need numberic hardcoded uid/gid and can't use strings here or something? I don't like these types of buried hardcodes, waaay too easy to bite us later. @@ +38,5 @@ > +match=re.search(".*.p([0-9]*)\..*", hostname) > +vlan=int(match.group(1)) > +if not os.path.exists(path_to_bmm_host): > + local_file = open(path_to_bmm_host, 'wb') > + local_file.write("mobile-imaging-0%02i.p%i.releng.scl1.mozilla.com" % (vlan, vlan)) is this `mobile-imaging-0[VLAN].p[VLAN]` naming guaranteed to hold, or is there a chance we could have a .p23 with a mobile-imaging-010 for example.
Attachment #687113 -
Flags: review?(bugspam.Callek)
Attachment #687113 -
Flags: review-
Attachment #687113 -
Flags: feedback+
Assignee | ||
Comment 28•12 years ago
|
||
Where is the secrets file so I can add that extra key? You can PM me if preferred.
Attachment #687247 -
Attachment is obsolete: true
Attachment #687745 -
Flags: review?(bugspam.Callek)
Comment 29•12 years ago
|
||
Comment on attachment 687745 [details] [diff] [review] BuildSlaves-tests.py.erb - puppet changes for the masters Review of attachment 687745 [details] [diff] [review]: ----------------------------------------------------------------- Its buried on old-puppet that we use for the buildbot masters. https://wiki.mozilla.org/ReleaseEngineering/Puppet/Usage#Masters if you need more help in finding this secrets file we can go through it on irc.
Attachment #687745 -
Flags: review?(bugspam.Callek) → review+
Assignee | ||
Comment 30•12 years ago
|
||
I have addressed all of your comments. I would like setup_foopy.py to be used as the way of setting up every foopy. For now, we are only doing it with the foopies for b2g devices but I'm not stopping anyone for trying it for the other foopies. For now, we only have 10 mobile imaging servers because we're using 10 racks (one per rack). The naming as of now is mobile-imaging-0[1-10].p[1-10].releng.scl1.mozilla.com
Attachment #687775 -
Flags: review?(bugspam.Callek)
Assignee | ||
Updated•12 years ago
|
Attachment #687113 -
Attachment is obsolete: true
Comment 31•12 years ago
|
||
Comment on attachment 687775 [details] [diff] [review] setup_foopy.py, updated devices.json and manage_buildslaves.sh Review of attachment 687775 [details] [diff] [review]: ----------------------------------------------------------------- r+ with 1 real issue, and 2 nits ::: buildfarm/mobile/manage_buildslave.sh @@ +1,3 @@ > +#!/bin/bash > +PWD=`pwd` > +DEVICE=`basename $PWD` nit (non-blocking): we should probably sanity-check $DEVICE with some form of "startswith panda- or tegra-" check. To prevent running this with a wrong cwd. Since we use $DEVICE pretty indiscriminately below. @@ +3,5 @@ > +DEVICE=`basename $PWD` > +opt="$1" > +BB_PATH=/tools/buildbot-0.8.4-pre-moz2/bin/buildslave > +BB_PYTHON=/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 > +BB_TWISTD=/tools/buildbot-0.8.4-pre-moz2 that's not the twistd binary :-P @@ +20,5 @@ > + echo "We will run with the twistd command instead of calling buildslave" > + ${BB_PYTHON} ${BB_TWISTD} --no_save \ > + --rundir=/builds/$DEVICE \ > + --pidfile=/builds/$DEVICE/twistd.pid \ > + --python=/builds/$DEVICE/buildbot.tac bikeshed-nit, indent the carried-on lines slightly
Attachment #687775 -
Flags: review?(bugspam.Callek) → review+
Assignee | ||
Comment 32•12 years ago
|
||
Comment on attachment 687775 [details] [diff] [review] setup_foopy.py, updated devices.json and manage_buildslaves.sh I fixed the 2 problems.
Attachment #687775 -
Flags: checked-in+
Assignee | ||
Comment 33•12 years ago
|
||
Comment on attachment 687245 [details] [diff] [review] BuildSlaves Templates in buildbot-configs Checked in with fixes in comment 21.
Attachment #687245 -
Flags: checked-in+
Assignee | ||
Comment 34•12 years ago
|
||
Comment on attachment 687745 [details] [diff] [review] BuildSlaves-tests.py.erb - puppet changes for the masters checked-in and modified secrets.pp. Now looking at [root@buildbot-master19 ~]# tail -F /var/log/messages
Attachment #687745 -
Flags: checked-in+
Assignee | ||
Comment 35•12 years ago
|
||
Comment on attachment 687239 [details] [diff] [review] buildbot-config changes to add b2g dummy job I can finally run test-masters locally and have fixed the issue. This has been landed. Tomorrow I will be working on the sendchanges from the panda builds.
Attachment #687239 -
Flags: checked-in+
Assignee | ||
Comment 36•12 years ago
|
||
I will leave this bug open to setup the foopies with "manage_buildslave.sh" rather than "make.sh". I want all of it to be consistent. Not a high priority though.
Comment 37•12 years ago
|
||
In production
Assignee | ||
Comment 38•12 years ago
|
||
I setup again the current foopies: # ssh as cltbld cd /builds/tools hg pull -u hg id su - rm -rf /builds/panda-* cd /builds/tools/buildfarm/mobile python setup_foopy.py exit for i in `ls | grep panda`; do /builds/manage_buildslave.sh start $i; done
Assignee | ||
Comment 39•12 years ago
|
||
An improvement that we landed is this: http://hg.mozilla.org/build/tools/rev/6bf3e47a84d9 which got reviewed and landed in bug 802317: https://bug802317.bugzilla.mozilla.org/attachment.cgi?id=688923
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Updated•11 years ago
|
Product: mozilla.org → Release Engineering
Updated•6 years ago
|
Product: Release Engineering → Infrastructure & Operations
Updated•4 years ago
|
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•