Closed Bug 683976 Opened 13 years ago Closed 12 years ago

Add buildbot and updated runslave.py to the w64 image

Categories

(Infrastructure & Operations :: RelOps: General, task, P5)

x86_64
Windows Server 2008

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: dustin, Unassigned)

References

Details

(Whiteboard: [refplatform])

Attachments

(1 file, 4 obsolete files)

Bug 679818 suggests that we're not using reference machines anymore, so the work done in bug 637347 needs to be incorporated into the install process for future w64 systems.

I'm happy to help, but I don't know how that's done.
To be clear, this is about the *new* version of Buildbot - 0.8.4-pre-moz2.
I am adding custom run scripts for the image to update to the latest buildbot and runslave.py now.
Added a custom task during deployment to run the buildbot batch file that dustin sent me. For now I have added the batch file to the image but am playing on having the task have each new deploy look to a share on the WDS server for the batch file. This will make it easier to edit the batch file and keep our standards of trying to get away from having "golden images" I still need to test a push of a new image to verify the batch file works.
Let me know if you need a host to try on and I will pull one or few for you.
I was able to get this updated and I just need a host to test the new install on. Armen let me know which one I can work with please and thank you.
Status: NEW → ASSIGNED
(In reply to Matthew Larrain[:digipengi] from comment #5)
> I was able to get this updated and I just need a host to test the new
> install on. Armen let me know which one I can work with please and thank you.

You can use w64-ix-slave41. Sorry for the delay.
Testing image update on w64-ix-slave41
As digipengi mentions he is having trouble deploying runslave.py to the w64 machines as placing it under "C:\" requires dealing with an UAC prompt (I never learn how to circumvent those).

The reason this was placed in "C:\" was because it was common denominator for win2008, win2003, win7 and winxp.

If we decide to go down the path of placing it somewhere where we won't need to deal with admin problems we need to take the following into consideration:

1) buildbot.bat (aka buildbot-win64.bat) calls "C:\mozilla-build\start-buildbot.bat" [1]
2) start-buildbot.bat (aka start-buildbot-win64.bat) calls "python /c/runslave.py" [2]

This means that we would have to patch start-buildbot-win64.bat and re-deploy it.
This also implies that we would have to make all w64 slaves to get patched as well (runslave.py at a newer location and re-deploy start-buildbot.bat to point to the new location of runslave.py).

This is all document on the ref docs [3].

Let me know if this makes sense or not :).

[1] http://mxr.mozilla.org/build/source/puppet-manifests/modules/buildslave/files/buildbot-win64.bat
[2] http://hg.mozilla.org/build/puppet-manifests/file/c6e02dee3589/modules/buildslave/files/start-buildbot-win64.bat#l109
[3] https://wiki.mozilla.org/ReferencePlatforms/Win64#Startup_scripts
I am able to have WDS/MDT deploy the batch file but there is a part in it that requires user interaction of typing in the admin password.
colo-trip: --- → scl1
colo-trip: scl1 → ---
I will work on this and get digipengi a script that won't need admin privileges.

This will be a Q1 goal.
Assignee: mlarrain → nobody
Component: Server Operations: RelEng → Release Engineering
QA Contact: zandr → release
Whiteboard: [q1goal]
Assignee: nobody → armenzg
Priority: -- → P3
I have too much in my plate and have to prioritize work.

This is not a pressing issue as Windows 64-bit has not become a tier-1 platform.

There is also debate if we should use group policy up to certain point and from there we puppetize the rest with Windows (this bug sounds like a good candidate).
Assignee: armenzg → nobody
Priority: P3 → --
Whiteboard: [q1goal] → [q1goal][triagefollowup]
Priority: -- → P3
Unless we end up building win32 on win64, this bug is very low priority.
Status: ASSIGNED → NEW
Priority: P3 → P5
Whiteboard: [q1goal][triagefollowup] → [refplatform]
OS: All → Windows Server 2008
Hardware: All → x86_64
I downtimed the "w64-ix-slave41.winbuild.scl1:PING is CRITICAL: PING CRITICAL - Packet loss = 100%" alert for 180 days, if this lets that slave loose before then you'll want to undo that.
Digipengi: could you send this bug back or file a new one about putting w64-ix-slave41 back in the pool once you're done? I'm going to remove w64-ix-slave41 from bug 684019 so it stops lingering.
MaRu, I will try to install tomorrow morning buildbot assuming the newer location of runslave.py.

Steps to fix the runslave.py issue:
1) wget -OC:\Tools\runslave.py http://hg.mozilla.org/build/puppet-manifests/raw-file/tip/modules/buildslave/files/runslave.py
2) modify and deploy newer C:\mozilla-build\start-buildbot.bat

diff --git a/modules/buildslave/files/start-buildbot-win64.bat b/modules/buildslave/files/start-buildbot-win64.bat
--- a/modules/buildslave/files/start-buildbot-win64.bat
+++ b/modules/buildslave/files/start-buildbot-win64.bat
@@ -101,11 +101,11 @@ if "%WIN64%" == "1" (
 cd "%USERPROFILE%"
 :start
 
 echo "%date% %time% - About to run runslave.py"
 
 REM running this via 'bash' is critical - bash adds a bunch of items to PATH
 REM which the build steps expect to find.
 
-"%MOZILLABUILD%\msys\bin\bash" --login -c "python /c/runslave.py"
+"%MOZILLABUILD%\msys\bin\bash" --login -c "python /c/Tools/runslave.py"
 
 echo "%date% %time% - runslave.py finished"
This removes the need to be administrator to deploy runslave.py
This is working for me on staging.
Assignee: nobody → armenzg
Status: NEW → ASSIGNED
The only changes I have made to the original is:
* change production-puppet to scl3-production-puppet
* remove the echo off
* C drive for mozilla-build instead of D drive
* wget runslave.py to C:\Tools instead of C:\
** no more need for runas administrator
I need to figure out which way to go.

Here is how things are called:
buildbot.bat -> C:\mozilla-build\start-buildbot.bat -> C:\runslave.py

To deploy the newer buildbot we need also a newer runslave.py.
To deploy runslave.py unto C:\ we need to run as admin which does not allow us to automate it.  We need to fix this so we can use GPO.

We can deploy runslave.py to C:\Tools without any trouble *but* that means we need to deploy a start-buildbot.bat that calls C:\Tools\runslave.py instead of C:\runslave.py.
Unfortunately, start-builtbot.bat is owned by admin.
MaRu, can this be fixed? I guess we can do it manually. That would suck but it will allow us to move on.

On another note, we could deploy a newer buildbot.bat that calls to C:\**Tools**\start-buildbot.bat (rather than mozilla-buildbot).

At the end of the road we should not:
* have mozilla-build owned by Administrator
* delete C:\runslave.py
Armen: did you mean scl-production-puppet?  That's the puppet master in the datacenter where the w64 machines are (scl1).
This patch puts runslave.py and start-buildbot.bat in C:\Tools.
This patch puts a new buildbot.bat in place that calls C:\Tools\start-buildbot.bat

Current:
buildbot.bat -> C:\mozilla-build\start-buildbot.bat -> C:\runslave.py

Proposed change:
buildbot.bat -> C:\Tools\start-buildbot.bat -> C:\Tools\runslave.py

Works for you?
Attachment #626894 - Flags: review?(mlarrain)
Attachment #626894 - Flags: review?(coop)
Note: you can ignore pointing to my people account and think of URLs to puppet-manifests.
Comment on attachment 626894 [details] [diff] [review]
[puppet-manifests] C:\Tools is the new magic

This doesn't work.
guess-msvc.bat is under C:\mozilla-build (which is called by start-buildbot.bat).
Attachment #626894 - Flags: review?(mlarrain)
Attachment #626894 - Flags: review?(coop)
Attached file update-buildbot-win64.bat (obsolete) —
This is what MaRu is now testing.
Attachment #626866 - Attachment is obsolete: true
Attachment #626873 - Attachment is obsolete: true
Attachment #626894 - Attachment is obsolete: true
Blocks: 757950
Blocks: 756127
Blocks: 758008
I emailed this last night to MaRu.
I think we don't need the "runas" command if we are running it as Administrator. Otherwise, it will prompt us regardless of which user we run this script as.

This script only differs in the last command compared to the previous one.
Attachment #626953 - Attachment is obsolete: true
MaRu fixed this.

The only manual step now is fixing the auto-login (bug 758769).
Assignee: armenzg → server-ops-releng
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Component: Release Engineering → Server Operations: RelEng
QA Contact: release → arich
Resolution: --- → FIXED
Component: Server Operations: RelEng → RelOps
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: