Closed Bug 565402 Opened 12 years ago Closed 11 years ago
Install tool-chain on win64-ix-ref
4.05 KB, patch
|Details | Diff | Splinter Review|
143 bytes, text/plain
9.79 KB, patch
|Details | Diff | Splinter Review|
17.88 KB, patch
|Details | Diff | Splinter Review|
3.50 KB, patch
|Details | Diff | Splinter Review|
289 bytes, text/plain
16.78 KB, text/plain
The initial goal is to get a machine that: 1) can generate Windows 64bit builds 2) process jobs from buildbot 3) other slaves can be cloned of and be part of a builders pool 4) let's have in mind possibly generating 32bit builds (Let's not scope creep) To archive the first point I will follow this: * Read https://developer.mozilla.org/En/Developer_Guide/Build_Instructions/Windows_Prerequisites * Install: * MozillaBuild 1.4 * Microsoft Visual C++ 2010 Express * Microsoft Windows SDK(s) * Is Windows7 SDK enough? After I hand out the first 64 bit build to someone to QA points 2 and 3 should be tackled. Other tools that will need to be installed: * Buildbot * NRPE daemon * OPSI * VNC * ssh Check https://wiki.mozilla.org/ReferencePlatforms/Win32#Configuration_Details and see which configuration changes are still applicable for this new machine. So far I have seen these ones: # disable automatic updates # indexing # registry changes # environment vars # any extra twistd patches? # ssh keys (accept staging keys) # screen resolution for unit tests # java # autoit # opsi * timeouts * disable "shutdown event tracker" * OPSI hostkey generator # buildbot tac generator # cltbld autologin
Note that some files, such as buildbot-tac.py, the buildbot.bat startup file, and possibly others will need to be updated to be made aware of the win64 hostnames.
I think that I have set up the requirements correctly (I have passed configure) but I am getting build errors. c:/Users/Administrator/armenzg/mozilla-central/build/win32/vmwarerecordinghelper/vmwarerecordinghelper.cpp(66) : error C2143: syntax error : missing ';' before '}' Here are my settings: * MS 2008 SP2 (x64 bit) * VS2010 Pro * Win7 SDK * MozillaBuild 1.4 (plus msvc10-x64.bat) > . $topsrcdir/browser/config/mozconfig > > mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../objdir > ac_add_options --target=x86_64-pc-mingw32 > ac_add_options --host=x86_64-pc-mingw32 > ac_add_options --disable-ogg
I modified the "build/Makefile.in" file to don't build "win32" and I got the build all the way through (I will file a bug). Please find the builds in here: http://staging-stage.build.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/ and as one-time prof-of-concept in here: http://people.mozilla.com/~armenzg/win64/
I am also adding "win64-" just in case one day we have 64 bit machines that are not ix machines and this way we won't forget about making the change. Your call. What I have locally has different paths since I currently have the C drive but I am assuming that I will use the same paths once we create the others drives.
Attachment #448518 - Flags: review?(bhearsum)
You might be interested in the fact that WinSDK 7.1 was already released.
This is pretty much this file: http://hg.mozilla.org/mozilla-build/raw-file/224a69b28acf/start-msvc10-x64.bat where I have added at the end the following: > :start > "%MOZILLABUILD%\msys\bin\bash" --login -c /d/mozilla-build/start-buildbot.sh > "%MOZILLABUILD%\msys\bin\sleep" 30 removed: > %MOZILLABUILD%\msys\bin\bash" --login -i" and added the same type of logging that appears in this file: http://hg.mozilla.org/build/opsi-package-sources/raw-file/a467e0ee51ff/buildbot-startup/CLIENT_DATA/start-buildbot.bat bhearsum I will post a diff after this attachment.
Differences with this file: http://hg.mozilla.org/build/opsi-package-sources/raw-file/a467e0ee51ff/buildbot-startup/CLIENT_DATA/start-buildbot.bat bhearsum what are your thoughts on this file? Maybe we can call it from buildbot.bat (the startup script) with a parameter to either start on VC10 mode or VC8 mode so it can be shared with win32 and win64 machines.
Attachment #448589 - Flags: feedback?(bhearsum)
Comment on attachment 448518 [details] [diff] [review] buildbot.bat startup script changes Seems fine
Attachment #448518 - Flags: review?(bhearsum) → review+
Attachment #448589 - Flags: feedback?(bhearsum) → feedback?(catlee)
Now that I have C, D & E drives on the w64-ix-ref machine I can update this attachment by changing /c/mozilla-build to /d/mozilla-build.
Attachment #448587 - Attachment is obsolete: true
Yasm has been deployed to the slave https://wiki.mozilla.org/ReferencePlatforms/Win64#Add_yasm.exe_to_PATH
Comment on attachment 448589 [details] [diff] [review] diff between what I am running on win64-ix-ref and what is being used on win32 machines >--- /c/Users/Administrator/Desktop/start-buildbot.bat 2010-06-01 10:46:21 -0700 >+++ start-buildbot.bat 2010-06-01 12:18:04 -0700 >@@ -1,69 +1,91 @@ > @echo off > >-SET MOZ_MSVCVERSION=8 >- >-SET MOZILLABUILDDRIVE=%~d0% >-SET MOZILLABUILDPATH=%~p0% >-SET MOZILLABUILD=%MOZILLABUILDDRIVE%%MOZILLABUILDPATH% >+SET MOZ_MSVCVERSION=9 >+SET MOZBUILDDIR=%~dp0 >+SET MOZILLABUILD=%MOZBUILDDIR% > > set log="c:\tmp\buildbot-startup.log" > >-echo "Mozilla tools directory: %MOZILLABUILD%" >+echo "Mozilla tools directory: %MOZBUILDDIR%" > > REM Get MSVC paths > echo "%date% %time% - About to call guess-msvc.bat" >> %log% >-call "%MOZILLABUILD%\guess-msvc.bat" >+call "%MOZBUILDDIR%guess-msvc.bat" > > REM Use the "new" moztools-static >-set MOZ_TOOLS=%MOZILLABUILD%\moztools >+set MOZ_TOOLS=%MOZBUILDDIR%moztools-x64 > > rem append moztools to PATH > SET PATH=%PATH%;%MOZ_TOOLS%\bin > >-if "%VC8DIR%"=="" ( >- if "%VC8EXPRESSDIR%"=="" ( >- echo "%date% %time% - MSVC++8 not found" >> %log% >- ECHO "Microsoft Visual C++ version 8 was not found. Exiting." >+if "%VC10DIR%"=="" ( >+ if "%VC10EXPRESSDIR%"=="" ( >+ echo "%date% %time% - MSVC++10 not found" >> %log% >+ ECHO "Microsoft Visual C++ version 10 (2010) was not found. Exiting." > pause > EXIT /B 1 > ) > > if "%SDKDIR%"=="" ( > echo "%date% %time% - SDK not found" >> %log% > ECHO "Microsoft Platform SDK was not found. Exiting." > pause > EXIT /B 1 > ) > > rem Prepend MSVC paths >- echo "%date% %time% - About to call vcvars32.bat" >> %log% >- call "%VC8EXPRESSDIR%\Bin\vcvars32.bat" >+ echo "%date% %time% - About to call vcvars64.bat" >> %log% >+ call "%VC10EXPRESSDIR%\bin\amd64\vcvars64.bat" > >- SET USESDK=1 > rem Don't set SDK paths in this block, because blocks are early-evaluated. >-) else ( >- rem Prepend MSVC paths >- echo "%date% %time% - Calling vcvars32.bat in VC8DIR" >> %log% >- call "%VC8DIR%\Bin\vcvars32.bat" > >- rem If the SDK is Win2k3SP2 or higher, we want to use it >- if %SDKVER% GEQ 5 ( >- SET USESDK=1 >+ rem Fix problem with VC++Express Edition >+ if "%SDKVER%" GEQ "6" ( >+ rem SDK Ver.6.0 (Windows Vista SDK) and newer >+ rem do not contain ATL header files. >+ rem We need to use the Platform SDK's ATL header files. >+ SET USEPSDKATL=1 >+ ) >+ rem SDK ver.6.0 does not contain OleAcc.idl >+ rem We need to use the Platform SDK's OleAcc.idl >+ if "%SDKVER%" == "6" ( >+ if "%SDKMINORVER%" == "0" ( >+ SET USEPSDKIDL=1 > ) > ) >+) else ( >+ rem Prepend MSVC paths >+ rem The Win7 SDK (or newer) should automatically integrate itself into vcvars32.bat >+ ECHO Using VC 2010 built-in SDK >+ echo "%date% %time% - Calling vcvars64.bat in VC10DIR" >> %log% >+ call "%VC10DIR%\bin\amd64\vcvars64.bat" >+) > >-if "%USESDK%"=="1" ( >+if "%VC10DIR%"=="" ( > rem Prepend SDK paths - Don't use the SDK SetEnv.cmd because it pulls in > rem random VC paths which we don't want. > rem Add the atlthunk compat library to the end of our LIB >- set PATH=%SDKDIR%\bin;%PATH% >- set LIB=%SDKDIR%\lib;%LIB%;%MOZILLABUILD%\atlthunk_compat >- set INCLUDE=%SDKDIR%\include;%SDKDIR%\include\atl;%INCLUDE% >+ set "PATH=%SDKDIR%\bin\x64;%SDKDIR%\bin;%PATH%" >+ set "LIB=%SDKDIR%\lib\x64;%SDKDIR%\lib;%LIB%;%MOZBUILDDIR%atlthunk_compat" Why do you have quotes around the whole thing here? And do you need a ';' before atlthunk_compat, or is that intentional? Other than that, looks ok to me. I don't really know what I'm looking at though :) Maybe poke ted for a review? >+ >+ if "%USEPSDKATL%"=="1" ( >+ if "%USEPSDKIDL%"=="1" ( >+ set "INCLUDE=%SDKDIR%\include;%PSDKDIR%\include\atl;%PSDKDIR%\include;%INCLUDE%" >+ ) else ( >+ set "INCLUDE=%SDKDIR%\include;%PSDKDIR%\include\atl;%INCLUDE%" >+ ) >+ ) else ( >+ if "%USEPSDKIDL%"=="1" ( >+ set "INCLUDE=%SDKDIR%\include;%SDKDIR%\include\atl;%PSDKDIR%\include;%INCLUDE%" >+ ) else ( >+ set "INCLUDE=%SDKDIR%\include;%SDKDIR%\include\atl;%INCLUDE%" >+ ) >+ ) > ) > > cd "%USERPROFILE%" > :start > echo "%date% %time% - About to run start-buildbot.sh" >> %log% > "%MOZILLABUILD%\msys\bin\bash" --login -c /d/mozilla-build/start-buildbot.sh > echo "%date% %time% - start-buildbot.sh finished" >> %log% > "%MOZILLABUILD%\msys\bin\sleep" 30
Attachment #448589 - Flags: feedback?(catlee) → feedback+
The only difference is this: - start_cmd="/d/mozilla-build/python/scripts/buildbot start $slave_dir" + start_cmd="/d/mozilla-build/python25/scripts/buildbot start $slave_dir"
MozillaBuild1.5.1 comes with python2.6 instead of 2.5 so I had to register 2.6 instead.
Attachment #473151 - Attachment is obsolete: true
Attachment #473649 - Flags: review?(bhearsum) → review+
Status update ============= * I wasn't able to install OPSI after several days trying (see bug 596337) * After we got OPSI working we would need to install nagios and few easier things * Right now this work has very low priority as it does not block FF4 and there is just so much in my plate that can improve the release of FF4 * Before tackling this again I need to setup the Linux64 IX reference machine as it is a FF4 blocking platform
This patch fixes setting the right toolsdir for Win64.
Attachment #486067 - Flags: review?(bhearsum)
Attachment #486067 - Flags: review?(bhearsum) → review+
Priority: P5 → P3
Whiteboard: [win64] → [win64][q1goal]
We had to disable the Windows Error Reporting to avoid timing out on |make -k check|. https://wiki.mozilla.org/ReferencePlatforms/Win64#Disable_Windows_Error_Reporting
From my reading of: http://technet.microsoft.com/en-us/library/cc786709%28WS.10%29.aspx It seems that we don't need to do the 4GT tuning (aka /3GB switch) as done on the Win2k3 32-bit machines on bug 543034 . The only thing that I noticed that might be needed is to set the IMAGE_FILE_LARGE_ADDRESS_AWARE flag . : > 4-gigabyte tuning (4GT), also known as application memory tuning, or the /3GB > switch, is a technology (only applicable to 32 bit systems) that alters the > amount of virtual address space available to user mode applications. Enabling > this technology reduces the overall size of the system virtual address space > and therefore system resource maximums. : > 2 GB with IMAGE_FILE_LARGE_ADDRESS_AWARE cleared (default) > 4 GB with IMAGE_FILE_LARGE_ADDRESS_AWARE set
This addresses the issues you raised in https://bugzilla.mozilla.org/show_bug.cgi?id=567154#attachflag4 I am going to join the work of bug 567154 to this bug as it will make it harder to figure out what got fixed where.
Attachment #520752 - Flags: review?(bhearsum)
Comment on attachment 520752 [details] [diff] [review] [buildbotcustom] add win64 to our custom libraries (take 2) You should add something to reallyShort() in misc.py, too. Looks fine otherwise!
Attachment #520752 - Flags: review?(bhearsum) → review+
Status update: I have updated the Win64 reference platform documentation  and have requested machines to be cloned off it. Next week I will hope to have them on staging and until then I will be landing the patches which I will ask for review as they get approved. I will also be looking at the current state of the Win7x64 testing machines. I am going to update some patches and change some bug dependencies to reflect reality. Sorry in advance for bugmail produced from it!  https://wiki.mozilla.org/ReferencePlatforms/Win64
Comment on attachment 520752 [details] [diff] [review] [buildbotcustom] add win64 to our custom libraries (take 2) http://hg.mozilla.org/build/buildbotcustom/rev/18f466407438
Attachment #520752 - Flags: checked-in+
Comment on attachment 486067 [details] [diff] [review] fix setting tools dir for Win64 Landed with attachment 520752 [details] [diff] [review].
Attachment #486067 - Attachment is obsolete: true
This got checked-in on bug 567154#c21 (pasting on this bug to keep everything in the same place): http://hg.mozilla.org/build/buildbot-configs/rev/61f8b2f31012
Nothing to be done until bug 645024 is fixed.
Priority: P4 → P5
(In reply to comment #31) > Nothing to be done until bug 645024 is fixed. As mentioned ^^
Assignee: armenzg → nobody
Assignee: nobody → armenzg
Status: NEW → ASSIGNED
This patch is pretty much as attachment 519761 [details] which I know to be working but added at the end the chunk to call runslave.py.
Comment on attachment 519478 [details] Start up buildbot.bat for win64 without passwords (v1) This attachment is obsolete by this: http://hg.mozilla.org/build/opsi-package-sources/raw-file/2d51f32ff0f2/buildbot-startup/CLIENT_DATA/buildbot.bat
Attachment #519478 - Attachment is obsolete: true
Comment on attachment 473651 [details] start-buildbot.sh for MozillaBuild1.5.1 This is obsoleted by slave-alloc support.
Attachment #473651 - Attachment is obsolete: true
Comment on attachment 473649 [details] [diff] [review] buildbot-tac.py changes for win64 Obsoleted by slave-alloc support.
Attachment #473649 - Attachment is obsolete: true
Comment on attachment 448518 [details] [diff] [review] buildbot.bat startup script changes Obsoleted by slave-alloc support.
Attachment #448518 - Attachment is obsolete: true
Tools are setup and the machine is ready. If we need to reopen or file following bugs then we will.
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
This is part of work in bug 645024.
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.