Closed Bug 565402 Opened 12 years ago Closed 11 years ago

Install tool-chain on win64-ix-ref

Categories

(Release Engineering :: General, defect, P5)

x86
macOS
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: armenzg, Assigned: armenzg)

References

Details

(Whiteboard: [win64][q1goal])

Attachments

(7 files, 14 obsolete files)

4.05 KB, patch
catlee
: feedback+
Details | Diff | Splinter Review
143 bytes, text/plain
Details
9.79 KB, patch
bhearsum
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
17.88 KB, patch
armenzg
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
3.50 KB, patch
Details | Diff | Splinter Review
289 bytes, text/plain
Details
16.78 KB, text/plain
Details
The initial goal is to get a machine that:
1) can generate Windows 64bit builds
2) process jobs from buildbot
3) other slaves can be cloned of and be part of a builders pool
4) let's have in mind possibly generating 32bit builds (Let's not scope creep)

To archive the first point I will follow this:
* Read https://developer.mozilla.org/En/Developer_Guide/Build_Instructions/Windows_Prerequisites
* Install:
    * MozillaBuild 1.4
    * Microsoft Visual C++ 2010 Express
    * Microsoft Windows SDK(s)
         * Is Windows7 SDK enough?

After I hand out the first 64 bit build to someone to QA points 2 and 3 should be tackled.

Other tools that will need to be installed:
    * Buildbot
    * NRPE daemon
    * OPSI
    * VNC
    * ssh

Check https://wiki.mozilla.org/ReferencePlatforms/Win32#Configuration_Details and see which configuration changes are still applicable for this new machine. So far I have seen these ones:
# disable automatic updates
# indexing
# registry changes
# environment vars
# any extra twistd patches?
# ssh keys (accept staging keys)
# screen resolution for unit tests
# java
# autoit
# opsi
    * timeouts
    * disable "shutdown event tracker"
    * OPSI hostkey generator
# buildbot tac generator
# cltbld autologin
Note that some files, such as buildbot-tac.py, the buildbot.bat startup file, and possibly others will need to be updated to be made aware of the win64 hostnames.
Attached file first build log (obsolete) —
I think that I have set up the requirements correctly (I have passed configure) but I am getting build errors.

c:/Users/Administrator/armenzg/mozilla-central/build/win32/vmwarerecordinghelper/vmwarerecordinghelper.cpp(66) : error C2143: syntax error : missing ';' before '}'

Here are my settings:
* MS 2008 SP2 (x64 bit)
* VS2010 Pro
* Win7 SDK
* MozillaBuild 1.4 (plus msvc10-x64.bat)

> . $topsrcdir/browser/config/mozconfig
> 
> mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../objdir
> ac_add_options --target=x86_64-pc-mingw32
> ac_add_options --host=x86_64-pc-mingw32
> ac_add_options --disable-ogg
I modified the "build/Makefile.in" file to don't build "win32" and I got the build all the way through (I will file a bug).

Please find the builds in here:
http://staging-stage.build.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/
and as one-time prof-of-concept in here:
http://people.mozilla.com/~armenzg/win64/
Priority: P2 → P4
Priority: P4 → P2
I am also adding "win64-" just in case one day we have 64 bit machines that are not ix machines and this way we won't forget about making the change. Your call.

What I have locally has different paths since I currently have the C drive but I am assuming that I will use the same paths once we create the others drives.
Attachment #448518 - Flags: review?(bhearsum)
You might be interested in the fact that WinSDK 7.1 was already released.
This is pretty much this file:
http://hg.mozilla.org/mozilla-build/raw-file/224a69b28acf/start-msvc10-x64.bat
where I have added at the end the following:
> :start
> "%MOZILLABUILD%\msys\bin\bash" --login -c /d/mozilla-build/start-buildbot.sh
> "%MOZILLABUILD%\msys\bin\sleep" 30
removed:
> %MOZILLABUILD%\msys\bin\bash" --login -i" 
and added the same type of logging that appears in this file:
http://hg.mozilla.org/build/opsi-package-sources/raw-file/a467e0ee51ff/buildbot-startup/CLIENT_DATA/start-buildbot.bat

bhearsum I will post a diff after this attachment.
Differences with this file:
http://hg.mozilla.org/build/opsi-package-sources/raw-file/a467e0ee51ff/buildbot-startup/CLIENT_DATA/start-buildbot.bat

bhearsum what are your thoughts on this file?
Maybe we can call it from buildbot.bat (the startup script) with a parameter to either start on VC10 mode or VC8 mode so it can be shared with win32 and win64 machines.
Attachment #448589 - Flags: feedback?(bhearsum)
Comment on attachment 448518 [details] [diff] [review]
buildbot.bat startup script changes

Seems fine
Attachment #448518 - Flags: review?(bhearsum) → review+
Attachment #448589 - Flags: feedback?(bhearsum) → feedback?(catlee)
Attachment #445363 - Attachment is obsolete: true
Now that I have C, D & E drives on the w64-ix-ref machine I can update this attachment by changing /c/mozilla-build to /d/mozilla-build.
Attachment #448587 - Attachment is obsolete: true
Comment on attachment 448589 [details] [diff] [review]
diff between what I am running on win64-ix-ref and what is being used on win32 machines

>--- /c/Users/Administrator/Desktop/start-buildbot.bat   2010-06-01 10:46:21 -0700
>+++ start-buildbot.bat  2010-06-01 12:18:04 -0700
>@@ -1,69 +1,91 @@
> @echo off
> 
>-SET MOZ_MSVCVERSION=8
>-
>-SET MOZILLABUILDDRIVE=%~d0%
>-SET MOZILLABUILDPATH=%~p0%
>-SET MOZILLABUILD=%MOZILLABUILDDRIVE%%MOZILLABUILDPATH%
>+SET MOZ_MSVCVERSION=9
>+SET MOZBUILDDIR=%~dp0
>+SET MOZILLABUILD=%MOZBUILDDIR%
> 
> set log="c:\tmp\buildbot-startup.log"
> 
>-echo "Mozilla tools directory: %MOZILLABUILD%"
>+echo "Mozilla tools directory: %MOZBUILDDIR%"
> 
> REM Get MSVC paths
> echo "%date% %time% - About to call guess-msvc.bat" >> %log%
>-call "%MOZILLABUILD%\guess-msvc.bat"
>+call "%MOZBUILDDIR%guess-msvc.bat"
> 
> REM Use the "new" moztools-static
>-set MOZ_TOOLS=%MOZILLABUILD%\moztools
>+set MOZ_TOOLS=%MOZBUILDDIR%moztools-x64
> 
> rem append moztools to PATH
> SET PATH=%PATH%;%MOZ_TOOLS%\bin
> 
>-if "%VC8DIR%"=="" (
>-    if "%VC8EXPRESSDIR%"=="" (
>-        echo "%date% %time% - MSVC++8 not found" >> %log%
>-        ECHO "Microsoft Visual C++ version 8 was not found. Exiting."
>+if "%VC10DIR%"=="" (
>+    if "%VC10EXPRESSDIR%"=="" (
>+   echo "%date% %time% - MSVC++10  not found" >> %log%
>+        ECHO "Microsoft Visual C++ version 10 (2010) was not found. Exiting."
>         pause
>         EXIT /B 1
>     )
> 
>     if "%SDKDIR%"=="" (
>         echo "%date% %time% - SDK not found" >> %log%
>         ECHO "Microsoft Platform SDK was not found. Exiting."
>         pause
>         EXIT /B 1
>     )
> 
>     rem Prepend MSVC paths
>-    echo "%date% %time% - About to call vcvars32.bat" >> %log%
>-    call "%VC8EXPRESSDIR%\Bin\vcvars32.bat"
>+    echo "%date% %time% - About to call vcvars64.bat" >> %log%
>+    call "%VC10EXPRESSDIR%\bin\amd64\vcvars64.bat"
> 
>-    SET USESDK=1
>     rem Don't set SDK paths in this block, because blocks are early-evaluated.
>-) else (
>-    rem Prepend MSVC paths
>-    echo "%date% %time% - Calling vcvars32.bat in VC8DIR" >> %log%
>-    call "%VC8DIR%\Bin\vcvars32.bat"
> 
>-    rem If the SDK is Win2k3SP2 or higher, we want to use it
>-    if %SDKVER% GEQ 5 (
>-      SET USESDK=1
>+    rem Fix problem with VC++Express Edition
>+    if "%SDKVER%" GEQ "6" (
>+        rem SDK Ver.6.0 (Windows Vista SDK) and newer
>+        rem do not contain ATL header files.
>+        rem We need to use the Platform SDK's ATL header files.
>+        SET USEPSDKATL=1
>+    )
>+    rem SDK ver.6.0 does not contain OleAcc.idl
>+    rem We need to use the Platform SDK's OleAcc.idl
>+    if "%SDKVER%" == "6" (
>+        if "%SDKMINORVER%" == "0" (
>+          SET USEPSDKIDL=1
>     )
> )
>+) else (
>+    rem Prepend MSVC paths
>+    rem The Win7 SDK (or newer) should automatically integrate itself into vcvars32.bat
>+    ECHO Using VC 2010 built-in SDK
>+    echo "%date% %time% - Calling vcvars64.bat in VC10DIR" >> %log%
>+    call "%VC10DIR%\bin\amd64\vcvars64.bat"
>+)
> 
>-if "%USESDK%"=="1" (
>+if "%VC10DIR%"=="" (
>     rem Prepend SDK paths - Don't use the SDK SetEnv.cmd because it pulls in
>     rem random VC paths which we don't want.
>     rem Add the atlthunk compat library to the end of our LIB
>-    set PATH=%SDKDIR%\bin;%PATH%
>-    set LIB=%SDKDIR%\lib;%LIB%;%MOZILLABUILD%\atlthunk_compat
>-    set INCLUDE=%SDKDIR%\include;%SDKDIR%\include\atl;%INCLUDE%
>+    set "PATH=%SDKDIR%\bin\x64;%SDKDIR%\bin;%PATH%"
>+    set "LIB=%SDKDIR%\lib\x64;%SDKDIR%\lib;%LIB%;%MOZBUILDDIR%atlthunk_compat"

Why do you have quotes around the whole thing here?  And do you need a ';' before atlthunk_compat, or is that intentional?

Other than that, looks ok to me.  I don't really know what I'm looking at though :)  Maybe poke ted for a review?

>+
>+    if "%USEPSDKATL%"=="1" (
>+        if "%USEPSDKIDL%"=="1" (
>+            set "INCLUDE=%SDKDIR%\include;%PSDKDIR%\include\atl;%PSDKDIR%\include;%INCLUDE%"
>+        ) else (
>+            set "INCLUDE=%SDKDIR%\include;%PSDKDIR%\include\atl;%INCLUDE%"
>+        )
>+    ) else (
>+        if "%USEPSDKIDL%"=="1" (
>+            set "INCLUDE=%SDKDIR%\include;%SDKDIR%\include\atl;%PSDKDIR%\include;%INCLUDE%"
>+        ) else (
>+            set "INCLUDE=%SDKDIR%\include;%SDKDIR%\include\atl;%INCLUDE%"
>+        )
>+    )
> )
> 
> cd "%USERPROFILE%"
> :start
> echo "%date% %time% - About to run start-buildbot.sh" >> %log%
> "%MOZILLABUILD%\msys\bin\bash" --login -c /d/mozilla-build/start-buildbot.sh
> echo "%date% %time% - start-buildbot.sh finished" >> %log%
> "%MOZILLABUILD%\msys\bin\sleep" 30
Attachment #448589 - Flags: feedback?(catlee) → feedback+
Depends on: 579331
Priority: P2 → P3
Priority: P3 → P2
Priority: P2 → P3
Priority: P3 → P2
Attached file python25.reg (obsolete) —
Attachment #473649 - Flags: review?(bhearsum)
Attachment #449321 - Attachment is obsolete: true
The only difference is this:
- start_cmd="/d/mozilla-build/python/scripts/buildbot start $slave_dir"
+ start_cmd="/d/mozilla-build/python25/scripts/buildbot start $slave_dir"
Attached file python26.reg
MozillaBuild1.5.1 comes with python2.6 instead of 2.5 so I had to register 2.6 instead.
Attachment #473151 - Attachment is obsolete: true
Attachment #473649 - Flags: review?(bhearsum) → review+
Depends on: 596337
Depends on: 596735
Depends on: 597563
Depends on: 597573
Priority: P2 → P3
Priority: P3 → P4
No longer blocks: 588950
Depends on: 602612
Status update
=============
* I wasn't able to install OPSI after several days trying (see bug 596337)
* After we got OPSI working we would need to install nagios and few easier things
* Right now this work has very low priority as it does not block FF4 and there is just so much in my plate that can improve the release of FF4
* Before tackling this again I need to setup the Linux64 IX reference machine as it is a FF4 blocking platform
Depends on: 607316
Attached patch fix setting tools dir for Win64 (obsolete) — Splinter Review
This patch fixes setting the right toolsdir for Win64.
Attachment #486067 - Flags: review?(bhearsum)
Attachment #486067 - Flags: review?(bhearsum) → review+
Priority: P4 → P5
Priority: P5 → P3
Whiteboard: [win64] → [win64][q1goal]
Attachment #473661 - Attachment is obsolete: true
Attached file start-buildbot.bat (obsolete) —
Attachment #473650 - Attachment is obsolete: true
Attached file start-buildbot.bat (take 2) (obsolete) —
Attachment #519482 - Attachment is obsolete: true
No longer blocks: 561435
No longer depends on: 596337
We had to disable the Windows Error Reporting to avoid timing out on |make -k check|.
https://wiki.mozilla.org/ReferencePlatforms/Win64#Disable_Windows_Error_Reporting
Depends on: 642893
From my reading of:
http://technet.microsoft.com/en-us/library/cc786709%28WS.10%29.aspx

It seems that we don't need to do the 4GT tuning (aka /3GB switch) as done on the Win2k3 32-bit machines on bug 543034 [1].

The only thing that I noticed that might be needed is to set the IMAGE_FILE_LARGE_ADDRESS_AWARE flag [2].


[1]:
> 4-gigabyte tuning (4GT), also known as application memory tuning, or the /3GB
> switch, is a technology (only applicable to 32 bit systems) that alters the
> amount of virtual address space available to user mode applications. Enabling
> this technology reduces the overall size of the system virtual address space
> and therefore system resource maximums.

[2]:
> 2 GB with IMAGE_FILE_LARGE_ADDRESS_AWARE cleared (default)
> 4 GB with IMAGE_FILE_LARGE_ADDRESS_AWARE set
This addresses the issues you raised in https://bugzilla.mozilla.org/show_bug.cgi?id=567154#attachflag4

I am going to join the work of bug 567154 to this bug as it will make it harder to figure out what got fixed where.
Attachment #520752 - Flags: review?(bhearsum)
Comment on attachment 520752 [details] [diff] [review]
[buildbotcustom] add win64 to our custom libraries (take 2)

You should add something to reallyShort() in misc.py, too.

Looks fine otherwise!
Attachment #520752 - Flags: review?(bhearsum) → review+
Depends on: 645024
Status update:
I have updated the Win64 reference platform documentation [1] and have requested machines to be cloned off it.

Next week I will hope to have them on staging and until then I will be landing the patches which I will ask for review as they get approved.

I will also be looking at the current state of the Win7x64 testing machines.

I am going to update some patches and change some bug dependencies to reflect reality.
Sorry in advance for bugmail produced from it!

[1] https://wiki.mozilla.org/ReferencePlatforms/Win64
Comment on attachment 520752 [details] [diff] [review]
[buildbotcustom] add win64 to our custom libraries (take 2)

http://hg.mozilla.org/build/buildbotcustom/rev/18f466407438
Attachment #520752 - Flags: checked-in+
Comment on attachment 486067 [details] [diff] [review]
fix setting tools dir for Win64

Landed with attachment 520752 [details] [diff] [review].
Attachment #486067 - Attachment is obsolete: true
This got checked-in on bug 567154#c21 (pasting on this bug to keep everything in the same place):
http://hg.mozilla.org/build/buildbot-configs/rev/61f8b2f31012
Attachment #523652 - Flags: review+
Attachment #523652 - Flags: checked-in+
Priority: P3 → P4
Nothing to be done until bug 645024 is fixed.
Priority: P4 → P5
(In reply to comment #31)
> Nothing to be done until bug 645024 is fixed.

As mentioned ^^
Assignee: armenzg → nobody
No longer depends on: 645024
Assignee: nobody → armenzg
Status: NEW → ASSIGNED
This patch is pretty much as attachment 519761 [details] which I know to be working but added at the end the chunk to call runslave.py.
Attachment #519761 - Attachment is obsolete: true
Attachment #537611 - Attachment is obsolete: true
Attachment #538014 - Flags: review?(dustin)
Comment on attachment 519478 [details]
Start up buildbot.bat for win64 without passwords (v1)

This attachment is obsolete by this:
http://hg.mozilla.org/build/opsi-package-sources/raw-file/2d51f32ff0f2/buildbot-startup/CLIENT_DATA/buildbot.bat
Attachment #519478 - Attachment is obsolete: true
Comment on attachment 473651 [details]
start-buildbot.sh for MozillaBuild1.5.1

This is obsoleted by slave-alloc support.
Attachment #473651 - Attachment is obsolete: true
Comment on attachment 473649 [details] [diff] [review]
buildbot-tac.py changes for win64

Obsoleted by slave-alloc support.
Attachment #473649 - Attachment is obsolete: true
Comment on attachment 448518 [details] [diff] [review]
buildbot.bat startup script changes

Obsoleted by slave-alloc support.
Attachment #448518 - Attachment is obsolete: true
Tools are setup and the machine is ready.

If we need to reopen or file following bugs then we will.
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Attachment #538014 - Flags: review?(dustin)
This is part of work in bug 645024.
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.