737994 - Get experimental Win8 Metro builds going on "elm"

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Jim Mathies [:jimm]]

What we need to get started:

1) Get win (32-bit/64-bit) mobile desktop nightlies going again, which were stopped a few months ago, run any mobile specific tests we currently have. We’ll be adding to these tests over time.

2) Get Windows 8 Metro experimental builds going

This is a larger task. For building metro you need:

a) Win7 or |Win8 beta| for the build machine

b) Beta version of VC11 installed (similar to our other VC installs, no sample code, editors, etc. Just build tools, Win8 SDK (now called a ‘Kit’), and redist packages.) 

c) Unreleased MozillaBuild VC11 batch scripts - 

Bug 686837 - 
https://hg.mozilla.org/mozilla-build/file/3ba6c690b036

d) Modified LIBPATH - 

export LIBPATH='C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\LIB;C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\ATLMFC\LIB;C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0\ExtensionSDKs\Microsoft.VCLibs\11.0\References\CommonConfiguration\neutral;C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\vcpackages;C:\Program Files (x86)\Windows Kits\8.0\Windows Metadata;'

e) Windows fennec mobile build config with additional options –

ac_add_options --enable-win8metro
ac_add_options --with-windows-version=602

f) Various initial experimental patches that landed on elm need to land on mc, including build config changes - tracked by bug 737975.

From here you should be able to do a build. Note exes produced do not run on the desktop, and no tests can be performed at this point. Also currently the win mobile builds only generates zip installs, we plan to get an installer going soon.

Note on 64-bit builds using the new compiler and tools, we would like to get 64-bit builds, but on Win8/VMWare these builds regularly crash during the build. 32 bit builds work fine on both Win8/Win7.  (I still need to test 64-bit builds  on Win7 to see if they are more reliable.)

g) We’ll need some way of exposing the success of these builds via tinderbox or similar.

h) We would also like to get these builds going on the Elm branch, which is where our experimental work will take place. Note Elm currently has all the patches necessary to build Win8 Metro builds.

Comment 1

[Jim Mathies [:jimm]]

Also should note - the beta build tools expire 3/2013. We'll have to wipe those and install the final release tools when they come out.

Comment 2

[Jim Mathies [:jimm]]

Updating the title of this bug - the current plan is to fold the metro build into fx builds so that both browser front ends are built at the same time. Hence fennec xul builds are no longer needed. This doesn't affect the new toolset requirements/changes listed here.

Comment 3

[Jim Mathies [:jimm]]

(Updated after experimenting on one of our build servers, plus additional information based on our newer plans for fx integration)

For building metro you need:

a) Svr2008 SP1, Win7, or |Win8 beta| for the build machine

b) install VC11 Beta, full install

c) update mozilla build tools to the latest release

d) Add unreleased MozillaBuild VC11 batch scripts (bug 686837) - 

https://hg.mozilla.org/mozilla-build/file/3ba6c690b036

e) Set modified LIBPATH - 

export LIBPATH='C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\LIB;C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\ATLMFC\LIB;C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0\ExtensionSDKs\Microsoft.VCLibs\11.0\References\CommonConfiguration\neutral;C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\vcpackages;C:\Program Files (x86)\Windows Kits\8.0\Windows Metadata;'

f) Build config:

Currently using Fennec builds for testing on Elm. The plan is to integrate in with the fx desktop build. (bug 747347)

Fennec mobile: use fennec mobile build config plus –

ac_add_options --enable-win8metro
ac_add_options --with-windows-version=602

Firefox: Fx build config plus (bug 737969) –

ac_add_options --enable-metro

We’ll need some way of exposing the success of these builds via tinderbox or similar.

We would also like to get these builds going on the Elm branch, which is where our branch work will take place with regular merges to mc.

Comment 4

[Jim Mathies [:jimm]]

For the vc11 install, I've been using the 'professional' version.

Comment 5

[Armen [:armenzg]]

jimm, how did you install Mozilla Build? The installer fails to install because C:\mozilla-build has a different ownership.

For VS11, did you use the .iso or the .exe?

Comment 6

[Armen [:armenzg]]

FTR, it works if I sign in as the "Administrator" user.

Comment 7

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #6)
> FTR, it works if I sign in as the "Administrator" user.

On that image you guys lent me I ran into problems as the normal user. Ultimately I logged out and logged back in as admin and from there, everything went ok.

Comment 8

[Armen [:armenzg]]

jimm, can someone check in the 2 mozconfigs into the tree?
I would like to be sure that I am using the right thing rather than patching:
http://hg.mozilla.org/mozilla-central/file/default/browser/config/mozconfigs/win32/nightly

thanks

Comment 9

[Armen [:armenzg]]

We also have these:
http://hg.mozilla.org/mozilla-central/file/default/build/win32/mozconfig.vs2010
http://hg.mozilla.org/mozilla-central/file/default/build/win32/mozconfig.vs2010-win64

Comment 10

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #8)
> jimm, can someone check in the 2 mozconfigs into the tree?
> I would like to be sure that I am using the right thing rather than patching:
> http://hg.mozilla.org/mozilla-central/file/default/browser/config/mozconfigs/
> win32/nightly
> 
> thanks

Not sure I understand, which tree and what mozconfigs? We currently don't have all the metro code checked in on mozilla-central yet (working on it). we can file a bug on adding the metro option to mc configs, but we wouldn't want to checkin the changes until we get all the way to the end of our vs11 migration.

Comment 11

[Armen [:armenzg]]

If we could land the vs11 mozconfigs on m-c or elm it would help me to trigger jobs on staging. I can give you patches if that helps.

Comment 12

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #11)
> If we could land the vs11 mozconfigs on m-c or elm it would help me to
> trigger jobs on staging. I can give you patches if that helps.

Ok, so a "vs2011-metro" and "vs2011-metro-win64" on elm, that is easy to get in. I'll file a bug on landing the same configs on mc.

Comment 13

[Armen [:armenzg]]

I cloned mozilla-central. Should I be checking out from Elm?

I have these:
http://pastebin.mozilla.org/1638818
http://pastebin.mozilla.org/1638816

I might be messing up the LIBPATH in the mozconfig-metro (2nd pastebin)

I will continue looking at this in the morning.

and I get this:
configure: error: $(CC) test failed.  You must have MS VC++ in your path to buil
d.
*** Fix above errors and then restart with               "make -f client.mk buil
d"
make[2]: *** [configure] Error 1
make[2]: Leaving directory `/e/win8_test_build/mozilla-central'
make[1]: *** [obj-firefox/Makefile] Error 2
make[1]: Leaving directory `/e/win8_test_build/mozilla-central'
make: *** [build] Error 2

Comment 14

[Jim Mathies [:jimm]]

I'm going to set these up and do complete builds to be sure they are working. Will post back here once I'm done.

One question - are we keeping the direct x sdk on d? (might as well, but I wanted to confirm.) It's currently in /d/sdks/dx10/include according to the configs.

Comment 15

[Jim Mathies [:jimm]]

Interesting, on my local system I have a separate variable  for this - 

DXSDK_DIR='C:\Program Files (x86)\Microsoft DirectX SDK (June 2010)\'

and nothing in my other standard vars. I wonder if we really need that last include.

Comment 16

[Jim Mathies [:jimm]]

Attachment: elm /build/winXX changes

Comment 17

[Jim Mathies [:jimm]]

(In reply to Jim Mathies [:jimm] from comment #16)
> Created attachment 623980 [details] [diff] [review]
> elm /build/winXX changes

AFAICT mozconfig.vs2011 and mozconfig.vs2011-win64 are the same, so we may not need both.

Comment 18

[Jim Mathies [:jimm]]

Attachment: elm /build/winXX changes

added the missing win64 file.

Armen I can check this in on elm if it will help. For builds, I'd suggest doing fx desktop builds using the elm repo for now, with the --enable-metro option. Over the next couple weeks we'll be merging the current mobile cade base with the fx desktop install in preparation for landing all of this on mc.

If we start with the elm repo will it be easy to switch these builders over to mc in a few weeks?

Comment 19

[Armen [:armenzg]]

Wrt dxsdk I don't know.

I will give your mozconfigs a spin.

We actually don't have a D drive on the win64 builders (only on the win32 builders). Nevertheless, I don't see it on your patch.

I will be using elm to build. It is not hard to switch elm builders to be m-c builders.

Comment 20

[Jim Mathies [:jimm]]

Would you like me to check this in on elm? Also, if I do should I add --enable-metro to the existing browser mozconfigs as well?

Comment 21

[Jim Mathies [:jimm]]

Attachment: elm /build/winXX changes

Minor update for the build/win64 config script - it was missing a proper WIN32_REDIST_DIR define.

Comment 22

[Armen [:armenzg]]

Attachment: win32-metro/ mozconfigs and mozconfig.vs2011 files

This patch are working and a build is happening as of now.

What do you think?

Comment 23

[Jim Mathies [:jimm]]

Comment on attachment 624167 [details] [diff] [review]
win32-metro/ mozconfigs and mozconfig.vs2011 files

Review of attachment 624167 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me!

::: browser/config/mozconfigs/win32-metro/debug
@@ +1,3 @@
> +ac_add_options --enable-debug
> +ac_add_options --enable-trace-malloc
> +ac_add_options --enable-signmar

doesn't this config need --enable-metro?

::: browser/config/mozconfigs/win32-metro/l10n-mozconfig
@@ +1,4 @@
> +ac_add_options --enable-update-channel=${MOZ_UPDATE_CHANNEL}
> +ac_add_options --enable-update-packaging
> +ac_add_options --enable-official-branding
> +ac_add_options --with-l10n-base=../../l10n-central

same here maybe?

Comment 24

[Jim Mathies [:jimm]]

Comment on attachment 624167 [details] [diff] [review]
win32-metro/ mozconfigs and mozconfig.vs2011 files

Review of attachment 624167 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/win64/mozconfig.vs2011
@@ +1,1 @@
> +export INCLUDE='C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\INCLUDE;C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\ATLMFC\INCLUDE;C:\Program Files (x86)\Windows Kits\8.0\include\shared;C:\Program Files (x86)\Windows Kits\8.0\include\um;C:\Program Files (x86)\Windows Kits\8.0\include\winrt'

Do we need the dx sdk include in the win64 builds?

Comment 25

[Armen [:armenzg]]

(In reply to Jim Mathies [:jimm] from comment #24)
> Comment on attachment 624167 [details] [diff] [review]
> win32-metro/ mozconfigs and mozconfig.vs2011 files
> 
> Review of attachment 624167 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: build/win64/mozconfig.vs2011
> @@ +1,1 @@
> > +export INCLUDE='C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\INCLUDE;C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\ATLMFC\INCLUDE;C:\Program Files (x86)\Windows Kits\8.0\include\shared;C:\Program Files (x86)\Windows Kits\8.0\include\um;C:\Program Files (x86)\Windows Kits\8.0\include\winrt'
> 
> Do we need the dx sdk include in the win64 builds?

I don't know.
For now, let's focus on build/win32/mozconfig.vs2011-win64 and browser/config/mozconfigs/win32-metro/nightly. I will remove the other files from my patch later on.

I will work on the patches for buildbot to attach this machine to it.

How important is the Fennec mobile win8 builds? As important as desktop I assume?
(I am trying to figure out if both should hit the road at the same time or they can be a week apart).

Comment 26

[Jim Mathies [:jimm]]

I think you can skip fennec builds completely and just concentrate on fx builds going with the metro option enabled using the elm repo. We're in the middle of moving off the fennec code base. So for now we'll be testing our fx changes on elm, and landing patches on mc. In a few weeks we'll have all the code on mc.

Long term - we'll want nightly fx elm builds. Our plan is to expunge the elm repo once we've landed everything on mc and start using elm as our main development branch which we'll merge with mc on a regular basis. (like fx-team and inbound.)

Comment 27

[Armen [:armenzg]]

I have triggered my first set of Windows 8 builds running on staging.
Tomorrow I will check where the job might fail.
Next will be to reproduce my setup steps to get a second machine.

jimm, for now I am focusing on getting dependent builds rather than nightly builds. Does that sound good?

Comment 28

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #27)
> I have triggered my first set of Windows 8 builds running on staging.
> Tomorrow I will check where the job might fail.
> Next will be to reproduce my setup steps to get a second machine.
> 
> jimm, for now I am focusing on getting dependent builds rather than nightly
> builds. Does that sound good?

sounds ok to me.

Comment 29

[Armen [:armenzg]]

I got this failure:
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1337277111.1337277129.31994.gz

cl : Command line warning D9025 : overriding '/Fdgenerated.pdb' with '/Fdhost_nsinstall_win.pdb'
make[5]: *** [host_nsinstall_win.obj] Error 2
make[4]: *** [export_tier_base] Error 2
make[3]: *** [tier_base] Error 2
make[2]: *** [default] Error 2
make[1]: *** [realbuild] Error 2
make: *** [build] Error 2
program finished with exit code 2
elapsedTime=2.862000

Comment 30

[Jim Mathies [:jimm]]

Comment on attachment 624167 [details] [diff] [review]
win32-metro/ mozconfigs and mozconfig.vs2011 files

Review of attachment 624167 [details] [diff] [review]:
-----------------------------------------------------------------

My mistake, I didn't look at these closely enough -

::: build/win32/mozconfig.vs2011
@@ +1,5 @@
> +export INCLUDE='C:\Tools\msvs11\VC\INCLUDE;C:\Tools\msvs11\VC\ATLMFC\INCLUDE;C:\Program Files (x86)\Windows Kits\8.0\include\shared;C:\Program Files (x86)\Windows Kits\8.0\include\um;C:\Program Files (x86)\Windows Kits\8.0\include\winrt;C:\Tools\sdks\dx10\include'
> +export LIBPATH='/c/Tools/msvs11/VC/LIB:/c/Tools/msvs11/VC/ATLMFC/LIB:/c/Program Files (x86)/Microsoft SDKs/Windows/v8.0/ExtensionSDKs/Microsoft.VCLibs/11.0/References/CommonConfiguration/neutral:/c/Tools/msvs11/VC/vcpackages:/c/Program Files (x86)/Windows Kits/8.0/Windows Metadata'
> +export LIB='C:\Tools\msvs11\VC\LIB;C:\Tools\msvs11\VC\ATLMFC\LIB;C:\Program Files (x86)\Windows Kits\8.0\lib\win8\um\x86;C:\Tools\sdks\dx10\lib'
> +export PATH="/c/Tools/msvs11/Common7/IDE:/c/Tools/msvs11/VC/BIN:/c/Tools/msvs11/Common7/Tools:/c/Tools/msvs11/VC/vcpackages:/c/Tools/msvs11/Common7/IDE:/c/Program Files (x86)/Windows Kits/8.0/bin/x86:${PATH}"
> +export WIN32_REDIST_DIR='/c/Tools/msvs11/VC/redist/x86/Microsoft.VC110.CRT'

This is build/win32/mozconfig.vs2011, which is what I think you are pulling in for this build. These paths are wrong - they are still using '/c/Tools/msvs11' which isn't going to work. In the original patch I posted I updated these to point to various vs11 program file paths. I think you just need to use the path info I posted in my original patch.

::: build/win32/mozconfig.vs2011-win64
@@ +1,5 @@
> +export INCLUDE='C:\Tools\msvs11\VC\INCLUDE;C:\Tools\msvs11\VC\ATLMFC\INCLUDE;C:\Program Files (x86)\Windows Kits\8.0\include\shared;C:\Program Files (x86)\Windows Kits\8.0\include\um;C:\Program Files (x86)\Windows Kits\8.0\include\winrt;C:\Tools\sdks\dx10\include'
> +export LIBPATH='/c/Tools/msvs11/VC/LIB:/c/Tools/msvs11/VC/ATLMFC/LIB:/c/Program Files (x86)/Microsoft SDKs/Windows/v8.0/ExtensionSDKs/Microsoft.VCLibs/11.0/References/CommonConfiguration/neutral:/c/Tools/msvs11/VC/vcpackages:/c/Program Files (x86)/Windows Kits/8.0/Windows Metadata'
> +export LIB='C:\Tools\msvs11\VC\LIB;C:\Tools\msvs11\VC\ATLMFC\LIB;C:\Program Files (x86)\Windows Kits\8.0\lib\win8\um\x86;C:\Tools\sdks\dx10\lib'
> +export PATH="/c/Tools/msvs11/Common7/IDE:/c/Tools/msvs11/VC/BIN:/c/Tools/msvs11/Common7/Tools:/c/Tools/msvs11/VC/vcpackages:/c/Tools/msvs11/Common7/IDE:/c/Program Files (x86)/Windows Kits/8.0/bin/x86:${PATH}"
> +export WIN32_REDIST_DIR='/c/Tools/msvs11/VC/redist/x86/Microsoft.VC110.CRT'

These are the old paths as well.

::: build/win64/mozconfig.vs2011
@@ +1,4 @@
> +export INCLUDE='C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\INCLUDE;C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\ATLMFC\INCLUDE;C:\Program Files (x86)\Windows Kits\8.0\include\shared;C:\Program Files (x86)\Windows Kits\8.0\include\um;C:\Program Files (x86)\Windows Kits\8.0\include\winrt'
> +export LIB='C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\LIB\amd64;C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\ATLMFC\LIB\amd64;C:\Program Files (x86)\Windows Kits\8.0\lib\win8\um\x64'
> +export LIBPATH='/c/Program Files (x86)/Microsoft Visual Studio 11.0/VC/LIB/amd64:/c/Program Files (x86)/Microsoft Visual Studio 11.0/VC/ATLMFC/LIB/amd64:/c/Program Files (x86)/Microsoft SDKs/Windows/v8.0/ExtensionSDKs/Microsoft.VCLibs/11.0/References/CommonConfiguration/neutral:/c/Program Files (x86)/Microsoft Visual Studio 11.0/VC/vcpackages:/c/Program Files (x86)/Windows Kits/8.0/Windows Metadata'
> +export PATH=${PATH}:'/c/Program Files (x86)/Microsoft Visual Studio 11.0/VC/BIN/amd64:/c/Program Files (x86)/Microsoft Visual Studio 11.0/VC/BIN/x86_amd64:/c/Program Files (x86)/Windows Kits/8.0/bin/x64:/c/Program Files (x86)/Microsoft Visual Studio 11.0/VC/vcpackages:/c/Program Files (x86)/Microsoft Visual Studio 11.0/Common7/IDE/:/c/Program Files (x86)/Microsoft Visual Studio 11.0/Common7/Tools'

These are the right paths. For some reason the right path data only got imported into your win64 mozconfig. You need to update the win32 configs as well.

Comment 31

[Armen [:armenzg]]

We install vs11 under C:\Tools\msvs11 on the build machines for consistency and originally probably to deal with white spaces.

I am a little closer. Tomorrow I should have buildbot working.

Nevertheless, here is the info for the record.

These are the files I use:
http://hg.mozilla.org/users/armenzg_mozilla.com/elm/file/default/build/win32/mozconfig.vs2011-win64
http://hg.mozilla.org/users/armenzg_mozilla.com/elm/file/default/browser/config/mozconfigs/win32-metro/nightly

Here is my latest log:
link -NOLOGO -OUT:nsinstall.exe -PDB:nsinstall.pdb host_nsinstall_win.obj  -MACHINE:X86  
host_nsinstall_win.obj : fatal error LNK1112: module machine type 'x64' conflicts with target machine type 'X86'
make[5]: Leaving directory `/e/builds/moz2_slave/elm-w32-metro/build/obj-firefox/config'
make[4]: Leaving directory `/e/builds/moz2_slave/elm-w32-metro/build/obj-firefox'
make[3]: Leaving directory `/e/builds/moz2_slave/elm-w32-metro/build/obj-firefox'
make[2]: Leaving directory `/e/builds/moz2_slave/elm-w32-metro/build/obj-firefox'
make[1]: Leaving directory `/e/builds/moz2_slave/elm-w32-metro/build'
make[5]: *** [nsinstall.exe] Error 88
make[4]: *** [export_tier_base] Error 2
make[3]: *** [tier_base] Error 2
make[2]: *** [default] Error 2
make[1]: *** [realbuild] Error 2
make: *** [build] Error 2

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1337291396.1337291438.20443.gz&fulltext=1

Comment 32

[Jim Mathies [:jimm]]

I was under the impression the vc 11 installer didn't provide the option of changing the install location. Is this not the case?

Comment 33

[Armen [:armenzg]]

(In reply to Jim Mathies [:jimm] from comment #32)
> I was under the impression the vc 11 installer didn't provide the option of
> changing the install location. Is this not the case?

It did not allow me to install the SDKs at other locations but the VC is.

Comment 34

[Armen [:armenzg]]

To keep people on the loop, I am making progress:
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1337720121.1337721215.29874.gz&fulltext=1

There is a lot of environment variables that I am fine tuning:
https://etherpad.mozilla.org/armenzg-win8

Today I made a lot of progress compared with the last two days.

Comment 35

[Armen [:armenzg]]

Attachment: [buildbot-configs] win32-metro buildbot config changes

What do you think of loading environment variables this way?

Comment 36

[Armen [:armenzg]]

Attachment: [buildbotcustom] add env variables for msvs11

Comment 37

[Chris Cooper [:coop] (he/him)]

Comment on attachment 626515 [details] [diff] [review]
[buildbot-configs] win32-metro buildbot config changes

Review of attachment 626515 [details] [diff] [review]:
-----------------------------------------------------------------

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #35
> What do you think of loading environment variables this way?

As long as it's consistent with other platforms, it's fine.

Some nits inline.

::: mozilla/config.py
@@ +511,5 @@
> +        'win32-metro': {
> +            'product_name': 'firefox',
> +            'app_name': 'browser',
> +            'brand_name': 'Minefield',
> +            'base_name': 'WINNT 6.1 x86-64 metro %(branch)s',

Seems to be a disconnect between the platform name "win32-metro" and the base name. Should the platform name be win64-metro?

::: mozilla/project_branches.py
@@ +263,4 @@
>          'create_snippet': True,
>          'create_partial': True,
>          'lock_platforms': True,
> +        'repo_path': 'users/armenzg_mozilla.com/elm',

Do you really want this pointing to you personal repo? Why not use whatever project branch the devs are doing the work on?

@@ +272,4 @@
>                  'nightly_signing_servers': 'nightly-signing',
>              },
>              'win32-debug': {},
> +            'win32-metro': {},

Again, win64-metro?

Comment 38

[Armen [:armenzg]]

Hi jimm,
Could you please let me know if this build works?
http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/elm-win32-metro/1337798028/

Comment 39

[Chris Cooper [:coop] (he/him)]

Comment on attachment 626517 [details] [diff] [review]
[buildbotcustom] add env variables for msvs11

Review of attachment 626517 [details] [diff] [review]:
-----------------------------------------------------------------

I wonder if there is a way to set the drive prefix for all the mozilla-build paths to C: or D: based on platform.

::: env.py
@@ +94,5 @@
> +    "PDBSTR_PATH": '/c/Program Files/Debugging Tools for Windows (x64)/srcsrv/pdbstr.exe',
> +    "HG_SHARE_BASE_DIR": 'e:/builds/hg-shared',
> +    "WINDOWSSDKDIR": 'C:\\Program Files (x86)\\Windows Kits\\8.0\\',
> +    "PATH": \
> +        MozillaBuild15Paths + \

Why bother creating MozillaBuild15Paths if this is the only place that uses it?

Comment 40

[Armen [:armenzg]]

Attachment: [m-c] mozconfig changes for msvs11

I would like to land this patch so I can point to the normal elm repo.

On another note, we're creating win8 builds that are 32-bit, correct? (this is to address a question by coop) Even though the machine that creates the build is a Win2008 64-bit machine.

Comment 41

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #38)
> Hi jimm,
> Could you please let me know if this build works?
> http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/elm-win32-
> metro/1337798028/

These builds are incredibly unstable. Fx launches but crashes shortly after loading a page.

I just finished up a release 32-bit build of elm tip using win7 as the build host and found no issues, so can we clobber the obj dir here, checkout elm tip, and fire up a clean build to test again? Seems like something got out of sync with these.

Comment 42

[Jim Mathies [:jimm]]

Comment on attachment 626575 [details] [diff] [review]
[m-c] mozconfig changes for msvs11

These paths look ok. Note though landing on elm does not mean these changes will get to mozilla-central as the two repos currently aren't being merged. Final build config should be posted and reviewed by a build config peer and landed on mc.

Comment 43

[Jim Mathies [:jimm]]

> On another note, we're creating win8 builds that are 32-bit, correct?

correct.

Comment 44

[Armen [:armenzg]]

Last night I generated a newer build by pulling all your changes from elm (rather than my 1-week stale checkout).

I am now triggering the same build but clobbered and based off the official elm repo since I just landed the mozconfig changes.

Comment 45

[Armen [:armenzg]]

Comment on attachment 626575 [details] [diff] [review]
[m-c] mozconfig changes for msvs11

http://hg.mozilla.org/projects/elm/rev/e2ea969a2c27

Comment 46

[Armen [:armenzg]]

Attachment: [puppet-manifests] add win64-metro

I am reusing the win64 password for win64-metro since the pools will be merged in down the road.

Comment 47

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #44)
> Last night I generated a newer build by pulling all your changes from elm
> (rather than my 1-week stale checkout).
> 
> I am now triggering the same build but clobbered and based off the official
> elm repo since I just landed the mozconfig changes.

Hate to say it but you might want to refresh again as we had a bad mc merge a day or so ago. fix was just checked in to elm.

Comment 48

[Armen [:armenzg]]

I am creating a new build.
Nevertheless I tried giving it a shot for the fun of it.
It didn't start.
I got this in the event log.

Log Name:      Application
Source:        Application Error
Date:          5/25/2012 4:20:00 PM
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      talos-w8-hp-001.winbuild.scl1.mozilla.com
Description:
Faulting application name: firefox.exe, version: 15.0.0.4526, time stamp: 0x4fbd2ee2
Faulting module name: MSVCR110.dll, version: 11.0.50214.1, time stamp: 0x4f39f705
Exception code: 0xc0000409
Fault offset: 0x0009b985
Faulting process id: 0xb6c
Faulting application start time: 0x01cd3acce79ed41b
Faulting application path: C:\Users\mozilla\Desktop\firefox-15.0a1.en-US.win32\firefox\firefox.exe
Faulting module path: C:\Users\mozilla\Desktop\firefox-15.0a1.en-US.win32\firefox\MSVCR110.dll
Report Id: 25a44943-a6c0-11e1-a107-b499baa91b69
Faulting package full name: 
Faulting package-relative application ID: 
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Level>2</Level>
    <Task>100</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2012-05-25T23:20:00.000000000Z" />
    <EventRecordID>472</EventRecordID>
    <Channel>Application</Channel>
    <Computer>talos-w8-hp-001.winbuild.scl1.mozilla.com</Computer>
    <Security />
  </System>
  <EventData>
    <Data>firefox.exe</Data>
    <Data>15.0.0.4526</Data>
    <Data>4fbd2ee2</Data>
    <Data>MSVCR110.dll</Data>
    <Data>11.0.50214.1</Data>
    <Data>4f39f705</Data>
    <Data>c0000409</Data>
    <Data>0009b985</Data>
    <Data>b6c</Data>
    <Data>01cd3acce79ed41b</Data>
    <Data>C:\Users\mozilla\Desktop\firefox-15.0a1.en-US.win32\firefox\firefox.exe</Data>
    <Data>C:\Users\mozilla\Desktop\firefox-15.0a1.en-US.win32\firefox\MSVCR110.dll</Data>
    <Data>25a44943-a6c0-11e1-a107-b499baa91b69</Data>
    <Data>
    </Data>
    <Data>
    </Data>
  </EventData>
</Event>

Comment 49

[Jim Mathies [:jimm]]

Are these build currently generating build logs? I'd like to do a build using the exact same build config this builder is using.

Comment 50

[Armen [:armenzg]]

I was hoping the logs were on ftp but they are not there because this is still on staging.

For reference the build is on http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/elm-win32-metro/1337959731

I am going to finalize my patches and setup to enable these as a production-like system on Monday/Tuesday.

Comment 51

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #50)
> I was hoping the logs were on ftp but they are not there because this is
> still on staging.
> 
> For reference the build is on
> http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/elm-win32-
> metro/1337959731
> 
> I am going to finalize my patches and setup to enable these as a
> production-like system on Monday/Tuesday.

The crashiness of these builds has gone away, they are working fine on win8. I have a pretty good idea about the area of the code that is causing load problems on non-win8, although this code is working on my local builds so I really can't debug much until we get regular builds going so I can check some debug code in on elm.

Comment 52

[Armen [:armenzg]]

Status summary:
* I have only one machine working (w64-ix-slave22)
* There are another 2 that should be working but are not yet (w64-ix-slave{41,42})

On Monday I should determine what is the difference between the two sets.

On the bright side, our win64 re-imaging process is now fixed.

Comment 53

[Armen [:armenzg]]

It seems that slave41 had issues with the installation of msvs11 or it is somehow borked.
I tried running C:\Tools\msvs11\VC\bin\amd64\cl.exe and I would get this:
fatal error C1510: Cannot load language resource clui.dll.
I am going to re-install msvs11 and see what happens.
If it doesn't work I will ask IT to re-image it.

I fixed slave 42 to have the right keys to upload builds.
This means we have 2 working machines.

I am going to test slave 11 since IT recently re-imaged it.

I will also look into grabbing another slave to replace slave #20 since I messed it up somehow.

Comment 54

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #53)
> It seems that slave41 had issues with the installation of msvs11 or it is
> somehow borked.
> I tried running C:\Tools\msvs11\VC\bin\amd64\cl.exe and I would get this:
> fatal error C1510: Cannot load language resource clui.dll.
> I am going to re-install msvs11 and see what happens.

A couple people have had this problem with the beta installer, a complete re-install of vc should address it.

Comment 55

[Armen [:armenzg]]

Attachment: [buildbot-configs] win32-metro builds for elm

FTR these are 32-bit builds. I removed x86-64 from the name.

Comment 56

[Chris Cooper [:coop] (he/him)]

Comment on attachment 627770 [details] [diff] [review]
[buildbot-configs] win32-metro builds for elm

Review of attachment 627770 [details] [diff] [review]:
-----------------------------------------------------------------

r+ if you fix up the platform name to consistently be either win32-metro or win64-metro.

::: mozilla/config.py
@@ +502,5 @@
>              # must be overridden explicitly.
>              'nightly_signing_servers': 'dep-signing',
>              'dep_signing_servers': 'dep-signing',
>          },
> +        'win32-metro': {

win64-metro?

::: mozilla/project_branches.py
@@ +274,5 @@
>              'win64': {
>                  'nightly_signing_servers': 'nightly-signing',
>              },
>              'win32-debug': {},
> +            'win32-metro': {},

win64-metro?

Comment 57

[Armen [:armenzg]]

Out of curiosity, I decided to try to install it in an unattended manner. [1]
I am surprised the docs says VS2012 instead of VS2011.

I tried doing it through a shared drive but I get warnings about trying to run a program from the Internet.
\\w64-ix-slave40\VS11_BETA_PRO_ENU\vs_professional.exe /admin \\w64-ix-slave40\VS11_BETA_PRO_ENU\AdminDeployment.xml /quiet /norestart

I posted this on msdn social:
http://social.msdn.microsoft.com/Forums/en-US/vssetup/thread/c86c769a-8b63-4e10-a10c-52387aee2573
########################################################################
Hi,
I am trying to install VS2011 beta unattended but I can't. [1]

I am running this:
C:\Tools\VS11_BETA_PRO_ENU\vs_professional.exe /admin C:\Tools\VS11_BETA_PRO_ENU\packages\AdminDeployment.xml /quiet /norestart
The contents of AdminDeployment.xml are below [2]

My latest error is:
[0E98:0B50][2012-05-28T13:25:35]: MUX:  Admin Deployment metadata validation errors:
[0E98:0B50][2012-05-28T13:25:35]: MUX:  Line:20 Pos:5  The element 'SelectableItemCustomizations' in namespace 'http://schemas.microsoft.com/wix/2011/AdminDeployment' has incomplete content. List of possible elements expected: 'SelectableItemCustomization' in namespace 'http://schemas.microsoft.com/wix/2011/AdminDeployment'.

What I need to know:

    how can I validate the xml file? Visiting http://schemas.microsoft.com/wix/2011/AdminDeployment is useless

There are also these issues in the documentation (which they should take):

    For the XML to work at all it has to be under "packages"
    "package" on "\\ServerName\IDEinstall\package\AdminDeployment.xml" should be "packages" (plural)
    NoWeb="Yes" is incorrect. NoWeb="yes" is valid
    All of the SelectableItemCustomization have to be commented out
    TargetDir has to be created before trying to run it

[1] http://msdn.microsoft.com/en-us/library/ee225237%28v=vs.110%29.aspx
[2]

<?xml version="1.0" encoding="utf-8"?>
<AdminDeploymentCustomizations xmlns="http://schemas.microsoft.com/wix/2011/AdminDeployment">
   <BundleCustomizations TargetDir="C:\Tools\msvs11" NoWeb="yes"/>

   <SelectableItemCustomizations>
     <!--<SelectableItemCustomization Id="TFC" Hidden="no" Selected="yes"/>-->
     <!-- <SelectableItemCustomization Id="CppLibraries" Hidden="no" Selected="no"/> -->
     <!--<SelectableItemCustomization Id="WebTools" Hidden="yes" Selected="yes"/>-->
     <!--<SelectableItemCustomization Id="SQLExpress" Hidden="no" Selected="no"/>-->
     <!--<SelectableItemCustomization Id="ADONETEFT" Hidden="no" Selected="no"/>-->
     <!--<SelectableItemCustomization Id="SQLPubWiz" Hidden="no" Selected="no"/>-->
     <!--<SelectableItemCustomization Id="OfficeTools" Hidden="yes" Selected="yes"/>-->
     <!--<SelectableItemCustomization Id="SharepointTools" Hidden="yes" Selected="yes"/>-->
     <!--<SelectableItemCustomization Id="LightSwitch" Hidden="yes" Selected="yes"/>-->
     <!--<SelectableItemCustomization Id="PreEmptiveDotfuscator" Hidden="yes" Selected="yes" />-->
     <!--<SelectableItemCustomization Id="SQLCLRTypesHidden" Hidden="yes" Selected="yes"/>-->
     <!--<SelectableItemCustomization Id="SQLProviderServicesHidden" Selected="yes"/>-->
     <!--<SelectableItemCustomization Id="SQLSharedManagementObjectsHidden" Hidden="yes" Selected="yes"/>-->
     <!--<SelectableItemCustomization Id="SQLSyncFrameworkHidden" Selected="yes"/>-->
  </SelectableItemCustomizations>
</AdminDeploymentCustomizations>

Comment 58

[Armen [:armenzg]]

Comment on attachment 626788 [details] [diff] [review]
[puppet-manifests] add win64-metro

http://hg.mozilla.org/build/puppet-manifests/rev/ea9eee8d5552

Comment 59

[Armen [:armenzg]]

Comment on attachment 627770 [details] [diff] [review]
[buildbot-configs] win32-metro builds for elm

http://hg.mozilla.org/build/buildbot-configs/rev/bd409064c929

It is also live on production. I will trigger a win32-metro build.

Comment 60

[Armen [:armenzg]]

I am adjusting the summary for "elm" only and dependent builds.

The builds are now being produced from production masters rather than my staging one.
This is the first one created:
http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/elm-win32-metro/1338316317

jimm, when do you estimate to see these builds running on m-c?

Comment 61

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #60)
> I am adjusting the summary for "elm" only and dependent builds.
> 
> The builds are now being produced from production masters rather than my
> staging one.
> This is the first one created:
> http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/elm-win32-
> metro/1338316317
> 
> jimm, when do you estimate to see these builds running on m-c?

Two weeks or so. I'd like to debug these builds, are symbols from these builds being sent to our symbol server?

Comment 62

[Jim Mathies [:jimm]]

'firefox.exe' (Win32): Loaded 'C:\Users\jim\Desktop\firefox\xul.dll'. Cannot find or open the PDB file.

Doesn't appear to be. Any ideas how we can get the symbol files exposed?

Comment 63

[Armen [:armenzg]]

I will work on it.

Comment 64

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #63)
> I will work on it.

Not a major rush Armen, I was just curious if we'll have symbols at some point. 

Another question, are we planning on doing a combination of debug and release builds?

Comment 65

[Rail Aliiev [:rail]]

Make sure that you apply your changes in production_config.py to {staging,preproduction}_config.py files. Right know preproduction and staging configs are broken. :/

Comment 66

[Armen [:armenzg]]

(In reply to Jim Mathies [:jimm] from comment #64)
> (In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment
> #63)
> > I will work on it.
> 
> Not a major rush Armen, I was just curious if we'll have symbols at some
> point. 
> 
OK cool

> Another question, are we planning on doing a combination of debug and
> release builds?

I don't exactly what the question means. I will be adding debug builds in another iteration. The release work will come after that.

On another note, I see the builds showing up on tbpl. They're orange due to a buildbot thingy. I will fix that this week.

Comment 67

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #66)
> On another note, I see the builds showing up on tbpl. They're orange due to
> a buildbot thingy. I will fix that this week.

The automated builds are working though, which is great to see!

Comment 68

[Armen [:armenzg]]

Hi jimm,
I see the automation working but trying to run the build on a Windows 8 machine did not work.

I clicked on it and nothing happened.
I run it as compatibility mode (XP Sp3), it started up but it crashed.

Comment 69

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #68)
> Hi jimm,
> I see the automation working but trying to run the build on a Windows 8
> machine did not work.
> 
> I clicked on it and nothing happened.
> I run it as compatibility mode (XP Sp3), it started up but it crashed.

Hmm, not sure what the problem is. Build 1338398932 launches fine on both win7 and win8 here. Is there anything useful in about:crashes?

Comment 70

[Armen [:armenzg]]

This time it worked. This time when I RDP'ed into it I saw the metro UI (the other times I got the desktop directly) which is the first time. I clicked on the "desktop" tile and clicked on "firefox.exe" and opened without any trouble.

The crash ftr is:
https://crash-stats.mozilla.com/report/index/90c081ca-906d-45b6-8ab8-d337a2120531

Comment 71

[Armen [:armenzg]]

It seems that the XP SP3 property was still in effect.
Removing that makes the build unusable.

Comment 72

[Jim Mathies [:jimm]]

Hey Armen, just checking, those builders seem to have stopped. Maybe this was intentional. Our last build was 30-May-2012 16:58 and we had a checkin after that. Plus the regular vc10 elm builds seem to have stopped as well. The self serve build page shows four builds pending since yesterday.

Comment 73

[Armen [:armenzg]]

Attachment: metro slaves should only do metro builds

jimm, the metro slaves were taking other builds as well.
We have recently switched that *all* w32 builds should run on w64 slaves so they are highly utilized now.
For now, metro slaves should only do metro builds.

Unless coop disagrees which I would understand since we need capacity for the win32 builds.

Comment 74

[Armen [:armenzg]]

jimm says that we are deciding to switch to the VS2012 release candidate.
I will be installing later on the professional version.
http://www.microsoft.com/visualstudio/11/en-us/downloads#professional

I wonder if we should have a new bug since this one is already long enough.

What do you think?

Comment 75

[Jim Mathies [:jimm]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #74)
> jimm says that we are deciding to switch to the VS2012 release candidate.
> I will be installing later on the professional version.
> http://www.microsoft.com/visualstudio/11/en-us/downloads#professional
> 
> I wonder if we should have a new bug since this one is already long enough.
> 
> What do you think?

Fine with me. Please cc me on it. I think there is one path change related to meta data we might have to make too. See bug 755153.

Comment 76

[Armen [:armenzg]]

Comment on attachment 629283 [details] [diff] [review]
metro slaves should only do metro builds

This code has been merged to the production branch and masters have been reconfigured around 7:40 AM PDT.

Comment 77

[Armen [:armenzg]]

This is done as in having builds.
I am going to switch to VS2012 in bug 761321.