Closed Bug 774910 (enable-8.0sdk) Opened 12 years ago Closed 11 years ago

Migrate mozilla build to the Windows 8.0 SDK

Categories

(Firefox Build System :: General, defect)

x86
Windows 7
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: mlarrain, Assigned: jimm)

References

(Depends on 1 open bug)

Details

(Whiteboard: [metro-mvp][LOE:-][metro-it2])

Attachments

(1 file, 3 obsolete files)

21:03 < MaRu> got another possible change I'd like your thoughts on, the Windows SDK that gets installed (currently version 7) also installs .net framework 3.5 but it doesn't allow for a silent or quiet install. Would we be safe to install version 7.1 that installs .net framwork 4.0 instead?
21:12 < armenzg> MaRu: /me thinks
21:20 < armenzg> MaRu: I will ask in #developers
21:23 < armenzg> MaRu: I asked and they say it should not be a problem but bsmedberg says "I wouldn't immediately ship a release without testing, of course ;-)"
21:24 < armenzg> MaRu: in other words, we could have problem without wanting it
21:24 < armenzg> MaRu: the only way out would be do a staging release with modified win64 machines and ask QA to qualify the builds
21:25 < armenzg> in other words, it is expensive (people-resources), it requires extra coordination and need to be done early on a release schedule and hope that there is no chemspill
21:26 < armenzg> I don't want to be a party pooper but wanted to give you all the info that needs to be taken into consideration
21:27 < arr> (we should record this info in a bug)

It's going to be tough to install version 7 as there is no quiet install method. I can for the time being try and build an MSI for it and hopefully that will work for us. We should however look at moving to the newer 7.1 build out for this.
Adding jimm and bsmedberg in case they want to add anything else.
Assignee: server-ops-releng → mlarrain
jimm and bsmedberg do you have any insight on this? Just trying to stay on point with using updated tools if possible as using the same toolset is easier to troubleshoot/manage.
(In reply to Matthew Larrain[:MaRu] from comment #2)
> jimm and bsmedberg do you have any insight on this? Just trying to stay on
> point with using updated tools if possible as using the same toolset is
> easier to troubleshoot/manage.

The difference between 7.0 and 7.1 is the .net version - 

http://en.wikipedia.org/wiki/Microsoft_Windows_SDK

Since vc10 came with the 7.1 sdk I seriously doubt we would run into any problems by switching.
Summary: Issues using Windows SDK version 7 vs Windows SDK version 7.1 → Choose/test Windows SDK version 7.1 vs Windows SDK version 7
I agree with Jim as we have to install net framework 4.0 for VS2012 anyways.
So who needs to sign off on this decision, hwine?  Can we switch to SDK version 7.1 for the new imaging process?
I'll get a decision - ftr, this is for windows builders.
(In reply to Hal Wine [:hwine] from comment #6)
> I'll get a decision - ftr, this is for windows builders.

Have we considered maybe skipping 7.1 and jumping to 8.0? I know the gfx folks would appreciate this quite a bit.
I don't think we can do this without testing and without landing a patch in a bunch of branches:
http://mxr.mozilla.org/mozilla-central/source/build/win32/mozconfig.vs2010-win64
http://mxr.mozilla.org/mozilla-central/source/build/win64/mozconfig.vs2010

Can we get a slave re-imaged with *just* 7.1 SDK and run it through staging and see what we hit?
(In reply to Amy Rich [:arich] [:arr] from comment #5)
> So who needs to sign off on this decision, hwine?  Can we switch to SDK
> version 7.1 for the new imaging process?

Short answer: we can't switch SDK without a bunch of preliminary work.

We have to ensure that there continues to be a way to build all supported releases, including ESR. At the moment, we still believe these new builders-for-win-8 will also support all other windows platforms.

The next step in a 7.1 sdk qualification would be what is in comment #8

The first step to qualify an 8.x (comment #7) would be for dev to show 8.x still generates a usable binary for the currently supported OS: XP, win7(32&64)
(In reply to Hal Wine [:hwine] from comment #9)
> (In reply to Amy Rich [:arich] [:arr] from comment #5)
> > So who needs to sign off on this decision, hwine?  Can we switch to SDK
> > version 7.1 for the new imaging process?
> 
> Short answer: we can't switch SDK without a bunch of preliminary work.
> 
> We have to ensure that there continues to be a way to build all supported
> releases, including ESR. At the moment, we still believe these new
> builders-for-win-8 will also support all other windows platforms.
> 
> The next step in a 7.1 sdk qualification would be what is in comment #8
> 
> The first step to qualify an 8.x (comment #7) would be for dev to show 8.x
> still generates a usable binary for the currently supported OS: XP,
> win7(32&64)

We're going to take the 8.0 sdk for a spin with vc2010 in the next few weeks, will report back on the results.
Summary: Choose/test Windows SDK version 7.1 vs Windows SDK version 7 → Choose/test Windows SDK version
Depends on: 799188
I've just finished up doing basic testing with the win8 sdk in bug 799188. Everything look ok. I've posted basic sdk setup there as well.

Next steps:

1) Work out how to install and configure the sdk on a 2008 builder.
2) get a builder set up doing 32-bit builds that QA can smoke test.
2) point those builds at test slaves and work up a full suite of test runs.
Awesome news Jim thanks for keeping me up to speed on this. Once I get around to working on the 08 builders again I can try automating the SDK install.
(In reply to Matthew Larrain[:MaRu] from comment #12)
> Awesome news Jim thanks for keeping me up to speed on this. Once I get
> around to working on the 08 builders again I can try automating the SDK
> install.

A couple notes -

1) we'll need the 8.0 sdk to build the metro browser frontend on mc. We're currently working on back porting the metro widget bits to vs 2010, but we'll still need the sdk for api definitions. Which means all builders will have to migrate to 8.0. The back porting work should be complete in a week or so.

[Once we've confirmed this works this bug will gain the attention of people driving the metro release.]

2) we may need to make a minor change to an 8.0 sdk header. It's a simple string replace. I'm hoping this is something we can automate as a post the install step?
Depends on: 794983
Whiteboard: metro-mvp
Currently our builders are running in a workgroup so there is no quick or centrally manged way to push this to all the builders. I will look at the 8.0 SDK and see how easily I can get the SDK to install via automation.
Good news the 8.0 SDK has command line flags for quiet install as well as where to install it to. I very much like this new SDK for those reason. we can automate the install fairly easily with this info. sdksetup.exe /quiet /installpath [path] /norestart
If you get me a w64 slave with the newer SDK on it I can spin it on staging.

Let me know if you would like me to set a machine for you to test on.
I'd love it if you could get me a w64 box to test this on. I can work on it today as well.
Depends on: 804734
MaRu, w64-ix-slave79 has been set aside for you.
I have gotten SDK 8.0 to deploy with an automated task sequence in MDT.
The SDK has been installed on w64-ix-slave79 in C:\Program Files(x86)\Windows Kits\8.0
Blocks: 805553
I will run the slave through staging in bug 805553.
(In reply to Matthew Larrain[:MaRu] from comment #19)
> I have gotten SDK 8.0 to deploy with an automated task sequence in MDT.
> The SDK has been installed on w64-ix-slave79 in C:\Program
> Files(x86)\Windows Kits\8.0

Will we also be able to automate a change to a header? Specifically:

C:\Program Files (x86)\Windows Kits\8.0\Include\winrt\asyncinfo.h

line 67:

enum class AsyncStatus {

to

enum /*class*/ AsyncStatus {
Whiteboard: metro-mvp → [metro-mvp][LOE:?]
yeah i can replace asyncinfo.h with a modified version of it.
Depends on: 806799
No longer blocks: 805553
Depends on: 805553
Armen can we get confirmation of if we need to remove the old SDK before installing the new one or are we good to use both on there?
We don't need to remove the previous SDK and in fact we should not since that is being used by older branches like the ESR.

I have put w64-ix-slave79 on the production pool but will need one more day of data to determine if the SDK deployment causes any regressions.

So far on staging it did not cause any problems.
Thanks for the update. Def. what I wanted to hear :)
Depends on: 807061
One new twist with the 8.0 sdk - it does not include a redist folder. So we'll need to distribute the redist folder that comes with vs11 (assuming we aren't planning on installing it with this upgrade) to the builders so they have the crt we run with.
No longer depends on: 794983
No longer depends on: 805553
Depends on: 794983, 799121
(In reply to Jim Mathies [:jimm] from comment #26)
> One new twist with the 8.0 sdk - it does not include a redist folder. So
> we'll need to distribute the redist folder that comes with vs11 (assuming we
> aren't planning on installing it with this upgrade) to the builders so they
> have the crt we run with.

Update to this, this turns out to be incorrect. We'll be distributing the vs10 runtime as usual.
No longer depends on: 794983
Ok, Elm is now producing testable builds using the new 8.0 SDK and Visual Studio 2010.
(In reply to Alex Keybl [:akeybl] from comment #56)
> (In reply to Armen Zambrano G. [:armenzg] from comment #55)
> > Hi akeybl, lsblakk,
> > What is the process to request switching from current Windows 7.0 SDK to
> > Windows 8.0 SDK?
> > Do you know?
> 
> I think the right path forward would be an email to dev-platform that
> includes the proposal, reasoning for making the change, any known issues,
> and a request for any reasons to *not* move forward with the proposal.
Summary: Choose/test Windows SDK version → Migrate mozilla build to the Windows 8.0 SDK
Depends on: 810141
Depends on: 812664
Armen, it looks like this is on you and jimm for the moment.  When we get prepared to install this on the builders, well, we'll figure that part out.
Assignee: mlarrain → armenzg
Assignee: armenzg → server-ops-releng
We have figured out how to deploy it and we're going to do so today.
Let's keep this bug open to add this script to the task sequence when re-imaging win64 machines.
https://github.com/armenzg/bearded-ironman/blob/master/win2008_64bit/install_windows_standalone_sdk_8.0.bat

Feel free to move the files download to another location.

Let me know if you have any questions.
I've pushed a couple builds to try using the new sdk, so far everything looks good. I plan to push a few more to tweak path info until I get a minimalist set of lib, libpath, and include settings.

For the final landing of build config changes on mc we probably want to wait until the next merge, which will be the week of 2013-01-06.  I'll post final win64 builder config patches here once I'm done testing.
This brings up a question - are we using 32-bit builders currently for any release/mc/try builds? I haven't updated those configs since we didn't push the sdk out to 32-bit builders (bug 810141).
We *only* use win32 builders for mozilla-esr10/comm-esr10 which will be done in February.
We don't need to deploy the newer SDK to them.
(In reply to Jim Mathies [:jimm] from comment #35)
> Path changes:
> 
> https://hg.mozilla.org/try/rev/efab8c7ddcd9

These are for 32-bit app build. Should I also update the 64-bit app build config and test? I'm guessing yes, even though we'll be dropping support, we probably want the config scripts to be up to date.
(In reply to Jim Mathies [:jimm] from comment #36)
> (In reply to Jim Mathies [:jimm] from comment #35)
> > Path changes:
> > 
> > https://hg.mozilla.org/try/rev/efab8c7ddcd9
> 
> These are for 32-bit app build. Should I also update the 64-bit app build
> config and test? I'm guessing yes, even though we'll be dropping support, we
> probably want the config scripts to be up to date.

It will probably reduce the work to bring it up to date the day that we want to get it back going.

Thanks jimm.
Blocks: 815658
Hi jimm,
I'm trying to figure out what is the work left on this bug.
AFAIU we are done from the releng side.

The only thing I can think of is to take the Elm sub-pool and ask IT to re-image and make sure that the Windows 8 SDK gets installed properly.

Are you using this bug as a tracking bug? It sounds it should be a Build Config bug tracking on when you do the switch to the Windows 8 SDK on mozilla-central (rather than a Server Ops bug).

I would like to clarify this to make it clear what is left for Relops and Releng.

Thanks.
Attached patch build config patch v.1 (obsolete) — Splinter Review
Assignee: server-ops-releng → jmathies
Component: Server Operations: RelEng → Build Config
Product: mozilla.org → Core
QA Contact: arich
Version: other → Trunk
(In reply to Armen Zambrano G. [:armenzg] from comment #38)
> Hi jimm,
> I'm trying to figure out what is the work left on this bug.
> AFAIU we are done from the releng side.

Looks that way.

> The only thing I can think of is to take the Elm sub-pool and ask IT to
> re-image and make sure that the Windows 8 SDK gets installed properly.

Yep, although there's no rush on that... we like having the dedicated pool. :) We should do this anyway so we are testing out the builder pool on elm from now until we land this for mc. Can you file a follow up?

> Are you using this bug as a tracking bug? It sounds it should be a Build
> Config bug tracking on when you do the switch to the Windows 8 SDK on
> mozilla-central (rather than a Server Ops bug).

Yes this bug can track other bugs if need be.
Whiteboard: [metro-mvp][LOE:?] → [metro-mvp][LOE:?][land after next merge: ~2013-01-06]
Blocks: 815680
From a talk with Armen and another talk with MaRu:

* we're going to try to avoid wholesale reimaging, preferring to install the SDK via SSH on existing hosts.  It's known how to do that -- I just need to know what /installpath to use, and when to do it

* the MDT process to build new machines with the SDK installed is basically in place - it just doesn't specify an /installpath yet, and probably should be tested once or twice more
(In reply to Dustin J. Mitchell [:dustin] from comment #41)
> From a talk with Armen and another talk with MaRu:
> 
> * we're going to try to avoid wholesale reimaging, preferring to install the
> SDK via SSH on existing hosts.  It's known how to do that -- I just need to
> know what /installpath to use, and when to do it
> 
> * the MDT process to build new machines with the SDK installed is basically
> in place - it just doesn't specify an /installpath yet, and probably should
> be tested once or twice more

We took the default value. This means it is most likely not needed to be specified.

Nevertheless, here's the value:
"C:\Program Files (x86)\Windows Kits\8.0"

We can test re-imaging one or two machines.
Oh, great, that makes things pretty easy.  I'll find you in irc today to borrow a machine to test on (I need to make sure the process works anyway, before MaRu goes)
I've done a test deployment on w64-ix-slave20 (set aside by armen).  Please check that out and let us know if it works.
(In reply to Amy Rich [:arich] [:arr] from comment #44)
> I've done a test deployment on w64-ix-slave20 (set aside by armen).  Please
> check that out and let us know if it works.

Hi arr,
We need to deploy this file as well:
https://bug810141.bugzilla.mozilla.org/attachment.cgi?id=681123
The difference is in line 66:
num class AsyncStatus {
vs
num *class* AsyncStatus {

This can be seen in:
https://github.com/armenzg/bearded-ironman/blob/master/win2008_64bit/install_windows_standalone_sdk_8.0.bat

FTR a new sdksetup.exe was issued in Nov. 15th which is different to what is on the IT servers (I'm pretty sure).

jimm, do you think the different SDKs should cause trouble?

arr, you can also take the install_windows_standalone_sdk_8.0.bat as is an use it if you want. The file would have to live somewhere different than dev-stage01 though.
Feel free to re-image w64-ix-slave20 anytime.

FTR the modified file is not something that MaRu and us knew at the time.
OK, so our test was a little premature..  any other details you want to share before we try again? :)
(In reply to Dustin J. Mitchell [:dustin] from comment #47)
> OK, so our test was a little premature..  any other details you want to
> share before we try again? :)

All details are there! I should have been explicit in my last comment (that's what I attempted to do).

I wasn't sure what info or script was on your side.
(In reply to Armen Zambrano G. [:armenzg] from comment #45)
> FTR a new sdksetup.exe was issued in Nov. 15th which is different to what is
> on the IT servers (I'm pretty sure).
> 
> jimm, do you think the different SDKs should cause trouble?

I'm currently running this with no problems. Whatever they changed they didn't feel it had to be documented.

I'm confused though, they pulled that link down and then put it back up after the 15th, so I'm assuming the distribution to the win64 builders was with the new sdk? The comments in bug 810141 seem to indicate so.
(In reply to Jim Mathies [:jimm] from comment #49)
> (In reply to Armen Zambrano G. [:armenzg] from comment #45)
> > FTR a new sdksetup.exe was issued in Nov. 15th which is different to what is
> > on the IT servers (I'm pretty sure).
> > 
> > jimm, do you think the different SDKs should cause trouble?
> 
> I'm currently running this with no problems. Whatever they changed they
> didn't feel it had to be documented.
> 
> I'm confused though, they pulled that link down and then put it back up
> after the 15th, so I'm assuming the distribution to the win64 builders was
> with the new sdk? The comments in bug 810141 seem to indicate so.

Correct. We ended up using the sdksetup.exe from the 15th. What I did notice between the two different SDKs is that the first one does not work anymore and the second one downloads several hundreds of MBs and that is why I had one more reason to do a StandaloneSDK
http://dev-stage01.build.mozilla.org/pub/mozilla.org/mozilla/libraries/win32/sdks/sdksetup.nov13th.exe
* This is the old one which I renamed it to indicate the last day I used it
http://dev-stage01.build.mozilla.org/pub/mozilla.org/mozilla/libraries/win32/sdks/sdksetup.exe
* This is the new one which downloads hundreds of MBs.
http://dev-stage01.build.mozilla.org/pub/mozilla.org/mozilla/libraries/win32/sdks/sdksetup.zip
* This is the StandaloneSDK version of the new one

I hope this helps!
Comment on attachment 685665 [details] [diff] [review]
build config patch v.1

I need to update to these, I clipped out the msvc 10 atl headers, but we still rely on those in some places.
Attachment #685665 - Attachment is obsolete: true
Removing bug 815680 from blocking list as it is more accurate.
No longer blocks: 815680
Whiteboard: [metro-mvp][LOE:?][land after next merge: ~2013-01-06] → [metro-mvp][LOE:-][land after next merge: ~2013-01-06]
Whiteboard: [metro-mvp][LOE:-][land after next merge: ~2013-01-06] → [metro-mvp][LOE:-][land after next merge: ~2013-01-06][metro-it2]
Depends on: elm-merge
Blocks: enable-metro
Alias: enable-8.0sdk
I grabbed sdksetup.exe and stashed it in \\releng\data\apps\misc\bug774910 for safe-keeping.
Attached patch 64bit builder mozconfigs (obsolete) — Splinter Review
Attached patch 64bit builder mozconfigs (obsolete) — Splinter Review
- cleaned up comment headers
Attachment #699707 - Attachment is obsolete: true
Pushed to try w/the patch in bug 827643 to get good win64 builds - 

https://tbpl.mozilla.org/?noignore=1&tree=Try&rev=bb40e60527b4
- removed a couple double colons in INCLUDES
Attachment #699712 - Attachment is obsolete: true
Comment on attachment 699753 [details] [diff] [review]
64bit builder mozconfigs

This is shaping up nicely on try. Need a final r+ to land on m-c.
Attachment #699753 - Flags: review?(armenzg)
Comment on attachment 699753 [details] [diff] [review]
64bit builder mozconfigs

LGTM.
Attachment #699753 - Flags: review?(armenzg) → review+
No longer blocks: enable-metro
Blocks: elm-merge
No longer depends on: elm-merge
https://hg.mozilla.org/mozilla-central/rev/0faa1d47ea80
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Whiteboard: [metro-mvp][LOE:-][land after next merge: ~2013-01-06][metro-it2] → [metro-mvp][LOE:-][metro-it2]
http://blogs.msdn.com/b/vcblog/archive/2012/10/08/windows-xp-targeting-with-c-in-visual-studio-2012.aspx

See comment by Ibrahim Damlaj.

"The reason we have reintroduced the Windows 7 SDK is that the Windows 8 SDK has officially dropped support for Windows XP and Windows Server 2003. What this means is that the binaries under the “Windows Kits/8.0/bin” directory are no longer guaranteed to generate code that runs on Windows XP/2003. Additionally, the Windows 8 SDK headers and libs may have removed deprecated Windows XP APIs with no mitigation. As @Mike described above, you may choose to target the standard SDK by not switching  to v110_xp. However, if your application compiles and runs we cannot guarantee it will continue to do so in future releases and updates of Visual Studio."

Changing to Windows 8.0 SDK might or might not make Firefox incompatible with XP SP3. Just a heads up.
We are still using the msvc 10 tool chain so we don't have to worry about binary compat with xp. As far as the headers go we've seen no issues thus far. Mozilla's build targets a lower api set so I doubt the sdk itself can cause us any harm.
Blocks: 839775
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.