Closed Bug 505504 Opened 13 years ago Closed 13 years ago

Add Windows 7 SDK to build machines

Categories

(Release Engineering :: General, defect, P1)

x86
Windows 7
defect

Tracking

(status1.9.2 beta1-fixed)

RESOLVED FIXED
Tracking Status
status1.9.2 --- beta1-fixed

People

(Reporter: robarnold, Assigned: bhearsum)

References

Details

Attachments

(5 files, 6 obsolete files)

1.44 KB, text/plain
Details
4.70 KB, patch
catlee
: review+
bhearsum
: checked-in+
Details | Diff | Splinter Review
7.31 KB, patch
catlee
: review+
bhearsum
: checked-in+
Details | Diff | Splinter Review
705 bytes, patch
ted
: review+
bhearsum
: checked-in+
Details | Diff | Splinter Review
838 bytes, patch
gozer
: review+
Callek
: review+
Details | Diff | Splinter Review
Need to support the new windows 7 features such as taskbar previews, jumplists and download progress.
I'm guessing we won't switch over the machines till the final version of the SDK is out?
We'll want to have builds with the SDK before the final one is out. I'm not sure when the final SDK will be out.

The RC SDK should be good enough until the final one is released:
http://www.microsoft.com/downloads/details.aspx?FamilyID=6db1f17f-5f1e-4e54-a331-c32285cdde0c&displaylang=en
When does this need to be done by?
Which SDK would no longer be used ? The Vista one ? Which branches would this change be made fore ?
This should be done before we branch so before the end of next week.

We would be retiring the Vista SDK and using the Win 7 SDK for mozilla-central and the upcoming 1.9.2 branch.

When the final SDK comes out, I'll file another bug to replace the RC SDK with it.
It would be useful to have a try server build using this as well.
Flags: blocking1.9.2?
Depends on: 506112
The Windows 7 RTM SDK was released a few days ago -- http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=71deb800-c591-4f97-a900-bea146e4fae1

For some reason, the 64-bit download isn't directly accessible right now. I got it through editing the 32-bit URL: http://download.microsoft.com/download/2/E/9/2E911956-F90F-4BFB-8231-E292A7B6F287/GRMSDKX_EN_DVD.iso
Blocks: win7support
Waiting for Rob's test builds on an experimental VM.
Assignee: nobody → tellrob
Install steps:
1. Grab and install the SDK:
http://www.microsoft.com/downloads/details.aspx?familyid=C17BA869-9671-4330-A63E-1FD44E0E2505&displaylang=en
2. Grab and install this VC8 hotfix:
http://support.microsoft.com/kb/949009/
3. Make sure configure runs with --with-windows-version=601 (editing the mozconfig would work)

I have vc8 and vc9 builds, haven't run them through tests yet but a quick smoke test shows nothing obviously wrong.
Is there any reason why bug 507494 (present on "Depends on" list, currently RESOLVED FIXED) is inaccessible?
(In reply to comment #10)
> Is there any reason why bug 507494 (present on "Depends on" list, currently
> RESOLVED FIXED) is inaccessible?

It's the same way for me - so I'm not sure. I would guess that someone used the IT ticket request to file it though - those bugs are put in the 'infra' group automatically.
Flags: blocking1.9.2? → blocking1.9.2+
Rob, Any update for this?
Depends on: 509837
When I last left off on this, I was seeing errors on the test machines due to missing symbols. I suspect the mozconfig I was instructed to use (http://hg.mozilla.org/build/buildbot-configs/file/tip/mozilla2/win32/mozilla-central/nightly/mozconfig) is not the right one for unit tests (perhaps http://hg.mozilla.org/build/buildbot-configs/file/tip/mozilla2/win32/mozilla-central/unittest/mozconfig is?).
Hey Rob, so to do the testing, I assume I'll need access to some sort of image? I think I remember you getting access to a test build instance someplace? Or are we past that, or can I do the testing here locally and hand changes to you? Not sure what steps I need to take to get a build going. Care to fill me in?
(In reply to comment #13)
> When I last left off on this, I was seeing errors on the test machines due to
> missing symbols. 

Do you remember more details here ? eg This was from the builds that were submitted to our staging infrastructure ? Or more info on the error.

You correctly identified the location of the unit test mozconfig.
This was from builds submitted to the staging infrastructure. They wanted some .zip file with symbols I think (I wish I could find the error but it's been too long). After it complained about that it gave up a little while later.
Assignee: tellrob → jmathies
(In reply to comment #18)
> Builds:
> 
> WINNT 5.2 mozilla-central test mochitests - success
> http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1252450992.1252452766.19419.gz
> 
> WINNT 5.2 mozilla-central test everythingelse - complete barf
> http://tinderbox.mozilla.org/showlog.cgi?tree=MozillaTest&errorparser=unittest&logfile=1252452740.1252456200.25476.gz&buildtime=1252452740&buildname=WINNT%205.2%20mozilla-central%20test%20everythingelse&fulltext=1

Looks like these failures may well have been an issue with the code on m-c - after doing a full update and a rebuild, the failures went away on the image. Will try to get another test run fire off today once I get the code off the image.
Builds:

WINNT 5.2 mozilla-central test mochitests - success

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1252964941.1252967246.13706.gz

WINNT 5.2 mozilla-central test everythingelse - success

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1252967213.1252969378.30518.gz

The same set of steps posted on comment 17 were taken, I just did an update from m-c, rebuilt, and repackaged.
(In reply to comment #8)
> Waiting for Rob's test builds on an experimental VM.

John, is there anything else I need to do before we schedule the upgrade?
Priority: -- → P2
Comment on attachment 399354 [details]
steps followed on image

Just had a chance to read these instructions - why does it talk about installing VS2005? Our VMs already have it.
(In reply to comment #22)
> (From update of attachment 399354 [details])
> Just had a chance to read these instructions - why does it talk about
> installing VS2005? Our VMs already have it.

I was just listing every requirement. Rob got things started on the test image and started with the sdk. (see comment 9)
(In reply to comment #23)
> (In reply to comment #22)
> > (From update of attachment 399354 [details] [details])
> > Just had a chance to read these instructions - why does it talk about
> > installing VS2005? Our VMs already have it.
> 
> I was just listing every requirement. Rob got things started on the test image
> and started with the sdk. (see comment 9)

Ah, thanks for the info. That makes much more sense.
I see this is blocking-1.9.2+ - does that mean it blocks the upcoming beta, or some other release?
jimm said we could claim this, so we could work on rolling this out; we can reassign back to him if the steps dont work out for some reason.

bhearsum: I just confirmed with damons, this *is* required for the upcoming FF3.6beta1.
Assignee: jmathies → nobody
This is a P1 blocker, so needed for beta 1, yes.
Priority: P2 → P1
Jimm: can you give your existing Firefox build to Tony/Juan/Marcia? They'd like
to sanity check the build on other older versions of windows we still support. 

Also, can you list out here what testing you've already done on older o.s. to
avoid possible duplicate work?
Ok, here is the latest build - 

m-c changeset: 83af6b9b03b1 2009-09-24 02:13 +0900

On these files, if they need to be collected multiple times, someone please make a local copy for everyone as I don't have a lot of bandwidth.

install:
http://www.mathies.com/mozilla/firefox-3.7a1pre.en-US.win32.installer.exe
symbols:
http://www.mathies.com/mozilla/firefox-3.7a1pre.en-US.win32.crashreporter-symbols.zip
tests:
http://www.mathies.com/mozilla/firefox-3.7a1pre.en-US.win32.tests.tar.bz2
package:
http://www.mathies.com/mozilla/firefox-3.7a1pre.en-US.win32.zip

As far as testing goes, I've run the install on win7, vista, both seem to run fine. The resulting package also had a full suite of tests run on winnt 5.2.
Thanks Jim.  QA is going to go through your builds for a smoketest on windows 7, windows XP, windows Vista, and win2000.   Juan has drafted up a testplan at https://wiki.mozilla.org/QA/Windows7_SDK/Initial_Builds_Generated.   We plan to approach the first iteration in this direction, but if you have any thoughts or areas that should require extra attention, please comment here.    Tony
We tested the installer across Windows platforms and we didn't find any problems. Next we should take a look at some updates, but so far everything looks like the builds you can currently get from ftp.
one thing we didnt cover are complete and partial mar updates.  John says he is investigating this process, but we would like to be sure to test this on the next iteration if possible before this goes live somehow.
Tested a complete update by modifying the update channel preference to ''nightly'' and I got offered an update to the latest nightly. The update applied correctly on two platforms I tried, Win 7 and 2000. The resulting builds seem fine after a spot check.
I've been working on getting the win7 image rolled out across the build farm. I'm hitting an issue with the SDK deployment that we're hitting because of when it runs. The short version is: The SDK installer breaks when run as the SYSTEM user, most likely because USERPROFILE is unset. I'm still looking for a way to work around this. I'm hoping to have this sorted out on Monday or Tuesday, but it's slow going right now still.

At the same time, I'm working on making sure that m-c/1.9.2/project branches can use this SDK, while 1.9.1 can still use the old one. This part is pretty easy, just need to test it some more.
Assignee: nobody → bhearsum
vars that need to be set for builds that want the 7.0 sdk:
INCLUDE=D:\sdks\v7.0\\include;D:\sdks\v7.0\\include\atl;D:\msvs8\VC\ATLMFC\INCLUDE;D:\msvs8\VC\INCLUDE;D:\msvs8\VC\PlatformSDK\include;
LIB=D:\sdks\v7.0\\lib;D:\msvs8\VC\ATLMFC\LIB;D:\msvs8\VC\LIB;D:\msvs8\VC\PlatformSDK\lib;D:\msvs8\SDK\v2.0\lib;;D:\mozilla-build\\atlthunk_compat
PATH=d:\sdks\v7.0\bin (plus all the other stuff, omitted for brevity)
SDKDIR=D:\sdks\v7.0\

Not much, really. I'm not certain yet how to deal with PATH. If we set it in the Buildbot configs it will override the entire PATH in the host environment, which means we're maintaining PATH in two places. Maybe the best thing to do is specify PATH/LIB/INCLUDE for all builds, regardless of the SDK being used. That would probably be better than maintaining it in the configs for some, and on the slaves for others. Probably simpler, too.
Oh, I forgot to mention exactly what options I'm using for the install. Here they are:
All of the "Windows Headers and Libaries" section (header files plus libraries for x86, x64, and ia64)
All of "Windows Development Tools" (Win32 Development Tools, .NET Development Tools)
Application Verifier
Debugging Tools for Windows

I did not install any of:
Documentation
Samples
Visual C++ Compilers
Redistributable components.
Attached patch win7 sdk + hotfix OPSI package (obsolete) — Splinter Review
This is a horrible, hacky way to work around the fact that the Windows 7 installer requires a *real* user with administrative rights (eg, the SYSTEM user that OPSI runs as isn't good enough). Basically, the following happens when the package is set to install:
* win7-sdk.ins does a few things:
** copies the CLIENT_DATA directory to the slave
** creates an account with admin rights, sets it to automatically login at the next boot
** sets local_win7-sdk.ins to run at next login
** reboots the system
When the system comes back up, the new account logs in, runs local_win7-sdk.ins, which installs the sdk + the hotfix, and then reboots again.
Finally, OPSI cleans up d:\tmp, removes the account it created, and restores the old automatic login.

The script was created based off of http://www.opsi.org/opsi_wiki/TemplateForInstallationsAsTemporaryLocalAdmin

Not sure who to tag for review, I'll be poking tomorrow though.
Comment on attachment 403716 [details] [diff] [review]
win7 sdk + hotfix OPSI package

changeset:   22:1a4ca5538b0c
Binaries landed in opsi-binaries - too many files to list here, though
Attachment #403716 - Flags: checked-in+
Alright, now that we've got a working deployment solution here's what I'm going to do:
* This afternoon: Roll out the SDK on a few production machines as a last minute paranoid test.
* Before I go today: Set the SDK to install on the rest of the production machines - over the course of the next 24 hours or so they'll all pick it up as they reboot.
* Friday: After verifying that all of the machines got the upgrade, check in the necessary mozconfig changes to enable the SDK on: mozilla-central, mozilla-1.9.2, TraceMonkey, Electrolysis, and Places.
Alright, I've deployed the new SDK on win32-slave31, 32, 33, and 34 without issue. I've marked it for installation on all of the rest. I expect every slave to have synced up by the end of tomorrow.
Pretty straightforward, I think.
Attachment #403894 - Flags: review?(catlee)
The SDK has been deployed on all of the moz2-win32-* and try-win32-* at this point. It required some manual kicking on some of them, which is a bit alarming, but as best I can tell there were no failures.

Once the mozconfig patch gets review we're ready to flip it on at any point.
Well, it turns out that we're already using the new SDK everywhere - including 1.9.1. In my haste to get this pushed out I forgot that we needed to override some MozillaBuild settings to make it ignore the presence of the new SDK.

So, since it's already been flipped on for all branches it's now easier to obsolete my current mozconfig patch and do the inverse: force the old sdk for 1.9.1. I'm putting together a patch that does it right now.
This is great work; thanks for wrestling OPSI and the SDK to the mat.  Think it's worth giving feedback to OPSI or MSFT people to improve it in the future?  (If nothing else, worth blogging the template and whatnot, since bugzilla isn't indexed by search engines.)
(In reply to comment #44)
> This is great work; thanks for wrestling OPSI and the SDK to the mat.  Think
> it's worth giving feedback to OPSI or MSFT people to improve it in the future? 
> (If nothing else, worth blogging the template and whatnot, since bugzilla isn't
> indexed by search engines.)

I've been in contact with the OPSI folks for awhile, and I do hope to get a win7 sdk wrap-up post out there pretty soon. I'm not sure who I'd contact at MSFT to complain about the SDK ;), but if someone knows, I will write that e-mail!
Attached patch force the Vista sdk on 1.9.1 (obsolete) — Splinter Review
Same style as the previous mozconfig patch, a bit simpler this way, though.
Attachment #403716 - Attachment is obsolete: true
Attachment #404145 - Flags: review?(catlee)
Attachment #404145 - Attachment is obsolete: true
Attachment #404160 - Flags: review?(nthomas)
Attachment #404145 - Flags: review?(catlee)
Comment on attachment 404160 [details] [diff] [review]
use msys paths, because they actually work

>diff --git a/mozilla2/win32/mozilla-1.9.1/unittest/mozconfig b/mozilla2/win32/mozilla-1.9.1/unittest/mozconfig
>--- a/mozilla2/win32/mozilla-1.9.1/unittest/mozconfig
>+++ b/mozilla2/win32/mozilla-1.9.1/unittest/mozconfig

As we discussed on IRC, this may not work because of the way UnittestBuildFactory overrides the env. Lets see what you find out.

>+. $topsrcdir/configs/mozilla2/win32/include/vista-sdk-mozconfig
>\ No newline at end of file

I'd fix all of these. Otherwise looks OK.
Not tagging review on this yet, because I'm running it overnight in staging for further testing.

By doing this, we can default all of the branches to the 7.0 sdk. As with the other 1.9.1 stuff, we'll override in the mozconfig for that branch.
Attachment #404160 - Attachment is obsolete: true
Attachment #404160 - Flags: review?(nthomas)
Attached patch 1.9.1 mozconfig overrides (obsolete) — Splinter Review
This is very similar to the most recent patch, except I found out that our unittest builds require a different INCLUDE/LIB setting. I have no idea why, given that they run on the same machines, but here we are.

These patches are running in staging overnight, if things look good I'll get review and land tomorrow.
Attachment #404173 - Flags: review?(catlee)
Comment on attachment 404173 [details] [diff] [review]
remove PATH/LIB/INCLUDE specs from unittest vars

This one is ready for review
After running overnight and some more testing this morning, here's what I've discovered to work:
* LIB/INCLUDE/SDKDIR need Windows-style paths or else configure will fail the 'cl can create binaries' test.
* PATH needs MSYS style paths (probably because the : in c:/ breaks the splitting), or else the Makefiles can't find any binaries they need.

This has been explicitly tested in the clobber build and dep build scenario on 1.9.1 in staging. I tested TraceMonkey against the env.py changes overnight and had no issues with it.
Attachment #404174 - Attachment is obsolete: true
Attachment #404247 - Flags: review?(catlee)
Attachment #404247 - Flags: review?(catlee) → review+
Comment on attachment 404173 [details] [diff] [review]
remove PATH/LIB/INCLUDE specs from unittest vars

Looks ok, we agreed on IRC to leave the win-ref-platform environment untouched.
Attachment #404173 - Flags: review?(catlee) → review+
Comment on attachment 404247 [details] [diff] [review]
mozconfig updates, final version (please)

changeset:   1571:08a8db4b3d89
Attachment #404247 - Flags: checked-in+
Comment on attachment 404173 [details] [diff] [review]
remove PATH/LIB/INCLUDE specs from unittest vars

changeset:   429:81fb93762011
Attachment #404173 - Flags: checked-in+
I kicked new builds of:
m-c unittest
1.9.1 unittest/build/leak test/nightly

All of them are using the SDK they should be, and all appear to be building fine. I'll keep this bug open until they actually complete their builds, but we're almost done here.
Attachment #403894 - Attachment is obsolete: true
Attachment #403894 - Flags: review?(catlee)
All of my builds went green using the proper SDKs. As far as I know, we're done here!
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Hmm, looking at a try build log, did we forget to up the sdk build flag?

Adding configure options from /e/builds/sendchange-slave/win32-hg/build/.mozconfig:
  --enable-application=browser
  --enable-optimize
  --disable-debug
  --enable-jemalloc


This is missing the sdk build version directive:

ac_add_options --with-windows-version=601

Maybe the default value should be upped in configure?
http://mxr.mozilla.org/mozilla-central/source/configure.in#476
(In reply to comment #58)
> Hmm, looking at a try build log, did we forget to up the sdk build flag?
> 
> Adding configure options from
> /e/builds/sendchange-slave/win32-hg/build/.mozconfig:
>   --enable-application=browser
>   --enable-optimize
>   --disable-debug
>   --enable-jemalloc
> 
> 
> This is missing the sdk build version directive:
> 
> ac_add_options --with-windows-version=601
> 
> Maybe the default value should be upped in configure?
> http://mxr.mozilla.org/mozilla-central/source/configure.in#476

You're right, we totally did.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attachment #404309 - Flags: review?(ted.mielczarek)
Attachment #404309 - Attachment is obsolete: true
Attachment #404310 - Flags: review?(ted.mielczarek)
Attachment #404309 - Flags: review?(ted.mielczarek)
Attachment #404310 - Flags: review?(ted.mielczarek) → review+
Comment on attachment 404310 [details] [diff] [review]
update the comment, too

Landed on mozilla-central: changeset:   33397:e53f6b632bce
and mozilla-1.9.2: changeset:   32265:c2066385a890
Attachment #404310 - Flags: checked-in+
Status: REOPENED → RESOLVED
Closed: 13 years ago13 years ago
Resolution: --- → FIXED
We had some dep builds fail on mozilla-central. I set all of the slaves to clobber, we should be OK now.
There's some bustage for 1.9.2 l10n, eg while configuring
...
  INCLUDE=d:\sdks\v7.0\\include;d:\sdks\v7.0\\include\atl;d:\msvs8\VC\ATLMFC\INCLUDE;d:\msvs8\VC\INCLUDE;d:\msvs8\VC\PlatformSDK\include;
  LIB=d:\sdks\v7.0\\lib;d:\msvs8\VC\ATLMFC\LIB;d:\msvs8\VC\LIB;d:\msvs8\VC\PlatformSDK\lib;d:\msvs8\SDK\v2.0\lib;;D:\mozilla-build\\atlthunk_compat
...
loading cache ./config.cache
...
checking for Windows SDK being recent enough... no
configure: error: You are targeting Windows version 0x06010000, but your SDK only supports up to version 0x06000000. Install and use an updated SDK, or target a lower version using --with-windows-version. See https://developer.mozilla.org/En/Windows_SDK_versions for more details on fixing this.

http://production-master.build.mozilla.org:8010/builders/Firefox%20mozilla-1.9.2%20win32%20l10n%20build/builds/861/steps/configure/logs/stdio

I've set a clobber on both the dep and nightly builds to resolve this, and that seems to have fixed up the dep build I rebuilt.
I have verified that 1.9.2 win32 L10n nightly are working again
Setting 1.9.2beta1-fixed
Blocks: 520765
QA will be running some spotchecks on these builds across Windows 7, XP, Vista, and 2000.   A call to the public for extra sets of eyes has also been made:  http://quality.mozilla.org/blogs/calling-vista-and-windows-7-nightly-testers.   Stay tuned for any comments.
I am able to crash consistently on http://photoshopdisasters.blogspot.com/ win7 x64 running: 
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a1pre) Gecko/20091008 Minefield/3.7a1pre (.NET CLR 3.5.30729)

I have all my add-ons disabled and I think my plugins are all up to date. 

I can be reached at davidcraig5@yahoo.com for further questions.
Attachment #429352 - Flags: review?(gozer) → review+
Comment on attachment 429352 [details] [diff] [review]
(AAv1-CC) configure.in: copy it to comm-central
[Checkin: Comment 71]

This is good; (stealing review from KaiRo). The thing to note here, is that if you don't do --with-windows-version. We do hit the req of the Win7 SDK on m-c [& m-1.9.2] anyway.
Attachment #429352 - Flags: review?(kairo) → review+
Comment on attachment 429352 [details] [diff] [review]
(AAv1-CC) configure.in: copy it to comm-central
[Checkin: Comment 71]


http://hg.mozilla.org/comm-central/rev/3adc3cc1ec18
Attachment #429352 - Attachment description: (AAv1-CC) configure.in: copy it to comm-central → (AAv1-CC) configure.in: copy it to comm-central [Checkin: Comment 71]
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.