Closed Bug 520765 Opened 11 years ago Closed 11 years ago

Win32 Fennec Desktop nightly builds are failing because of Win 7 SDK deployment

Categories

(Release Engineering :: General, defect, P3)

x86
Windows Server 2003
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: armenzg, Assigned: armenzg)

References

Details

Attachments

(2 files, 3 obsolete files)

I am going to clobber as Nick did in bug 505504#c64 and see if it fixes it.

INCLUDE=d:\msvs9\VC\ATLMFC\INCLUDE;d:\msvs9\VC\INCLUDE;C:\Program Files\Microsoft SDKs\Windows\v6.0A\include;
LIB=d:\msvs9\VC\ATLMFC\LIB;d:\msvs9\VC\LIB;C:\Program Files\Microsoft SDKs\Windows\v6.0A\lib;
LIBPATH=C:\WINDOWS\Microsoft.NET\Framework\v3.5;C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727;d:\msvs9\VC\ATLMFC\LIB;d:\msvs9\VC\LIB;

checking for Windows SDK being recent enough... no
*** Fix above errors and then restart with               "make -f client.mk build"
make[2]: Leaving directory `/e/builds/moz2_slave/w32mob-1.9.2-nightly/mozilla-1.9.2'
make[1]: Leaving directory `/e/builds/moz2_slave/w32mob-1.9.2-nightly/mozilla-1.9.2'
configure: error: You are targeting Windows version 0x06010000, but your SDK only supports up to version 0x06000000. Install and use an updated SDK, or target a lower version using --with-windows-version. See https://developer.mozilla.org/En/Windows_SDK_versions for more details on fixing this.
make[2]: *** [configure] Error 1
make[1]: *** [/e/builds/moz2_slave/w32mob-1.9.2-nightly/mozilla-1.9.2/objdir/xulrunner/Makefile] Error 2
make: *** [build] Error 2

http://tinderbox.mozilla.org/showlog.cgi?log=Mobile/1254816002.1254816615.7998.gz&fulltext=1

http://tinderbox.mozilla.org/showlog.cgi?log=Mobile/1254816002.1254816839.10516.gz&fulltext=1
LIB/INCLUDE seem to be set wrong, somehow.
Looks like env.py is at fault again:
http://hg.mozilla.org/build/buildbotcustom/file/tip/env.py#l112 uses the 6.0 sdk in LIB/INCLUDE. Armen, can you try getting rid of the LIB/INCLUDE/SDKDIR variables in that dictionary?
The Maemo desktop builds are running on staging
Attachment #404824 - Flags: review?(bhearsum)
Comment on attachment 404824 [details] [diff] [review]
remove environment variables for winmo win32 that override the desired new values

changeset:   432:32d7e0c94360
Attachment #404824 - Flags: review?(bhearsum)
Attachment #404824 - Flags: review+
Attachment #404824 - Flags: checked-in+
Masters updated and reconfigured.

Triggering new nightly builds.
Green for both branches of win32 Fennec desktop builds

Resolving FIXED
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Attachment #404824 - Attachment description: remove environment variables for maemo win32 that override the desired new values → remove environment variables for winmo win32 that override the desired new values
Did you consider the impact of these changes on WinMO and WinCE builds ? ie the important consumers of MozillaEnvironments['winmo-arm']

If I look at a WinCE build we now have 
 INCLUDE=d:\sdks\v7.0\\include;d:\sdks\v7.0\\include\atl;d:\msvs8\VC\ATLMFC\INCLUDE;d:\msvs8\VC\INCLUDE;d:\msvs8\VC\PlatformSDK\include;
  LIB=d:\sdks\v7.0\\lib;d:\msvs8\VC\ATLMFC\LIB;d:\msvs8\VC\LIB;d:\msvs8\VC\PlatformSDK\lib;d:\msvs8\SDK\v2.0\lib;;D:\mozilla-build\\atlthunk_compat
and PATH referring to msvs9. So the compiler executables are mismatched with the INCLUDE and LIB settings.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
No, I have not thought of them because I just did what I was suggested to do.

After your comments I looked at the code and I can see now that MozillaEnvironments['winmo-arm'] is shared between WinMo, WinCe and Win32 Desktop builds.

Shall I create a new section for Win32 Desktop builds?
http://mxr.mozilla.org/build/source/buildbot-configs/mozilla2/mobile_config.py#96
Shall I point to some other environments section?
Does it need any environments variables or could it just work with the default values?
AIUI we need to use MSVC9 for WinCE and WinMO builds targeting the ARM architecture, while desktop builds of Fennec (ie XULRunner + the XUL app coming out of mobile-browser repo) could be the MSVC8 we use normally. So the desktop build would stop using the winmo-arm environment (which really means MSVC9), and that env would get updated to specify the 7.0 SDK instead of 6.0A. I'm testing this out in staging.
Assignee: armenzg → nthomas
Depends on: 518496
Ok, so with 3.6b1 imminent I'm going to back out 
 http://hg.mozilla.org/build/buildbotcustom/rev/32d7e0c94360
and re-break the desktop builds for win32. I don't want to regress the WinCE build by stirring the SDK soup a la comment #7.

If we want to switch WinMo/WinCE to the 7.0 SDK then we should do this:
 http://pastebin.mozilla.org/676303
That diff is against b0d53559b7be in buildbotcustom, the revision before 32d7e0c94360 landed. blassey is out today so could not get approval to make this change. We need to sync up with Mobile in general on the 7.0 SDK change (it may be a no-op but it's all a bit tangled right now and hard to tell). I had green builds in staging with that change.

To switch the desktop build to MSVS8 (VS2005) I think we'd want something like
  http://pastebin.mozilla.org/676305
When I tried that in staging I hit bug 506020 at 
  http://mxr.mozilla.org/mozilla-central/source/build/win32/Makefile.in#69
Renaming /d/mozilla-build/vim/vim72/install.exe lead to errors where it tried to copied MSVS8 files from the MSVS9 directory, presumably because of
http://hg.mozilla.org/build/buildbot-configs/file/de9cc3bae3a9/mozilla2/win32/mobile-desktop/nightly/mozconfig#l6
http://hg.mozilla.org/build/buildbotcustom/rev/b55b599f5f14
pm and pm02 reconfig'd at 2016 PDT.

Removing dependency on 3.6b1 tracking bug.
No longer depends on: 518496
Nick thanks for backing this out.

I will work on this on the next two days depending how my other stuff goes.
(In reply to comment #12)
Re-assigning back to Armen based on this comment.
Assignee: nthomas → armenzg
I have spent all morning on this when I thought it would be not that difficult to choose the right environment variables. I diffed the environment vairables from production logs of when it used to build, when it wouldn't build and with the environment variables of the slaves.

I barely know what I am doing so I decided to go the safe way.

The last 2 attached patches get the job done by creating a new environment variables section. This section is pretty much as the previous section but removing the same variables as in the backed out patch (https://bugzilla.mozilla.org/attachment.cgi?id=404824).

If this is good enough, let's do it; otherwise somebody take this bug since I want to focus on the multi-locale build for Fennec which has higher priority than fixing this.
Priority: -- → P2
Priority: P2 → P3
Blocks: 501794
Attachment #406460 - Flags: review?(nthomas)
Attachment #406462 - Flags: review?(nthomas)
I have done a successful run of the attached patches on staging.

Nick, can we have this fixed separately from the issue that WinCE and WinMo are using older SDKs?
Desktop fennec builds for win32 are broken right now, but WinCE builds are working. Desktop fennec builds for linux/mac are fine.

mfinkle is going to have a look at the conflicting env.setups required and see if he can help figure it out.
OS: Mac OS X → Windows Server 2003
(re the review request for attachments 406460 and 406462) 
I'm not at all fond of adding another hard-to-maintain environment to fix the Fennec desktop builds for Windows. It just makes it worse the next time we need to tweak the vars.

mfinkle, do you know if the desktop build must be compiled with VS2008 ? If it's a XUL app then perhaps we could (maybe even should) be using the VS2005 like the xulrunner nightly builds. If I try that with our standard build environment (VS2005 & the Windows 7 SDK) then configure says
 configure: warning: DirectDraw ddraw.h header not found or it's missing DDLOCK_WAITNOTBUSY, disabling DirectDraw surface.  If you have an older SDK (such as the CE5 SDK), try copying in ddraw.lib and ddraw.h from the WM6 SDK.
Is that relevant to a desktop build ? Thought that was WinMo/CE builds. There are other issues mentioned in comment #10 but I doubt they're not hard to fix.
Taking this again.

(In reply to comment #19)
> (re the review request for attachments 406460 and 406462) 
> I'm not at all fond of adding another hard-to-maintain environment to fix the
> Fennec desktop builds for Windows. It just makes it worse the next time we need
> to tweak the vars.
> 
I do agree but that was the only thing I had at hand for a quick fix if this bug was super important for any reasons. I will try to use other existent environment variables sections OR reduce the one I proposed to a really short number of lines.
Attachment #406460 - Flags: review?(nthomas)
Attachment #406462 - Flags: review?(nthomas)
After playing for a little while with the values of "win32-ref-platform" I ended up realizing that all I need was the PATH from "winmo-arm".
If I look at the resulting log I get these values:
> DEVENVDIR=d:\msvs8\Common7\IDE
> INCLUDE=d:\sdks\v7.0\\include;d:\sdks\v7.0\\include\atl;d:\msvs8\VC\ATLMFC\INCLUDE;d:\msvs8\VC\INCLUDE;d:\msvs8\VC\PlatformSDK\include;
> LIB=d:\sdks\v7.0\\lib;d:\msvs8\VC\ATLMFC\LIB;d:\msvs8\VC\LIB;d:\msvs8\VC\PlatformSDK\lib;d:\msvs8\SDK\v2.0\lib;;D:\mozilla-build\\atlthunk_compat
> LIBPATH=C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727;d:\msvs8\VC\ATLMFC\LIB
> SDKDIR=d:\sdks\v7.0\
> SDKVER=7
Are these good enough?


Aside of that, the file env.py could be quite simplified and make it easier to maintain if we use things like reusing values from the ref-platform environments by simply doing string substitutions. Maybe we could file a bug for improving this file.

MozillaEnvironments['win32-fennec'] = {
#     "DEVENVDIR":       MozillaEnvironments['win32-ref-platform']['DEVENVDIR'].replace('msvs8\VC','msvs9'),
#     "INCLUDE":         MozillaEnvironments['win32-ref-platform']['INCLUDE'].replace('v6.0','v7.0'),
#     "LIB":             MozillaEnvironments['win32-ref-platform']['LIB'].replace('v6.0','v7.0') + \
#                        ';d:\\msvs8\\SDK\\v2.0\\lib;D:\\mozilla-build\\atlthunk_compat',
#     "SDKDIR":          MozillaEnvironments['win32-ref-platform']['SDKDIR'].replace('v6.0','v7.0'),
#     "SDKVER":          '7',
     # Note that I am importing from winmo-arm instead of win32-ref-platform
     "PATH":            MozillaEnvironments['winmo-arm']['PATH'],
}
Attachment #406460 - Attachment is obsolete: true
These patches have worked on staging but I am running final clean builds for both branches to make sure that it all works as expected.
Attachment #408653 - Flags: review?(nthomas)
Attachment #408654 - Flags: review?(nthomas)
Green runs on a clean run.
Comment on attachment 408653 [details] [diff] [review]
[buildbotcustom] add new win32-fennec environment variables section 

r- 

>diff --git a/env.py b/env.py
>@@ -43,36 +43,38 @@ MozillaEnvironments['win32-ref-platform'
>+    "LIBPATH": 'C:\\WINDOWS\\Microsoft.NET\\Framework\\v2.0.50727;' + \
>+               'd:\\msvs9\\VC\\ATLMFC\\LIB;d:\\msvs9\\VC\\LIB;',

This was a stray hunk.

>+MozillaEnvironments['win32-fennec'] = {
>+    "PATH": MozillaEnvironments['winmo-arm']['PATH'],

This means we end up with a msvs9 style path, with msvs8 everything else (from MozBuild launching).
Attachment #408653 - Flags: review?(nthomas) → review-
Comment on attachment 406460 [details] [diff] [review]
[buildbotcustom] add new win32-fennec environment variables section

In the interests of making the pain stop, I'm going to un-obsolete this attachment and grant r+.
Attachment #406460 - Attachment is obsolete: false
Attachment #406460 - Flags: review+
Attachment #408654 - Flags: review?(nthomas) → review+
Attachment #404824 - Flags: checked-in+
Attachment #404824 - Attachment is obsolete: true
Attachment #408653 - Attachment is obsolete: true
Attachment #406460 - Flags: checked-in?
Attachment #408654 - Flags: checked-in?
Comment on attachment 408654 [details] [diff] [review]
[buildbot-configs] select "win32-fennec" environment variables for Win32 Fennec Desktop builds

http://hg.mozilla.org/build/buildbot-configs/rev/c0068547f206
Attachment #408654 - Flags: checked-in? → checked-in+
Comment on attachment 406460 [details] [diff] [review]
[buildbotcustom] add new win32-fennec environment variables section

http://hg.mozilla.org/build/buildbotcustom/rev/f49fd8b839ad
Attachment #406460 - Flags: checked-in? → checked-in+
Masters reconfigured and triggered some nightly builds.
Nightly builds for both branches
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mobile-trunk/fennec-1.0b5pre.en-US.win32.zip
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mobile-1.9.2/fennec-1.0b5pre.en-US.win32.zip

Resolving FIXED
Status: REOPENED → RESOLVED
Closed: 11 years ago11 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.