Windows SDK directory not detected properly on 64-bit python

RESOLVED FIXED in Firefox 55

Status

defect
RESOLVED FIXED
2 years ago
Last year

People

(Reporter: jgilbert, Assigned: bobowen)

Tracking

unspecified
mozilla56
Dependency tree / graph

Firefox Tracking Flags

(firefox55 fixed, firefox56 fixed)

Details

Attachments

(1 attachment, 1 obsolete attachment)

Reporter

Description

2 years ago
The SDKs we need are (on 64-bit systems) are in 'Program Files (x86)', not 'Program Files'. When we're using 32-bit python, we get the 32-bit registry view, with different keys and values from the 64-bit one. Using 64-bit python, we see only the Win10 SDK in 'Program Files', but this doesn't contain the includes and libraries.
Reporter

Updated

2 years ago
See Also: → 1344643
Comment hidden (mozreview-request)

Comment 3

2 years ago
mozreview-review
Comment on attachment 8866872 [details]
Bug 1364137 - Find the right registry keys for python64. -

https://reviewboard.mozilla.org/r/138460/#review143934

::: build/moz.configure/windows.configure:40
(Diff revision 1)
>      if host.kernel != 'WINNT':
>          return ()
>  
> -    return tuple(x[1] for x in get_registry_values(
> -        r'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Kits\Installed Roots'
> -        r'\KitsRoot*'))
> +    software_key = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, 'SOFTWARE')
> +    try:
> +        software_key = winreg.OpenKey(software_key, 'WOW6432Node')

Interesting, that's something I had done to find Visual Studio, but not the SDK.

However, I'd rather keep the get_registry_values function, even if it's only used here. Because if we end up needing it more in the future (we only recently removed other uses), I don't want ad-hoc code using winreg being written.

The status of HKLM\SOFTWARE vs HKLM\SOFTWARE\Wow6432Node entries is unclear. I remember that for Visual Studio, I had the opposite problem you have, which is that the Wow6432Node entries didn't have everything.

So it seems we should query both in all cases, and aggregate from there, picking the right things that exist.

::: build/moz.configure/windows.configure:45
(Diff revision 1)
> -        r'\KitsRoot*'))
> +        software_key = winreg.OpenKey(software_key, 'WOW6432Node')
> +    except WindowsError:
> +        # 32-bit python
> +        pass
> +
> +    sdks_key = winreg.OpenKey(software_key, 'Microsoft\Microsoft SDKs\Windows')

I have both SDK 8.1 and 10.0 installed, and only 8.1 is represented under MS\MS SDKs\Windows. They are however both under MS\Windows Kits\Installed Roots\KitsRoot* (in both Wow6432Node and normal).
Attachment #8866872 - Flags: review?(mh+mozilla) → review-
Reporter

Comment 4

2 years ago
mozreview-review-reply
Comment on attachment 8866872 [details]
Bug 1364137 - Find the right registry keys for python64. -

https://reviewboard.mozilla.org/r/138460/#review143934

> Interesting, that's something I had done to find Visual Studio, but not the SDK.
> 
> However, I'd rather keep the get_registry_values function, even if it's only used here. Because if we end up needing it more in the future (we only recently removed other uses), I don't want ad-hoc code using winreg being written.
> 
> The status of HKLM\SOFTWARE vs HKLM\SOFTWARE\Wow6432Node entries is unclear. I remember that for Visual Studio, I had the opposite problem you have, which is that the Wow6432Node entries didn't have everything.
> 
> So it seems we should query both in all cases, and aggregate from there, picking the right things that exist.

I really don't like the get_registry_values function because it's hard to figure out how to use, in comparison to the ample documentation for winreg. It was easier for me to understand how to use winreg than to understand what I could or couldn't expect from get_registry_values.

If a node is not in Wow6432Node, it won't show up for 32-bit programs, including our current python32 system. I would prefer not to cargo-cult and check-all-the-things.

> I have both SDK 8.1 and 10.0 installed, and only 8.1 is represented under MS\MS SDKs\Windows. They are however both under MS\Windows Kits\Installed Roots\KitsRoot* (in both Wow6432Node and normal).

Is it a usable 10, and is it installed properly? If you re-install it, does it still not show in \"Microsoft SDKs"\Windows?
Specifically, does your 10 SDK have the actual include/libraries paths? I had at least one 10 SDK that only had useless (to me) items, no includes or library files.
(In reply to Jeff Gilbert [:jgilbert] from comment #4)
> Comment on attachment 8866872 [details]
> Bug 1364137 - Find the right registry keys for python64. -
> 
> https://reviewboard.mozilla.org/r/138460/#review143934
> 
> > Interesting, that's something I had done to find Visual Studio, but not the SDK.
> > 
> > However, I'd rather keep the get_registry_values function, even if it's only used here. Because if we end up needing it more in the future (we only recently removed other uses), I don't want ad-hoc code using winreg being written.
> > 
> > The status of HKLM\SOFTWARE vs HKLM\SOFTWARE\Wow6432Node entries is unclear. I remember that for Visual Studio, I had the opposite problem you have, which is that the Wow6432Node entries didn't have everything.
> > 
> > So it seems we should query both in all cases, and aggregate from there, picking the right things that exist.
> 
> I really don't like the get_registry_values function because it's hard to
> figure out how to use, in comparison to the ample documentation for winreg.
> It was easier for me to understand how to use winreg than to understand what
> I could or couldn't expect from get_registry_values.

If something is not clear in the get_registry_values documentation, let's fix that. Your use case is pretty straightforward:
get_registry_values(r'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v*\InstallationFolder') will return something like:
 (
    ('vx.y', 'path'),
    ...
 )

> If a node is not in Wow6432Node, it won't show up for 32-bit programs,
> including our current python32 system. I would prefer not to cargo-cult and
> check-all-the-things.

If a node is not in HKLM\SOFTWARE\Wow6432Node in the 64-bits view, it won't show up under HKLM\SOFTWARE, true. But the point is that HKLM\SOFTWARE in the 64-bits view may contain values, and HKLM\SOFTWARE\Wow6432Node others, and the ones that work for the build can be in either place.

> > I have both SDK 8.1 and 10.0 installed, and only 8.1 is represented under MS\MS SDKs\Windows. They are however both under MS\Windows Kits\Installed Roots\KitsRoot* (in both Wow6432Node and normal).
> 
> Is it a usable 10, and is it installed properly? If you re-install it, does
> it still not show in \"Microsoft SDKs"\Windows?
> Specifically, does your 10 SDK have the actual include/libraries paths? I
> had at least one 10 SDK that only had useless (to me) items, no includes or
> library files.

It's the SDK that came installed alongside MSVC iirc, and it has the includes and library files. In fact, that's the SDK the build system picks to build Firefox in a start-shell.bat shell on that machine.
So, looking more closely, the SDK is *not* complete. It only contains the UCRT, which is required to build. And the build system needs to find both 8.1 and 10 SDK paths in that case, which using the Microsoft SDKs keys would break.
Blocks: 1380609
Assignee

Comment 7

2 years ago
I couldn't quite reproduce this issue, I was using a fresh Win7 VM with VS2017 and a version of MozillaBuild 3.0 (from Bug 1346656).

While the Win8.1 SDK in the 64-bit registry view pointed to |Program Files| the Win10 SDK pointed to |Program Files (x86)|, so things worked.

However it seems that different people's installations can be fairly inconsistent with these different registry views, so here's a try run with a patch to get both lots, using windows own access mask functionality:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b65d444f995a5c4e2b52ec4d973c38ce89206e3d
Assignee: jgilbert → bobowencode
Status: NEW → ASSIGNED
Assignee

Comment 8

2 years ago
The values that we need to find in the registry can be inconsistent across
different installations, so we retrieve values from both views in our search
for a valid SDK. This also ensures this works for 32-bit and 64-bit python.
Attachment #8887564 - Flags: review?(mh+mozilla)
Comment on attachment 8887564 [details] [diff] [review]
Get both 32-bit and 64-bit registry values when searching for the Windows SDK

Review of attachment 8887564 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/moz.configure/util.configure
@@ +240,5 @@
>  #   yields e.g.:
>  #     (r'C:\Program Files (x86)\Windows Kits\8.1\',)
>  #
>  #   get_registry_values(r'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\'
> +#                       r'Windows Kits\Installed Roots\KitsRoot8.1', True)

It would be preferable to pass get_32_and_64_bit=True because True alone doesn't tell much.

::: build/moz.configure/windows.configure
@@ +37,2 @@
>          r'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Kits\Installed Roots'
> +        r'\KitsRoot*', True))

same here.
Attachment #8887564 - Flags: review?(mh+mozilla) → review+

Comment 10

2 years ago
Pushed by bobowencode@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2626221a97cc
Get both 32-bit and 64-bit registry values when searching for the Windows SDK. r=glandium
https://hg.mozilla.org/mozilla-central/rev/2626221a97cc
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla56
Whiteboard: [checkin-needed-beta]
Attachment #8866872 - Attachment is obsolete: true

Updated

Last year
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.