1323381 - Building Firefox requires 8dot3 filenames ("GetShortPathName returned a long path name. Are 8dot3 filenames disabled?")

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P2)

Description

[harshitjain23897]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36

Steps to reproduce:

cd c:/
cd mozilla-source
cd mozilla-central
mach build


Actual results:

$ ./mach build
 0:02.36 c:\mozilla-build\mozmake\mozmake.EXE -f client.mk -s configure
 0:08.99 cd c:/mozilla-source/mozilla-central/obj-i686-pc-mingw32
 0:09.04 c:/mozilla-source/mozilla-central/configure
 0:09.82 Reexecuting in the virtualenv
 0:11.08 checking for a shell... C:/mozilla-build/msys/bin/sh.exe
 0:12.02 checking for host system type... i686-pc-mingw32
 0:12.02 checking for target system type... i686-pc-mingw32
 0:12.02 checking whether cross compiling... no
 0:12.06 checking for the target C compiler...
 0:12.06 DEBUG: _cc: Trying cl
 0:12.06 ERROR: GetShortPathName returned a long path name. Are 8dot3 filenames
disabled?
 0:12.12 *** Fix above errors and then restart with\
 0:12.12                "c:/mozilla-build/mozmake/mozmake.EXE -f client.mk build
"
 0:12.13 client.mk:375: recipe for target 'configure' failed
 0:12.13 mozmake.EXE: *** [configure] Error 1
2


Expected results:

The build should have been completed as per https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Windows_Prerequisites. I am a newbie please help.

Comment 1

[YF (Yang)]

See https://technet.microsoft.com/en-us/library/ff621566(v=ws.11).aspx or https://support.microsoft.com/kb/121007, try it.

Comment 2

[harshitjain23897]

I did that still the same error exist.
C:\mozilla-source>fsutil 8dot3name query
The registry state of NtfsDisable8dot3NameCreation is 1 (Disable 8dot3 name crea
tion on all volumes).

Comment 3

[YF (Yang)]

(In reply to harshitjain23897 from comment #2)
> I did that still the same error exist.
> C:\mozilla-source>fsutil 8dot3name query
> The registry state of NtfsDisable8dot3NameCreation is 1 (Disable 8dot3 name
> crea
> tion on all volumes).

Please enable this feature, via like "fsutil 8dot3name set 0", and then retry build.

Comment 4

[harshitjain23897]

(In reply to YF (Yang) from comment #3)
> (In reply to harshitjain23897 from comment #2)
> > I did that still the same error exist.
> > C:\mozilla-source>fsutil 8dot3name query
> > The registry state of NtfsDisable8dot3NameCreation is 1 (Disable 8dot3 name
> > crea
> > tion on all volumes).
> 
> Please enable this feature, via like "fsutil 8dot3name set 0", and then
> retry build.

I did that too still the error persists.

Comment 5

[YF (Yang)]

Did the results of "fsutil 8dot3name query" change?

Comment 6

[harshitjain23897]

Yes, The result is:

C:\mozilla-source>fsutil 8dot3name query
The registry state of NtfsDisable8dot3NameCreation is 0 (Enable 8dot3 name creat
ion on all volumes).

Comment 7

[Shu-yu Guo [:shu]]

I ran into this problem last night. The problem is that the 8dot3name bit on a volume applies at *file creation time*. Turning 8dot3name on after the fact does nothing to existing files: those files must be recreated.

harshitjan23897, open up cmd and type |dir /X| on your volume to see if the files have a short name. If not, you need to enable it, and reinstall your toolchain like Visual Studio. That's really annoying, though. Does someone know if there's a way to force creation of 8dot3names for existing files?

Comment 8

[JChinoz]

Same problem. Is there a reported fix for this?

Comment 9

[Shubham Bhadani]

Hello guys ,

I am facing this issue . I immediately need to resolve it . 
 ERROR: GetShortPathName returned a long path name. Are 8dot3 filenames disabled?
Please help

Comment 10

[greg]

Why is 8dot3name Creation needed for this?
It can speed up File Lookup and Creation a lot:

https://blogs.technet.microsoft.com/josebda/2012/11/13/windows-server-2012-file-server-tip-disable-8-3-naming-and-strip-those-short-names-too/

Comment 11

[Masatoshi Kimura [:emk]]

Currently Mozilla toolchain depends on short names.

As a workaround, you can add short names after the fact with following commands:
 fsutil file setshortname "C:\Program Files (x86)" PROGRA~1
 fsutil file setshortname "C:\Program Files (x86)\Microsoft Visual Studio 14.0" MICROS~1

You will have to run a Command Prompt as an administrator. Also, you will have to terminate all programs under "C:\Program Files (x86)" before running the commands. Otherwise the first command will fail with an access denied error even if you give administrator privileges.

You don't have to enable 8dot3name creation. It has no effect after the fact.

Comment 12

[Masatoshi Kimura [:emk]]

I think we should improve the error message because enabling 8dot3name creation does not solve the issue.

Comment 13

[Masatoshi Kimura [:emk]]

> fsutil file setshortname "C:\Program Files (x86)" PROGRA~1
 fsutil file setshortname "C:\Program Files (x86)" PROGRA~2
may be better.

Comment 14

[Jupeto]

some problem on esr52

$ fsutil 8dot3name query
Состояние реестра NtfsDisable8dot3NameCreation: 0 (создание имен 8dot3 включено для всех томов).

$ cd /
$ DIR P* /X
17.04.17  17:33    <DIR>          PROGRA~1     Program Files
17.04.17  17:42    <DIR>          PROGRA~2     Program Files (x86)

Comment 15

[Kelsey Gilbert [:jgilbert]]

This is insane. It's 2017.

:glandium, can you route this?

Comment 16

[Gregory Szorc [:gps]]

We can't easily support paths with spaces in them because of the way variables with spaces and command strings are propagated through the [make-based] build system. See also http://savannah.gnu.org/bugs/?712. This is why we disallow building if the source or object directory has a space in it. This is why we use 8dot3 filenames to reference paths in Program Files.

I don't think we can realistically not require 8dot3 filenames until we support building with !make.

Comment 17

[Kelsey Gilbert [:jgilbert]]

I got my home machine building without this requirement by judicious use of quoting of $LINK and friends.

Comment 18

[Mike Hommey [:glandium]]

Good for you, but IIRC, that can't be easily generalized.

Comment 19

[Kelsey Gilbert [:jgilbert]]

Attachment: 0001-Bug-1323381-Remove-8dot3-requirement-for-windows-bui.patch

Comment 20

[Gregory Szorc [:gps]]

Comment on attachment 8885599 [details] [diff] [review]
0001-Bug-1323381-Remove-8dot3-requirement-for-windows-bui.patch

Review of attachment 8885599 [details] [diff] [review]:
-----------------------------------------------------------------

This may very well appear to work. But as long as we use make for invoking paths with spaces, we can't do this. The main reason is that if a variable representing a path is used as a target or dependency/prerequisite, then make interprets it as multiple paths, fails to find the file, and marks the dependency/target as missing. This completely undermines incremental builds.

The workaround to this is to make files available in paths without spaces. For VC++ binaries, you can copy what the in-tree mozconfigs do. e.g. https://hg.mozilla.org/mozilla-central/file/03bcd6d65af6/build/win32/mozconfig.vs2015-win64. Essentially, ensure various environment variables like PATH contain VC++ and Windows SDK files in paths without spaces. You can symlink your existing install and then point the build system at that symlink.

To make things easier, we could potentially have a configure flag to point things at an alternate "root" directory for the VC++/SDK installs so users only need to define 1 or 2 alternate prefixes instead of mucking around with e.g. PATH. We could also teach `mach bootstrap` to create a symlink if 8dot3 filenames aren't available.

Comment 21

[Gregory Szorc [:gps]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ff088dde2885c3a665833dc67243ccc9a6bb007b is my (crude) attempt to offer a simple-ish workaround. It needs a bit of polish before review. The redundant target arch declaration is notably ugly.

(I fully expect that Try push to fail because I didn't test things locally.)

I decided to implement this as a mozconfig shell script because other (better) solutions are considerably more effort. Considering we accomplish the immediate goal of this bug (enable building when 8dot3 filenames aren't present for Visual Studio files) with what is essentially an in-tree mozconfig refactor + docs updates, I consider that an efficient solution!

Comment 22

[Jorg K (CEST = GMT+2)]

(In reply to Masatoshi Kimura [:emk] from comment #13)
> fsutil file setshortname "C:\Program Files (x86)" PROGRA~2
> may be better.
This was very hard to do since even in safe mode with console I got "access denied". In the end I managed to do this by starting my Windows 7 startup repair disk and chosing a command prompt there.

In the end, this didn't help. I have VS2015 installed in C:\vs2015 so I also did:
fsutil file setshortname "C:\vs2015" VS2015~1
although vs2015 wasn't too long to start with. It also didn't help.

The error message
0:06.60 DEBUG: fxc: Trying fxc.exe
0:06.60 ERROR: GetShortPathName returned a long path name. Are 8dot3 filenames disabled?
suggests that it's looking for fxc.exe which is in
C:\Program Files (x86)\Windows Kits\10\bin\x86\fxc.exe or
C:\Program Files (x86)\Windows Kits\10\bin\x64\fxc.exe

So in the end I did
fsutil file setshortname "Windows Kits" WINDOW~2
in the program files directory and that seemed to have worked for now.
Note that "Windows Defender" came before the "Windows Kits", hence the 2.

> You don't have to enable 8dot3name creation. It has no effect after the fact.
While it has no effect, you need to enable it or else the "fsutil file setshortname" command won't work.

Comment 23

[Masatoshi Kimura [:emk]]

(In reply to Jorg K (GMT+2) from comment #22)
> (In reply to Masatoshi Kimura [:emk] from comment #13)
> > fsutil file setshortname "C:\Program Files (x86)" PROGRA~2
> > may be better.
> This was very hard to do since even in safe mode with console I got "access
> denied". In the end I managed to do this by starting my Windows 7 startup
> repair disk and chosing a command prompt there.

Actually I installed VS2015 in D:. It would be a reason renaming was easier for me.

> In the end, this didn't help. I have VS2015 installed in C:\vs2015 so I also
> did:
> fsutil file setshortname "C:\vs2015" VS2015~1
> although vs2015 wasn't too long to start with. It also didn't help.
> 
> The error message
> 0:06.60 DEBUG: fxc: Trying fxc.exe
> 0:06.60 ERROR: GetShortPathName returned a long path name. Are 8dot3
> filenames disabled?
> suggests that it's looking for fxc.exe which is in
> C:\Program Files (x86)\Windows Kits\10\bin\x86\fxc.exe or
> C:\Program Files (x86)\Windows Kits\10\bin\x64\fxc.exe
> 
> So in the end I did
> fsutil file setshortname "Windows Kits" WINDOW~2
> in the program files directory and that seemed to have worked for now.
> Note that "Windows Defender" came before the "Windows Kits", hence the 2.

You are right. Sorry for forgetting a follow-up post about "Windows Kits".

> > You don't have to enable 8dot3name creation. It has no effect after the fact.
> While it has no effect, you need to enable it or else the "fsutil file
> setshortname" command won't work.

It worked for me without enabling 8dot3name creation on D:. Maybe a difference between Win7 and Win10? Or that's because I didn't disable 8dot3name creation "globally"?

Comment 24

[Paul Theriault [:pauljt] (no longer reading bugmail)]

The changset in comment 20 is probably a better workaround/solution, but if you are setting up a new build like I was, re-enabling shortnames (using fsutil) and, then reinstalling mozilla build tools, re-downloading source, and re-installing VS2017 into a directory that does NOT contain spaces fixed this issue for me.

Comment 25

[Ivan Pozdeev]

Attachment: 1323381-Make-8dot3-error-message-useful-for-fixing.patch

If the error message includes the faulty path and either says what's wrong with it that can be googled or how to fix it, this is going to address the problem for a user that hits it.

Comment 26

[Gregory Szorc [:gps]]

Comment on attachment 8993954 [details] [diff] [review]
1323381-Make-8dot3-error-message-useful-for-fixing.patch

Review of attachment 8993954 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks for the tweaked message! I'll land this for you: there's nothing more you should need to do.

Thank you for contributing your first patch!

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0ef51b53d1c1
Make 8dot3 error message useful for fixing; r=gps

Comment 28

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/0ef51b53d1c1