1759681 - `ERROR: Cannot find unzip` while mach configure

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Kagami Rosylight [:saschanaz] (they/them)]

> ./mach configure
 0:00.91 C:/Users/sasch/Documents/GitHub/gecko-dev/obj-x86_64-pc-mingw32\_virtualenvs\build\Scripts\python.exe C:/Users/sasch/Documents/GitHub/gecko-dev\configure.py
 0:01.05 Using Python 3.10.1 from C:\Users\sasch\Documents\GitHub\gecko-dev\obj-x86_64-pc-mingw32\_virtualenvs\build\Scripts\python.exe
 0:01.34 Adding configure options from C:\Users\sasch\Documents\GitHub\gecko-dev\mozconfig
 0:01.34   --disable-optimize
 0:01.34 checking for vcs source checkout... git
 0:01.41 checking for a shell... C:/mozilla-build/msys2/usr/bin/sh.exe
 0:01.41 checking for host system type... x86_64-pc-mingw32
 0:01.41 checking for target system type... x86_64-pc-mingw32
 0:01.80 checking whether cross compiling... no
 0:01.95 checking for Python 3... C:/Users/sasch/Documents/GitHub/gecko-dev/obj-x86_64-pc-mingw32/_virtualenvs/build/Scripts/python.exe (3.10.1)
 0:01.95 checking for git... C:/PROGRA~1/Git/cmd/git.exe
 0:01.97 checking for Git version... 2.34.0.windows.1
 0:02.00 checking for sparse checkout... no
 0:03.71 checking for wget... not found
 0:03.74 checking for ccache... not found
 0:03.74 checking for the target C compiler... C:/Users/sasch/.mozbuild/clang/bin/clang-cl.exe
 0:03.79 checking whether the target C compiler can be used... yes
 0:03.79 checking the target C compiler version... 13.0.1
 0:03.82 checking the target C compiler works... yes
 0:03.82 checking for the target C++ compiler... C:/Users/sasch/.mozbuild/clang/bin/clang-cl.exe
 0:03.87 checking whether the target C++ compiler can be used... yes
 0:03.87 checking the target C++ compiler version... 13.0.1
 0:03.90 checking the target C++ compiler works... yes
 0:03.90 checking for the host C compiler... C:/Users/sasch/.mozbuild/clang/bin/clang-cl.exe
 0:03.94 checking whether the host C compiler can be used... yes
 0:03.94 checking the host C compiler version... 13.0.1
 0:03.98 checking the host C compiler works... yes
 0:03.98 checking for the host C++ compiler... C:/Users/sasch/.mozbuild/clang/bin/clang-cl.exe
 0:04.02 checking whether the host C++ compiler can be used... yes
 0:04.02 checking the host C++ compiler version... 13.0.1
 0:04.04 checking the host C++ compiler works... yes
 0:04.07 checking for 64-bit OS... yes
 0:04.10 checking for Windows SDK... 0x0a00 in C:/PROGRA~2/WI3CF2~1/10/
 0:04.10 checking for Universal CRT SDK... 10.0.22000.0 in C:/PROGRA~2/WI3CF2~1/10/
 0:04.10 checking for linker... C:/Users/sasch/.mozbuild/clang/bin/lld-link.exe
 0:04.10 checking for host_linker... C:/Users/sasch/.mozbuild/clang/bin/lld-link.exe
 0:04.10 checking for the assembler... C:/PROGRA~2/MICROS~2/2022/BUILDT~1/VC/Tools/MSVC/1431~1.311/bin/HostX64/x64/ml64.exe
 0:04.10 checking for rc... C:/Users/sasch/.mozbuild/clang/bin/llvm-rc.exe
 0:04.10 checking for ar... C:/Users/sasch/.mozbuild/clang/bin/llvm-lib.exe
 0:04.13 checking for stdint.h... yes
 0:04.16 checking for inttypes.h... yes
 0:04.19 checking for malloc.h... yes
 0:04.23 checking for alloca.h... no
 0:04.24 checking for sys/byteorder.h... no
 0:04.27 checking for getopt.h... no
 0:04.30 checking for unistd.h... no
 0:04.32 checking for nl_types.h... no
 0:04.35 checking for cpuid.h... yes
 0:04.37 checking for fts.h... no
 0:04.40 checking for sys/statvfs.h... no
 0:04.41 checking for sys/statfs.h... no
 0:04.44 checking for sys/vfs.h... no
 0:04.48 checking for sys/mount.h... no
 0:04.49 checking for sys/quota.h... no
 0:04.52 checking for sys/queue.h... no
 0:04.55 checking for sys/types.h... yes
 0:04.57 checking for netinet/in.h... no
 0:04.60 checking for byteswap.h... no
 0:04.62 checking for memfd_create in sys/mman.h... no
 0:04.65 checking for perf_event_open system call... no
 0:04.66 checking for llvm_profdata... C:/Users/sasch/.mozbuild/clang/bin/llvm-profdata.exe
 0:04.66 checking for rustc... C:/Users/sasch/.cargo/bin/rustc.exe
 0:04.66 checking for cargo... C:/Users/sasch/.cargo/bin/cargo.exe
 0:04.79 Actually using 'C:\Users\sasch\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin\rustc.exe'
 0:04.91 Actually using 'C:\Users\sasch\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin\cargo.exe'
 0:04.91 checking rustc version... 1.57.0
 0:04.93 checking cargo version... 1.57.0
 0:05.01 checking for rust host triplet... x86_64-pc-windows-msvc
 0:05.07 checking for rust target triplet... x86_64-pc-windows-msvc
 0:05.07 checking for rustdoc... C:/Users/sasch/.cargo/bin/rustdoc.exe
 0:05.10 checking for cbindgen... C:/Users/sasch/.mozbuild/cbindgen/cbindgen.exe
 0:05.10 checking for rustfmt... C:/Users/sasch/.cargo/bin/rustfmt.exe
 0:05.12 checking for clang for bindgen... C:/Users/sasch/.mozbuild/clang/bin/clang.exe
 0:05.15 checking for libclang for bindgen... C:/Users/sasch/.mozbuild/clang/bin/libclang.dll
 0:05.15 checking that libclang is new enough... yes
 0:05.15 checking bindgen cflags... -x c++ -fno-sized-deallocation -fno-aligned-new -DTRACING=1 -DIMPL_LIBXUL -DMOZILLA_INTERNAL_API -DRUST_BINDGEN -DOS_WIN=1 -DWIN32=1 -D_CRT_USE_BUILTIN_OFFSETOF -DHAVE_VISIBILITY_HIDDEN_ATTRIBUTE=1 -std=gnu++17
 0:05.18 checking for tm_zone and tm_gmtoff in struct tm... no
 0:05.29 checking for _getc_nolock... yes
 0:05.38 checking for localeconv... yes
 0:05.40 checking for nodejs... C:\Users\sasch\.mozbuild\node\node.EXE (12.22.1)
 0:05.41 checking for tar... C:/Windows/system32/tar.exe
 0:05.41 checking for unzip... not found
 0:05.41 DEBUG: unzip: Looking for unzip
 0:05.41 ERROR: Cannot find unzip
Error running mach:

    ['configure']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file configure| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

Exception: Process executed with non-0 exit code 1: ['C:/Users/sasch/Documents/GitHub/gecko-dev/obj-x86_64-pc-mingw32\\_virtualenvs\\build\\Scripts\\python.exe', 'C:/Users/sasch/Documents/GitHub/gecko-dev\\configure.py']

  File "C:\Users\sasch\Documents\GitHub\gecko-dev\python\mozbuild\mozbuild\build_commands.py", line 254, in configure
    return driver.configure(
  File "c:\users\sasch\documents\github\gecko-dev\python\mozbuild\mozbuild\controller\building.py", line 1677, in configure
    status = self._run_command_in_objdir(
  File "c:\users\sasch\documents\github\gecko-dev\python\mozbuild\mozbuild\base.py", line 842, in _run_command_in_objdir
    return self.run_process(cwd=self.topobjdir, **args)
  File "C:\Users\sasch\Documents\GitHub\gecko-dev\python\mach\mach\mixin\process.py", line 183, in run_process
    raise Exception(

Sentry event ID: 01976105d4df46c3ae53453cd56ed036
Sentry is attempting to send 0 pending error messages
Waiting up to 2 seconds
Press Ctrl-Break to quit

Bisecting says bug 1759256 is the culprit. (I'm on mozillabuild 4.0pre0 installed in C:/mozilla-build and running this outside of mozillabuild)

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1759256

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

:mhentges, since you are the author of the regressor, bug 1759256, could you take a look?
For more information, please visit auto_nag documentation.

Comment 3

[Kagami Rosylight [:saschanaz] (they/them)]

Err, this is because the script wants to skip adding PATH when PATH has C:\mozilla-build\python310\ (or anything below mozillabuild) and that check was broken before bug 1759256.

It's a bit confusing, but I don't need that path anymore after the support for mach outside mozillabuild, so maybe not that important. I'll close this, but feel free to reopen if this looks important to anyone.

Comment 4

[Mitchell Hentges [:mhentges] 🦀]

It's a bit confusing, but I don't need that path anymore after the support for mach outside mozillabuild, so maybe not that important.

That's fair, the idea here was to avoid adding unnecessary MozillaBuild-related items to the $PATH if MozillaBuild itself has initialized them, or the developer is managing the MozillaBuild paths themselves.

Perhaps the messaging here could've been better, but it couldn't be solved with print statements ("not adding MozillaBuild to the $PATH because it already is!") as they would become spammy for users who always hit that code path.

Comment 5

[Kagami Rosylight [:saschanaz] (they/them)]

BTW, adding same things twice to PATH shouldn't affect anything IMO, what directed you to add such check? 👀

Comment 6

[Mitchell Hentges [:mhentges] 🦀]

BTW, adding same things twice to PATH shouldn't affect anything IMO, what directed you to add such check? 👀

Good question, three things:

• The user may want $MOZILLABUILD items to be in the path at a different priority level than how it's automatically added.
  • Though, I suppose that since the automatically-added paths are put at the very end, there would be no behavioural impact
• There's more than just the python\Scripts path: there's also $MOZILLABUILD\bin and $MOZILLABUILD\msys2\usr\bin. If the user wants some subset of these, that's possible with the conditional check? Although, I'm not sure why a subset would be wanted, considering it causes confusing errors like the one you're seeing
• Maybe slightly degraded performance, since every operation that may use the $PATH may look in the same directories multiple times

I'm definitely less convinced now that conditionally adding the directories was worthwhile :)

Comment 7

[Kagami Rosylight [:saschanaz] (they/them)]

Thanks! (although none of them looks convincing to me either 😝)

Comment 8

[Mitchell Hentges [:mhentges] 🦀]

Hmm, I think I'll leave this as-is for the moment, but if others run into the issue then it'll likely be time to unconditionally add the paths 👍

Comment 9

[James Teh [:Jamie]]

I just ran into this after latest mozilla-central required me to set MOZILLABUILD in order to build running mach outside of mozilla-build. (I've never needed this before now; I'm not sure why I need it now.) Anyway, after setting MOZILLABUILD, I ran into the error described here.

Comment 10

[Mitchell Hentges [:mhentges] 🦀]

Two questions for you Jamie:

1. What value did you set MOZILLABUILD to?
2. Can you share your $env:PATH/%PATH%?

Comment 11

[James Teh [:Jamie]]

(In reply to Mitchell Hentges [:mhentges] 🦀 from comment #10)

1. What value did you set MOZILLABUILD to?

c:/mozilla-build, which is where I have mozilla-build installed. (The forward slash is because I was too lazy to deal with proper escaping in the script where I'm setting it, not necessarily because of a bug.)

2. Can you share your $env:PATH/%PATH%?

Originally, my PATH included c:\mozilla-build\python so git-cinnabar could use python2. That's what was triggering the unzip problem. After I removed that, my PATH is now:

Path=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\Git\cmd;C:\Program Files (x86)\dotnet\;C:\Program Files\dotnet\;C:\Users\jamie\.cargo\bin;C:\Users\jamie\AppData\Local\Microsoft\WindowsApps;C:\Users\jamie\AppData\Roaming\Python\Python39\Scripts;c:\users\jamie\src\git-cinnabar;C:\program files\Python39\Scripts;

I don't recall exactly where c:\mozilla-build\python was positioned before I removed it, but it was somewhere near the end.

(For reference in case anyone wonders, I fixed the git-cinnabar problem a better way by setting GIT_CINNABAR_PYTHON.)

Comment 12

[alan]

I met the same issue in aarch64 system

When I ran './mach configure'
I got an error --
0:21.08 checking for tar... /usr/bin/tar
0:21.08 checking for unzip... not found
0:21.08 DEBUG: unzip: Looking for '-O GBK'
0:21.08 ERROR: Cannot find unzip

However, tar and unzip can both run in shell, they are in the directory '/usr/bin'.

And my path is '/home/jinbiao/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games' which contains '/usr/bin'.

Comment 13

[Mitchell Hentges [:mhentges] 🦀]

Thanks Jamie, that makes sense. I suppose that once you removed the c:\mozilla-build\python part from your PATH, then you no longer needed to set MOZILLABUILD anymore, right?

I think that this is enough spice to warrant removing the conditional adding of mozillabuild-y paths to the end of the PATH, I'll get a patch up.

---

Alan, can you create a new bug? I think you're seeing a different issue, because this one is Windows-specific, and it looks like you're on Linux.

Comment 14

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1759681: Conditionally add each MozillaBuild path

When using Mach outside of MozillaBuild, rather than avoiding adding
paths entirely if any part of MozillaBuild is already in the PATH,
instead conditionally add each one if it doesn't exist already.

This ensures no duplication of paths, while also avoiding the
not-uncommon case of developers manually adding MozillaBuild's Python to
their Windows PATH.

Comment 15

[James Teh [:Jamie]]

(In reply to Mitchell Hentges [:mhentges] 🦀 from comment #13)

I suppose that once you removed the c:\mozilla-build\python part from your PATH, then you no longer needed to set MOZILLABUILD anymore, right?

Oh! I didn't realise these two things were connected. Yes, I removed MOZILLABUILD and things still work as expected.

Comment 16

[alan]

(In reply to Mitchell Hentges [:mhentges] 🦀 from comment #13)

Alan, can you create a new bug? I think you're seeing a different issue, because this one is Windows-specific, and it looks like you're on Linux.

OK, Mitchell , I have created a new bug which has more details
https://bugzilla.mozilla.org/show_bug.cgi?id=1760246

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/20b1cdafc68b
Conditionally add each MozillaBuild path r=ahochheiden

Comment 18

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/20b1cdafc68b