1432892 - (Debian GNU/Linux) |mach boostrap| fails since the package |npm| is missing.

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[ISHIKAWA, Chiaki]

I am using Debian GNU/Linux.
After upgrading OS packages in mid-December, I cannot run |mach bootstrap|
since |npm| package which is installed by |mach| is no longer in the standard package (of stretch).

The failure log.

Sorry, I tried to change the LOCALE to C locale, but somehow, mach still spews out the error message in Japanese.
The crux of the error is
|E: パッケージ npm が見つかりません|,
which translates "package npm is not found".
Actually, it is |E: Unable to locate package npm| in official translation.

--- begin log from |mach bootstrap|
ishikawa@ip030: /NREF-COMM-CENTRAL/comm-centralishikawa@ip030:/NREF-COMM-CENTRAL/comm-central$ mozilla/mach bootstrap
kind=None       <- this and the following two lines are from
environ.get('TERM', 'unknown')=xterm-256color <--- the dump from a patch
environ.get('TERM', 'linux')=xterm-256color   <--- in Bug 1432867

Note on Artifact Mode:

Firefox for Desktop and Android supports a fast build mode called
artifact mode. Artifact mode downloads pre-built C++ components rather
than building them locally, trading bandwidth for time.

Artifact builds will be useful to many developers who are not working
with compiled code. If you want to work on look-and-feel of Firefox,
you want "Firefox for Desktop Artifact Mode".

Similarly, if you want to work on the look-and-feel of Firefox for Android,
you want "Firefox for Android Artifact Mode".

To work on the Gecko technology platform, you would need to opt to full,
non-artifact mode. Gecko is Mozilla's web rendering engine, similar to Edge,
Blink, and WebKit. Gecko is implemented in C++ and JavaScript. If you
want to work on web rendering, you want "Firefox for Desktop", or
"Firefox for Android".

If you don't know what you want, start with just Artifact Mode of the desired
platform. Your builds will be much shorter than if you build Gecko as well.
But don't worry! You can always switch configurations later.

You can learn more about Artifact mode builds at
https://developer.mozilla.org/en-US/docs/Artifact_builds.

Please choose the version of Firefox you want to build:
1. Firefox for Desktop Artifact Mode
2. Firefox for Desktop
3. Firefox for Android Artifact Mode
4. Firefox for Android
Your choice: 2
Executing as root: sudo apt-get install autoconf2.13 build-essential ccache nodejs python-dev python-pip python-setuptools unzip uuid zip npm
[sudo] ishikawa のパスワード:
パッケージリストを読み込んでいます... 0%パッケージリストを読み込んでいます... 100%パッケージリストを読み込んでいます... 完了
依存関係ツリーを作成しています... 100%
状態情報を読み取っています... 100%
 完了
E: パッケージ npm が見つかりません
Error running mach:

    ['bootstrap']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

CalledProcessError: Command '[u'sudo', u'apt-get', u'install', 'autoconf2.13', 'build-essential', 'ccache', 'nodejs', 'python-dev', 'python-pip', 'python-setuptools', 'unzip', 'uuid', 'zip', 'npm']' returned non-zero exit status 100

  File "/NREF-COMM-CENTRAL/comm-central/mozilla/python/mozboot/mozboot/mach_commands.py", line 32, in bootstrap
    bootstrapper.bootstrap()
  File "/NREF-COMM-CENTRAL/comm-central/mozilla/python/mozboot/mozboot/bootstrap.py", line 294, in bootstrap
    self.instance.install_system_packages()
  File "/NREF-COMM-CENTRAL/comm-central/mozilla/python/mozboot/mozboot/debian.py", line 103, in install_system_packages
    self.apt_install(*self.packages)
  File "/NREF-COMM-CENTRAL/comm-central/mozilla/python/mozboot/mozboot/base.py", line 352, in apt_install
    self.run_as_root(command)
  File "/NREF-COMM-CENTRAL/comm-central/mozilla/python/mozboot/mozboot/base.py", line 308, in run_as_root
    subprocess.check_call(command, stdin=sys.stdin)
  File "/usr/lib/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
^[]0;ishikawa@ip030: /NREF-COMM-CENTRAL/comm-centralishikawa@ip030:/NREF-COMM-CENTRAL/comm-central$ exit
exit

Script done on 2018-01-25 04:31:09+0900
--- end log from |mach bootstrap|


I have done |apt-get install npm| as a command and you see the English message when I set LANG to C.
--- begin log

root@ip030:/NREF-COMM-CENTRAL/comm-central# apt-get install npm
パッケージリストを読み込んでいます... 完了
依存関係ツリーを作成しています       
状態情報を読み取っています... 完了
E: パッケージ npm が見つかりません
root@ip030:/NREF-COMM-CENTRAL/comm-central# export LANG=C
root@ip030:/NREF-COMM-CENTRAL/comm-central# apt-get install npm
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package npm
root@ip030:/NREF-COMM-CENTRAL/comm-central# 
--- end log

To their credit, NPM developers have prepared a web page to cope with this:
https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions
(Oops, according to the web page, Ubunto may be also affected by this. If so, the future ubuntu users need to be warned. I am afraid that there are many more ubuntu users than Debian users among patch developers.)
HOWEVER, as an ordinary user, I have a bit of hesitation when I see |bash| is run by feeding it an output of a command that fetches something from a remote site, and under super privilege at that ...

Also, this was already noticed and reported to Debian, too. 
From a post by [e-mail] in dev-builds mailing list,
https://www.mail-archive.com/[e-mail]/msg01194.html
His post is found in the thread at the bottom.

--- begin quote ---

Could you please report a bug?

Mark added npm as a dependency on Debian [1], as I run Debian unstable, I didn't see it wasn't available for stretch or testing.

Sorry about that.

I also reported a bug on Debian side [2] so that it is clear that npm needs love in Debian.

Cheers,
Sylvestre


[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1424921

[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=888272
--- end quote ---

TIA

Comment 1

[Sylvestre Ledru [:Sylvestre]]

Attachment: Bug 1432892 - Comment the npm package dependency until Debian brings it back

Review commit: https://reviewboard.mozilla.org/r/215860/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/215860/

Comment 2

[Mark Banner (:standard8)]

Comment on attachment 8945755 [details]
Bug 1432892 - Comment the npm package dependency until Debian brings it back

https://reviewboard.mozilla.org/r/215860/#review221942

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fe16ec1d4e72
Comment the npm package dependency until Debian brings it back r=standard8

Comment 4

[Eliza Balazs [:ebalazs_]]

https://hg.mozilla.org/mozilla-central/rev/fe16ec1d4e72

Comment 5

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8945755 [details]
Bug 1432892 - Comment the npm package dependency until Debian brings it back

Approval Request Comment
[Feature/Bug causing the regression]: Bug 1424921
[User impact if declined]:
mach bootstrap won't work on Debian
https://hg.mozilla.org/releases/mozilla-beta/file/tip/python/mozboot/mozboot/debian.py
[Is this code covered by automated tests?]: No
[Has the fix been verified in Nightly?]: Yes
[Needs manual test from QE? If yes, steps to reproduce]: No
[List of other uplifts needed for the feature/fix]: None
[Is the change risky?]: No
[Why is the change risky/not risky?]: Just a build step
[String changes made/needed]: None

Comment 6

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8945755 [details]
Bug 1432892 - Comment the npm package dependency until Debian brings it back

Looks useful for developers or anyone building Firefox on Debian stable versions - let's uplift for the 59 beta 6 build.

Comment 7

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/releases/mozilla-beta/rev/1f833bb0e4c9

Comment 9

[Ricky Stewart]

Hey Sylvestre, I just stumbled upon this code in debian.py, which has been commented out for 2 years. I just checked and Debian does indeed have a package called npm. Can we uncomment this? Does it need to be conditional on a certain version of Debian or something?

Comment 10

[Sylvestre Ledru [:Sylvestre]]

Yeah, seems we can do it if the version is right, thanks :)