931875 - Install npm (via a newer version of node) on test machines

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: Puppet, task)

Description

[Gareth Aye [:gaye] (back from PTO)]

My test is failing since node && npm don't exist on the build machines. Can we install them? We can do

```
git clone [e-mail]:joyent/node.git
cd node
git checkout v0.11.7-release
./configure
make
make install
```

Or something similar via chef/puppet/cap/whatever.

Comment 1

[Gareth Aye [:gaye] (back from PTO)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=29778781&tree=Cedar

Comment 2

[Gareth Aye [:gaye] (back from PTO)]

dustin, wlach, and I have already deployed a script that mirrors all of the npm packages that we need to http://npm-mirror.pub.build.mozilla.org/ so that we can grab them from the build machines without leaving our datacenter(s).

Comment 3

[Jonathan Griffin (:jgriffin)]

:catlee, can you add 'npm' to our Ubuntu32/64 VM test slaves?

Comment 4

[James Lal [:lightsofapollo]]

npm is bundled with node.... If you install the latest stable node (0.10.x) (available from any linux package manager) you will get both node and npm.

Comment 5

[Gareth Aye [:gaye] (back from PTO)]

I know jlal :)... Are you wanting me to s/&&/+/ in the bug title?

Comment 6

[Gareth Aye [:gaye] (back from PTO)]

That should be less ambiguous...

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

Node is already installed on Ubuntu testers:
  http://hg.mozilla.org/build/puppet/file/5f98f74e6e8e/modules/talos/manifests/init.pp#l18

However, that's 0.8.18:

[[e-mail] ~]# nodejs --version
v0.8.18

So this gets a bit more complicated - is an upgrade going to affect whatever is already using node?  That was originally added for bug 831771, so /cc rail.

Comment 8

[Gareth Aye [:gaye] (back from PTO)]

If node is on the ubuntu test machines, then either I am confused or tbpl is not telling the truth... https://tbpl.mozilla.org/php/getParsedLog.php?id=29778781&tree=Cedar

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

You're confused :)  Node is installed.  NPM's not.  See comment 7.

Comment 10

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Found in triage, moving to correct component.

Comment 11

[Gareth Aye [:gaye] (back from PTO)]

Haha okay. Thanks everyone!

Comment 12

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Nick/Patrick/Josh: iirc you originally asked us to install node on our test machines. Any concerns if we upgrade?

Comment 13

[Patrick McManus [:mcmanus]]

nick can take this

Comment 14

[Rail Aliiev [:rail]]

"apt-get install npm" won't work on the current system because nodejs installed in bug 831771 "breaks" a lot of old stuff, including npm (1.1.4~dfsg-1) and some of the npm's dependedncies:

$ apt-cache show nodejs=0.8.18~dfsg1-1mozilla1 | grep '^\(Breaks\|Version\):'
Version: 0.8.18~dfsg1-1mozilla1
Breaks: buddycloud-server (<< 0.3.1-2), carto (<< 0.4.6+ds1-2~), coffeescript (<< 1.3.3-3~), node-bones (<< 2.0.1+ds1-3~), node-cli (<< 0.4.4~20120516-1~), node-express (<< 2.5.9-2~), node-get (<< 1.1.5+ds1-2~), node-jake (<< 0.2.32-2~), node-less (<< 1.3.0~20120310-3~), node-mapnik (<< 0.6.7-3~), node-mbtiles (<< 0.2.3-2~), node-optimist (<< 0.3.1-2~), node-semver (<< 1.0.13-2~), node-tilelive (<< 4.2.1-4~), node-uglify (<< 1.2.6-2~), node-vows (<< 0.6.3-2~), npm (<< 1.1.4~dfsg-2~)

Installing npm as a package may require backporting a newer npm and some of its dependencies (see http://packages.ubuntu.com/raring/npm)

Comment 15

[Jonathan Griffin (:jgriffin)]

Another solution, if two versions of node/npm can live on a machine side-by-side, would be to install node 0.10 at run-time via tooltool and the mozharness script.  Gareth, do you know if that's possible?

Comment 16

[u408661]

(In reply to John O'Duinn [:joduinn] from comment #12)
> Nick/Patrick/Josh: iirc you originally asked us to install node on our test
> machines. Any concerns if we upgrade?

No concerns here, this upgrade should be just fine.

Comment 17

[Dustin J. Mitchell [:dustin] (he/him)]

From Jonathan's comment, it sounds like we've rejected the option of just upgrading Node to 0.10.21 or something similarly recent, but I don't see that rejection in any comment.  That'd certainly be the easiest solution here, and the most forward-compatible (in another six months, 0.8.x is going to seem even more ancient!).  Is it an option?

Comment 18

[Jonathan Griffin (:jgriffin)]

(In reply to Dustin J. Mitchell [:dustin] (I read my bugmail; don't needinfo me) from comment #17)
> From Jonathan's comment, it sounds like we've rejected the option of just
> upgrading Node to 0.10.21 or something similarly recent, but I don't see
> that rejection in any comment.  That'd certainly be the easiest solution
> here, and the most forward-compatible (in another six months, 0.8.x is going
> to seem even more ancient!).  Is it an option?

Yes, that's the preferred approach.  I had been concerned about the effect upgrading may have on other users of node, but :hurley's comment #16 indicates this shouldn't be an issue, so let's proceed with that approach.

Comment 19

[Dustin J. Mitchell [:dustin] (he/him)]

Building with the debian tarball from
  https://launchpad.net/~chris-lea/+archive/node.js/+packages
seems to work just fine, but I need to do a bit more work to build for different architectures.

Comment 20

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: bug931875.patch

This also moves nodejs-debian out of the mozilla/ directory and into the directory with the .pp file.

Comment 21

[Rail Aliiev [:rail]]

Comment on attachment 825547 [details] [diff] [review]
bug931875.patch

Review of attachment 825547 [details] [diff] [review]:
-----------------------------------------------------------------

rubber stamp!

Comment 22

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: bug931875-p1.patch

The new package doesn't provide, and indeed conflicts with nodejs-legacy, and also doesn't provide /usr/bin/node.  So, fixing that..

Interdiff:

diff --git a/modules/packages/manifests/nodejs.pp b/modules/packages/manifests/nodejs.pp
index b8ae17e..b1c9999 100644
--- a/modules/packages/manifests/nodejs.pp
+++ b/modules/packages/manifests/nodejs.pp
@@ -5,11 +5,20 @@ class packages::nodejs {
     case $::operatingsystem {
         Ubuntu: {
             package {
-                # Install nodejs-legacy package which contains node -> nodejs symlink.
                 # This package is a recompiled version of
                 # https://launchpad.net/~chris-lea/+archive/node.js/+packages
-                ["nodejs", "nodejs-legacy"]:
+                "nodejs":
                     ensure => '0.10.21-1chl1~precise1';
+                # and it includes node.1.gz which conflicts with nodejs-legacy,
+                # despite not including /usr/bin/node
+                "nodejs-legacy":
+                    ensure => absent,
+                    before => Package['nodejs'];
+            }
+            file {
+                "/usr/bin/node":
+                    ensure => link,
+                    target => "/usr/bin/nodejs";
             }
         }
         default: {

Comment 23

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 825935 [details] [diff] [review]
bug931875-p1.patch

 https://hg.mozilla.org/build/puppet/rev/76ae1abab540

Comment 24

[Dustin J. Mitchell [:dustin] (he/him)]

Bug 931875: bump repoflag; a=bustage 
https://hg.mozilla.org/build/puppet/rev/6fab97f02e51

Comment 25

[Dustin J. Mitchell [:dustin] (he/him)]

OK, this should be good.

Each host will only pick up this change after the *next* job they run from now on.  So just about every host will be covered in a few hours, although there may be some stragglers.

Gareth, please re-open if this proves not to be the case, or if there are other problems!