Use --no-bin-links when running npm install in gaia-node-modules to support Windows

RESOLVED WONTFIX

Status

Firefox OS
Gaia::Build
RESOLVED WONTFIX
3 years ago
3 years ago

People

(Reporter: julienw, Assigned: julienw)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Assignee)

Description

3 years ago
"make node_modules" is broken on Windows because our tar contains symbolic links.

We should run npm install with --no-bin-links (source: [1]) to prevent this.

We need to fix gaia-node-modules and update the README file there.

We don't need to fix gaia's build because "npm install" has good defaults on Windows, the problem here is that we use a prebuilt node_modules that was built on a Unix system.

[1] http://askubuntu.com/a/269735
(Assignee)

Updated

3 years ago
Assignee: nobody → felash

Comment 1

3 years ago
Created attachment 8587229 [details] [review]
[gaia-node-modules] julienw:1150378-use-no-bin-links > mozilla-b2g:master
(Assignee)

Comment 2

3 years ago
Comment on attachment 8587229 [details] [review]
[gaia-node-modules] julienw:1150378-use-no-bin-links > mozilla-b2g:master

I created this by running

  find -type d -name .bin -exec rm -rf {} \+

Then I checked we have no more links:

  find -type l

I tried removing node_modules completely and running "npm install --ignore-scripts --no-bin-links" but the result was a lot bigger than the current repository, so I chose a more chirurgical approach :)

Comment 3

3 years ago
Created attachment 8587233 [details] [review]
[gaia] julienw:1150378-use-julien-node-modules > mozilla-b2g:master
(Assignee)

Comment 4

3 years ago
Comment on attachment 8587233 [details] [review]
[gaia] julienw:1150378-use-julien-node-modules > mozilla-b2g:master

Hey Alexandre, can you try running "make node_modules" with this branch on Windows ?

Thanks !
Attachment #8587233 - Flags: feedback?(poirot.alex)
Comment on attachment 8587233 [details] [review]
[gaia] julienw:1150378-use-julien-node-modules > mozilla-b2g:master

Please consider getting access to Windows...

Alex@ALEX-PC ~/Desktop/mozilla/gaia
$ make node_modules
# Running make without using a dependency ensures that we can run
# "make node_modules" with a custom NODE_MODULES_GIT_URL variable, and then
# run another target without specifying the variable
make modules.tar
/bin/bash: line 0: pushd: c:\Users\Alex\Desktop\mozilla\gaia\distribution: No su
ch file or directory
/bin/bash: line 0: popd: directory stack empty
make[1]: Entering directory `/c/Users/Alex/Desktop/mozilla/gaia'
Downloading latest node_modules package. This may take several minutes...
wget -c -nv https://github.com/julienw/gaia-node-modules/tarball/9af9abacfac000c
af6ddbc03daf2de242bdbd618 &&\
        mv 9af9abacfac000caf6ddbc03daf2de242bdbd618 "modules.tar"
2015-04-02 10:57:53 URL:https://codeload.github.com/julienw/gaia-node-modules/le
gacy.tar.gz/9af9abacfac000caf6ddbc03daf2de242bdbd618 [34308048] -> "9af9abacfac0
00caf6ddbc03daf2de242bdbd618" [1]
make[1]: Leaving directory `/c/Users/Alex/Desktop/mozilla/gaia'
tar --wildcards --strip-components 1 -x -m -f modules.tar "julienw-gaia-node-mod
ules-*/node_modules"
tar: node_modules/marionette-profile-builder/node_modules/mozilla-profile-builde
r/node_modules/packaged-webapp/node_modules/readdirp/examples/node_modules/event
-stream/node_modules/optimist/node_modules/wordwrap/example: Cannot mkdir: File
or path name too long
tar: Exiting with failure status due to previous errors
make: *** [node_modules] Error 2
Attachment #8587233 - Flags: feedback?(poirot.alex) → feedback-
(Assignee)

Comment 6

3 years ago
Thanks Alexandre !

The issue is that "tar" does not use UNC paths to extract the archive.

I don't think this is easy to workaround this. It may be easier to either:

* wait for a version of npm that flatten the dependencies
* don't use gaia-node-modules on Windows and fallback to npm install.
* fallback to npm install anyway in case using gaia-node-modules is failing ?

What do you think kevin?
Flags: needinfo?(kgrandon)
I am ok falling back to npm install if we want. I'd like to redirect the needinfo to James/Aus first though. I believe the FxOS automation team had some plan to finally kill gaia-node-modules and make things nice. I'm not sure of what kind of timeline though, so if we want a quick fix, falling back to npm install works for me.
Flags: needinfo?(kgrandon)
Flags: needinfo?(jlal)
Flags: needinfo?(aus)

Comment 8

3 years ago
Quick fix would be to fall back to 'npm install' on Windows as kgrandon mentioned.

Longer term (starting with Linux on Taskcluster) is to pre-built a node module cache tarball that is platform specific and should extract properly. This tarball will be indexed on taskcluster (like Mulet builds currently) and will be retrievable as part of 'make node_modules'.
Flags: needinfo?(aus)
(Assignee)

Comment 9

3 years ago
aus, if you look closely to the comments here, you'll see one of the issue is that _tar_ itself does not work properly on Windows with too long paths. You'll need to find a solution to that.
(NI just so that you notice this)
Flags: needinfo?(aus)

Comment 10

3 years ago
Julien, I think we'll be able to dodge that issue as we'll be building the tar.gz file on a windows machine itself and it will be the same package you get if you were to do 'npm install' from 'gaia/' on a windows box today.
Flags: needinfo?(aus)
(Assignee)

Comment 11

3 years ago
If we can do that, this resolves all problems :)

Do you want to keep this bug for working on this or should we close WONTFIX ?
Flags: needinfo?(aus)

Comment 12

3 years ago
Meh, we can mark as WONTFIX. We'll get it for free when we start using taskcluster-npm-cache node module as part of the build instead of just using it on taskcluster.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Flags: needinfo?(aus)
Resolution: --- → WONTFIX
Flags: needinfo?(jlal)
You need to log in before you can comment on or make changes to this bug.