1351907 - | yarn run start | fails on Windows

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect)

Description

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

$ yarn run start
yarn run v0.21.3
$ ./node_modules/neutrino/bin/neutrino start --presets ./neutrino-custom/development.js
'.\node_modules\neutrino\bin\neutrino' is not recognized as an internal or external command,
operable program or batch file.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.


16:29:22 <&emorley|pto> you need to yarn install
16:29:33 <&emorley|pto> either outside the VM or by running vagrant provision
16:29:59 <&emorley|pto> if you had a node_modules directory from ages ago, you might need to remove that in case it has cruft in it
16:30:16 <KWierso> this is on a machine brand new as of yesterday
16:30:34 <KWierso> after doing | yarn install --no-bin-links |
16:34:49 <KWierso> neutrino is definitely in node_modules/
16:37:23 — KWierso git checkout's an older commit before the switch to neutrino
16:39:11 <&emorley|pto> so I think the issue is that Windows doesn't understand files with a shebang line
16:44:55 <&emorley|pto> npm/yarn normally create a .cmd file, but if populated via vagrant provision that's not going to happen
16:45:19 <&emorley|pto> the workaround is to prefix the script commands in package.json with "node "
16:45:38 <&emorley|pto> or to run the command from within the VM
16:46:44 <&emorley|pto> could you file a bug?
16:50:07 <&emorley|pto> https://shapeshed.com/writing-cross-platform-node/#scripts-in-package-json

Comment 1

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

FWIW, Treeherder's UI runs via yarn just fine in the Win10 Bash on Windows shell (with the recently released update to Win10), so this isn't blocking me or making me use a VM. :)

Comment 2

[Ed Morley [:emorley]]

Ah good to hear.

I had a partial fix for this a few days ago (prefixing all the commands with "node "), however I discovered the commands containing environment variables don't work on Windows either, since it needs them to be of form `set FOO=bar node ...` not `FOO=bar node ...`.

One possible solution for this is a package called cross-env, however it turns out that with webpack 2 even the author recommends using the new webpack 2 `--env.foo=bar` options rather than environment variables. However Neutruino doesn't support this yet, so I've filed:
https://github.com/mozilla-neutrino/neutrino-dev/issues/165

See also discussion here (since it adds environment variables to more commands, so would otherwise break more of them on Windows):
https://github.com/mozilla/treeherder/pull/2309#pullrequestreview-30523134

Comment 3

[Ed Morley [:emorley]]

The new Neutrino v6 --options.env feature (https://neutrino.js.org/cli/#-options) will mean we can make package.json Windows-compatible without having to use cross-env or similar.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

I am not able to run this on windows 10 with the bash shell:
jmaher@JMAHER-WIN10:~/home/treeherder$ yarn install --no-bin-links
yarn install v0.27.5
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.90s.
jmaher@JMAHER-WIN10:~/home/treeherder$ yarn start
yarn start v0.27.5
$ ./node_modules/neutrino/bin/neutrino start --presets ./neutrino-custom/development.js
/mnt/c/Users/elvis/treeherder/node_modules/neutrino/bin/neutrino:53
run(args._[0], [...new Set(pkgPresets.concat(args.presets))]);
                ^^^

SyntaxError: Unexpected token ...
    at exports.runInThisContext (vm.js:53:16)
    at Module._compile (module.js:374:25)
    at Object.Module._extensions..js (module.js:417:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)
    at Function.Module.runMain (module.js:442:10)
    at startup (node.js:136:18)
    at node.js:966:3
error Command failed with exit code 1.
jmaher@JMAHER-WIN10:~/home/treeherder$


is it possible I have the wrong version of yarn, or node?

Comment 5

[Ed Morley [:emorley]]

What version of node is installed? Treeherder uses 7.10.0 currently (see package.json and .travis.yml).

Currently on Windows your best bet is to just use the Vagrant environment and run `yarn start` from there.

Comment 6

[Ed Morley [:emorley]]

s/best bet/solution with fewest hurdles/ (but I believe some people have got it working on Bash on Windows, so if you're fine with battling through, I'm happy to help :-))

Comment 7

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:node-windows > mozilla:master

Comment 8

[Ed Morley [:emorley]]

Comment on attachment 8920873 [details] [review]
[treeherder] mozilla:node-windows > mozilla:master

This is another piece split out of the Neutrino work, to try and reduce the PR size there :-)

Comment 9

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/2247a57d425339684f6be567aab55dd4422a01d7
Bug 1351907 - Improve yarn script Windows compatibility (#2863)

Typically the `package.json` commands call scripts that exist in the
top-level `node_modules/.bin/` directory, which are (a) already on
`PATH` when invoked by yarn, and (b) on Windows have `.cmd` script
shims that work around Windows' lack of shebang support.

However due to symlinks being problematic inside Virtualbox, we have
to use `--no-bin-links` with `yarn install`, which means the top-level
`.bin/` directory doesn't exist. As such, the `<package>/bin/` scripts
have to be used directly, which require prefixing with `node` to work
around lack of shebang support on Windows.

See:
https://shapeshed.com/writing-cross-platform-node/#scripts-in-package-json

Note that even with these changes, the `start:stage` command will not
work due to the environment variable prefix. This will be solved by
the switch to newer Neutrino, which supports setting environment
variables using the `--options` command line parameter instead.

Comment 10

[Ed Morley [:emorley]]

This is mostly fixed, bug 1364894 will do the rest :-)