1746334 - Windows: Tests not running due to yarn not being found

Status: RESOLVED FIXED

(DevTools :: General, defect, P2)

Description

[Gio de Guzman]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:96.0) Gecko/20100101 Firefox/96.0

Steps to reproduce:

0. Set up the codebase / mozilla-build on a Windows machine
1. Try to rundevtools-node-test-runner.js through ./mach node devtools/client/bin/devtools-node-test-runner.js --suite=shared_components in Mozilla Build or node devtools/client/bin/devtools-node-test-runner.js --suite=shared_components in another shell
2. Both attempts to run the tests fail due to yarn not being found

Attempts to install yarn either locally or through mach (via ./mach npm install -g yarn or something similar) does not resolve this issue. For a local shell, if yarn is available on the local machine, it is still not found; For mozilla-build, installing yarn via mach does not work / the script is still unable to run yarn.

Actual results:

$ ./mach node devtools/client/bin/devtools-node-test-runner.js --suite=shared_components
[devtools-node-test-runner] Extract suite argument
[devtools-node-test-runner] Found test suite: shared_components
[devtools-node-test-runner] Check `yarn` is available
[devtools-node-test-runner] ERROR: `yarn` is not installed. See https://yarnpkg.com/docs/install/

The error message is the same when run in a different shell as well.

Expected results:

Expected results are for the tests to run and pass successfully.

Ideally, yarn should be properly packaged with mozilla-build so it doesn't have to be installed on the local machine, and the test runs/finds yarn properly without adjustments to the local machine.

Alternatively, just the ability to run these tests with a local install of node/yarn, like we can on Linux, would be sufficient as well to allow development, although not perfect in terms of bundling the environment in a portable way.

Comment 1

[Mitchell Hentges [:mhentges] 🦀]

We've been slowly moving components from MozillaBuild to instead be managed by Mach: for example, Node hasn't been packaged with MozillaBuild for a couple years, and instead its fetched during bootstrapping.

installing yarn via mach does not work / the script is still unable to run yarn.

This sounds like the real bug to me.

Comment 2

[BugBot [:suhaib / :marco]]

The severity field is not set for this bug.
:Honza, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 3

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Julian, could you please look at this issue?

Comment 4

[Julian Descottes [:jdescottes]]

It should pickup the globally installed yarn.

I am not sure what ./mach npm install -g yarn is supposed to do compared to npm install -g yarn ?
If you run yarn -v in the command line before attempting to run the tests does it work for you?

Comment 5

[Julian Descottes [:jdescottes]]

Jan, can you reproduce the issue on a Windows machine? Are you unable to run the node tests?

Comment 6

[Gio de Guzman]

Here is the output from a Powershell terminal (attempting to run through both ./mach node and global node for thoroughness):

PS D:\Code\mozilla-unified> yarn -v
1.22.15
PS D:\Code\mozilla-unified> ./mach node devtools/client/bin/devtools-node-test-runner.js --suite=shared_components
No MOZILLABUILD environment variable found, terminating.
PS D:\Code\mozilla-unified> node devtools/client/bin/devtools-node-test-runner.js --suite=shared_components
[devtools-node-test-runner] Extract suite argument
[devtools-node-test-runner] Found test suite: shared_components
[devtools-node-test-runner] Check `yarn` is available
D:\Code\mozilla-unified\devtools\client\shared\components\test\node process.cwd()
<ref *1> Error: spawnSync D:\Code\mozilla-unified\mach ENOENT
    at Object.spawnSync (internal/child_process.js:1067:20)
    at spawnSync (child_process.js:602:24)
    at execFileSync (child_process.js:629:15)
    at runTests (D:\Code\mozilla-unified\devtools\client\bin\devtools-node-test-runner.js:151:5)
    at Object.<anonymous> (D:\Code\mozilla-unified\devtools\client\bin\devtools-node-test-runner.js:200:20)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12) {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'spawnSync D:\\Code\\mozilla-unified\\mach',
  path: 'D:\\Code\\mozilla-unified\\mach',
  spawnargs: [ 'npm', 'run', 'yarn', '--version' ],
  error: [Circular *1],
  status: null,
  signal: null,
  output: null,
  pid: 0,
  stdout: null,
  stderr: null
}
[devtools-node-test-runner] ERROR: `yarn` is not installed. See https://yarnpkg.com/docs/install/
PS D:\Code\mozilla-unified>

Here is the output from Mozilla-Build for the same three commands:

gianc@DESKTOP-PGIE5K7 /d/Code/mozilla-unified
$ yarn -v
/d/Code/mozilla-unified/node_modules/.bin/yarn: line 12: node: command not found

gianc@DESKTOP-PGIE5K7 /d/Code/mozilla-unified
$ ./mach node devtools/client/bin/devtools-node-test-runner.js --suite=shared_components
[devtools-node-test-runner] Extract suite argument
[devtools-node-test-runner] Found test suite: shared_components
[devtools-node-test-runner] Check `yarn` is available
d:\Code\mozilla-unified\devtools\client\shared\components\test\node process.cwd()
Error: spawnSync d:\Code\mozilla-unified\mach ENOENT
    at Object.spawnSync (internal/child_process.js:1042:20)
    at spawnSync (child_process.js:621:24)
    at execFileSync (child_process.js:648:15)
    at runTests (d:\Code\mozilla-unified\devtools\client\bin\devtools-node-test-runner.js:151:5)
    at Object.<anonymous> (d:\Code\mozilla-unified\devtools\client\bin\devtools-node-test-runner.js:200:20)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12) {
  errno: 'ENOENT',
  code: 'ENOENT',
  syscall: 'spawnSync d:\\Code\\mozilla-unified\\mach',
  path: 'd:\\Code\\mozilla-unified\\mach',
  spawnargs: [ 'npm', 'run', 'yarn', '--version' ],
  error: [Circular],
  status: null,
  signal: null,
  output: null,
  pid: 0,
  stdout: null,
  stderr: null
}
[devtools-node-test-runner] ERROR: `yarn` is not installed. See https://yarnpkg.com/docs/install/
Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python


gianc@DESKTOP-PGIE5K7 /d/Code/mozilla-unified
$ node devtools/client/bin/devtools-node-test-runner.js --suite=shared_components
bash: node: command not found

gianc@DESKTOP-PGIE5K7 /d/Code/mozilla-unified
$

Comment 7

[Jan Honza Odvarko [:Honza] (always need-info? me)]

I tested this on Win 10 and I am seeing exactly the same as specified in comment #0 and comment #6

honza MINGW64 /c/src/mozilla.org/mozilla-central/devtools/client/bin
$ node devtools-node-test-runner.js --suite=shared_components
[devtools-node-test-runner] Extract suite argument
[devtools-node-test-runner] Found test suite: shared_components
[devtools-node-test-runner] Check `yarn` is available
[devtools-node-test-runner] ERROR: `yarn` is not installed. See https://yarnpkg.com/docs/install/

Comment 8

[Julian Descottes [:jdescottes]]

Attachment: Bug 1746334 - [devtools] node tests: Suffix yarn command with .cmd on Windows

Comment 9

[Julian Descottes [:jdescottes]]

It seems that on windows we might have to add the ".cmd" suffix.
Can you try the attached patch?

Comment 10

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Still the same error on my machine.
Thanks for looking at this.

Comment 11

[Julian Descottes [:jdescottes]]

Thanks for checking, I just tested on windows, and I had a small mistake in the patch, can you test the latest version?

Comment 12

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Works for me now, thank you!

Test Suites: 68 passed, 68 total
Tests:       7 skipped, 772 passed, 779 total
Snapshots:   159 passed, 159 total

Comment 13

[Julian Descottes [:jdescottes]]

Thanks for checking Honza! Pushing to review. Hopefully that will also fix the issue from the summary with ./mach node.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8dfa72c559b1
[devtools] node tests: Suffix yarn command with .cmd on Windows r=Honza

Comment 15

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/8dfa72c559b1