1629993 - ubuntu1804 - correct nodejs archive url

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Edwin Takahashi (:egao | infrequent contributor)]

Current version of nodejs that is being installed by install-node.sh is referencing an old version that no longer is hosted on nodejs.

Comment 1

[Geoff Brown [:gbrown]]

Is it possible to rely on the node toolchain instead? Like https://searchfox.org/mozilla-central/rev/72e3388f74458d369af4f6cdbaeaacb719523b8c/taskcluster/ci/build/linux.yml#41.

Comment 2

[Mark Banner (:standard8)]

(In reply to Edwin Takahashi (:egao) from comment #0)

Current version of nodejs that is being installed by install-node.sh is referencing an old version that no longer is hosted on nodejs.

It is available it can be found here:

https://nodejs.org/download/release/v10.19.0/node-v10.19.0-linux-x64.tar.xz

It looks like it was accidentally changed to use the "latest" link rather than the link for the release.

(In reply to Geoff Brown [:gbrown] from comment #1)

Is it possible to rely on the node toolchain instead? Like https://searchfox.org/mozilla-central/rev/72e3388f74458d369af4f6cdbaeaacb719523b8c/taskcluster/ci/build/linux.yml#41.

This was attempted in bug 1493610, but caused issues with the npm step that we run: https://phabricator.services.mozilla.com/D6795#155141

We are still working towards vendoring node_modules, so we could conceivably wait a bit longer for that and just fix the node link for now.

Comment 3

[Edwin Takahashi (:egao | infrequent contributor)]

(In reply to Mark Banner (:standard8) from comment #2)

(In reply to Edwin Takahashi (:egao) from comment #0)

Current version of nodejs that is being installed by install-node.sh is referencing an old version that no longer is hosted on nodejs.

It is available it can be found here:

https://nodejs.org/download/release/v10.19.0/node-v10.19.0-linux-x64.tar.xz

It looks like it was accidentally changed to use the "latest" link rather than the link for the release.

(In reply to Geoff Brown [:gbrown] from comment #1)

Is it possible to rely on the node toolchain instead? Like https://searchfox.org/mozilla-central/rev/72e3388f74458d369af4f6cdbaeaacb719523b8c/taskcluster/ci/build/linux.yml#41.

This was attempted in bug 1493610, but caused issues with the npm step that we run: https://phabricator.services.mozilla.com/D6795#155141

We are still working towards vendoring node_modules, so we could conceivably wait a bit longer for that and just fix the node link for now.

Thanks for chiming in!

If we're to take the simple path, I can put up a patch to fix the path and that would take care of this issue.

On the other hand, I did some experiments and found that of the suites scheduled on CI, only xpcshell test suite actually requires the presence of nodejs on the system.

For xpcshell, the task itself already references linux64-node as a dependency here. Checking over the logs for any xpcshell test runs on linux1804 and it seems the node artifact is pulled into MOZ_FETCHES_DIR but is not used, so I made some changes to the way xpcshell test harness sets up the environment and was able to get green runs:
https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&resultStatus=superseded%2Cusercancel%2Cretry%2Cpending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&classifiedState=unclassified&revision=4d92b1e646a7ecccf8bcfbc6530e0e5c86cbc141&selectedJob=297611603

Given the above my consideration is to discontinue the installation of node on test docker images used in CI by removing the dependency of install-node.sh as part of the docker image creation process and use the node artifact produced by the toolchain task. I do not believe this will cause any unexpected issues either.

EDIT: I suppose my assertion was wrong, looks like xpcshell experiences multiple regressions if I do not install nodejs as part of the setup. Looks like a simple patch to fix the path is better use of my time at the moment.

Comment 4

[Mark Banner (:standard8)]

(In reply to Edwin Takahashi (:egao) from comment #3)

On the other hand, I did some experiments and found that of the suites scheduled on CI, only xpcshell test suite actually requires the presence of nodejs on the system.

That doesn't sound right. The ESLint job definitely uses node and npm. As do various other node builders.

Moving the ESLint task across to the linux64-node toolchain is what bug 1493610 was about. I'd be surprised if the issues raised there have resolved themselves.

EDIT: I suppose my assertion was wrong, looks like xpcshell experiences multiple regressions if I do not install nodejs as part of the setup. Looks like a simple patch to fix the path is better use of my time at the moment.

Note: It is best not to edit posts with new information as they don't get notified to email, I almost missed this - edit is more for correcting minor issues with typos or misplaced information.

Going the simpler route would obviously get this fixed for now, though we should file a bug on moving the xpcshell-test builders across. The node builders already have bug 1493610 and hopefully we'll be able to vendor soon.

Comment 5

[Mike Hommey [:glandium]]

I don't think the builds you point to use the ubuntu test image.

Comment 6

[Edwin Takahashi (:egao | infrequent contributor)]

(In reply to Mark Banner (:standard8) from comment #4)

That doesn't sound right. The ESLint job definitely uses node and npm. As do various other node builders.

Moving the ESLint task across to the linux64-node toolchain is what bug 1493610 was about. I'd be surprised if the issues raised there have resolved themselves.

Right, I meant that out of the tests run on CI like mochitests, xpcshell, etc. it seems nodejs isn't required by all of them, so if it is possible to move nodejs dependency to a toolchain task or some other method, it would help slim down the docker image a bit more, but that runs up against the issue you've been facing with Bug 1493610 so I agree that it isn't feasible at the moment.

Anyway, I will put up a patch to correct the path so that rebuilding the docker images won't be broken for the time being.

Comment 7

[Mark Banner (:standard8)]

(In reply to Mike Hommey [:glandium] from comment #5)

I don't think the builds you point to use the ubuntu test image.

newtab-unit-tests uses the ubuntu test image (assuming that's ubuntu1804-test). The lint jobs and some of the node jobs use the lint image.

However both the ubuntu test image and the lint images are using install-node.sh: https://searchfox.org/mozilla-central/search?q=install-node.sh&case=false&regexp=false&path=

So my understanding is that comment 0 indicates that both sets are currently broken if they needed to be rebuilt.

Comment 8

[Edwin Takahashi (:egao | infrequent contributor)]

Attachment: Bug 1629993 - fix node path in install-node.sh

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bf98ccee76dd
fix node path in install-node.sh r=gbrown

Comment 10

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/bf98ccee76dd