925022 - [marionette-js-runner] Stage marionette js runner on TBPL

Status: RESOLVED INCOMPLETE

(Testing Graveyard :: JSMarionette, defect)

Description

[Gareth Aye [:gaye] (back from PTO)]

In order to do this, we need releng (eg Dustin) to deploy the npm-mirror utility wlach and I wrote. Currently it lives on github and npmjs.org. If we were to install it from github, we could do something like...

git clone [e-mail]:mozilla-b2g/npm-mirror.git
git clone [e-mail]:mozilla-b2g/gaia.git
cd npm-mirror

# The following command syncs all of npm package dependencies
# for gaia from http://registry.npmjs.org to ../packages. It also.
# writes the json manifest files necessary to have a web server
# rooted at ../packages listening on port 8080 behave like
# a bonafide npm mirror.
bin/npm-mirror \
    --host http://localhost:8080 \
    --manifestFile ../marionette-js-runner/package.json \
    --registry http://registry.npmjs.org \
    --packageDir ../packages

cd ../packages
# Start a web server listening on port 8080

I will go ahead and make the changes aki suggested in 902124 and then push my
changeset to http://hg.mozilla.org/build/mozharness. Then jgriffin suggested
we do a test build on tbpl staging (cedar).

Comment 1

[Gareth Aye [:gaye] (back from PTO)]

Oh also I tested my sync utility with wlach's https://github.com/wlach/mirror-npm/blob/master/npmserver.py which is built on top of mozhttpd. I think other things would work fine, but they probably have to set the right mime types and also allow something in lalala/index.json to get served at http://localhost:8080/lalala/.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Over in bug 912973, I'm waiting for some changes to the mirror script before we can finish the deployment.  So that mirror's not ready yet, pending work by :wlach.

Comment 3

[Gareth Aye [:gaye] (back from PTO)]

Hey Dustin,

Just FYI, I hijacked some of wlach's work to turn it into a more "open source" node utility with things like unit/integration/lint suites and documentation so that perhaps the gaia and larger js community can help us maintain it. I'll take a look at the issues you brought up in 912973 later today.

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

That's aweseome!

Comment 5

[Gareth Aye [:gaye] (back from PTO)]

Hey Dustin!

1. I believe npm needs absolute urls.
2. I still mirror all of the versions of all of the packages, but if it's too much data I can do follow-up work.

Can we go ahead and try deploying the node npm-mirror utility as is (much like you did with wlach's last iteration)?

Comment 6

[Gareth Aye [:gaye] (back from PTO)]

I would like to point the mozharness I wrote to it this week ideally :)

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

Sure, but the big question there was how to provide the mirror at two different URLs, with a few possible solutions.  Was one of those selected and/or implemented?

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

OK, bug 912973 is closed, but AIUI there's some additional work to mirror devDependencies.  I can re-deploy the mirror script whenever that's necessary.

I'm a little leery of embedding http://npm-mirror.pub.build.mozilla.org into the code directly.  If this is run from Mozharness, then I suspect it should be a config variable (/cc aki).  Otherwise, I'm not sure what method is best to specify this link.

Comment 9

[Aki Sasaki (not active)]

(In reply to Dustin J. Mitchell [:dustin] from comment #8)
> I'm a little leery of embedding http://npm-mirror.pub.build.mozilla.org into
> the code directly.  If this is run from Mozharness, then I suspect it should
> be a config variable (/cc aki).

That's correct.

Comment 10

[Evan Tseng [:evanxd]]

cc yurenju,

I think we could use npm-mirror in TBPL once the bug is fixed.

Comment 11

[Firefox Bug Husbandry Bot]

Bulk closed as part of https://bugzilla.mozilla.org/show_bug.cgi?id=1422750