730705 - kumascript: Work with IT to deploy kumascript service

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Editing, defect, P1)

Description

[Les Orchard [:lorchard]]

kumascript is a new node.js service, and I'm not sure if we have a set of best practices for deploying such services. Work with IT to get the service up and running on stage9 in anticipation of a production launch.

Comment 1

[Les Orchard [:lorchard]]

I'm skimming over code for BrowserID [1] and Open Badges [2] to get an idea of a Mozilla-deployable service based on node.js. But, any and all suggestions are welcome. 

I'm not sure what the current best practice is for launching daemons on RHEL, but I want it to work with whatever that is. So far, I'm assuming it will be a script that needs to be launched into the background, and will fork a few worker processes. I may use "up" [3] for that, in bug 730706.

I can implement whatever is helpful for monitoring (eg. a known URL on the service that responds with 200 OK?). 

There's also some logging machinery [4] that we can use and configure as appropriate.

And, one more deployment detail for this thing is that it's not a public-facing service. It's an HTTP-based service, but it only needs to be accessible from Kuma's Django frontends. It can be stuck behind a firewall or whatever is usually done with, say, a MySQL server. It should also be able to live on a cluster of machines behind a load balancer / cache.

[1] https://github.com/mozilla/browserid/
[2] https://github.com/mozilla/openbadges/
[3] https://github.com/learnboost/up
[4] https://github.com/indexzero/winston

Comment 2

[Les Orchard [:lorchard]]

So far, kumascript requires at least one node.js module that needs native code compilation (ie. node-fibers [1]).

Considering that and all the other pure-JS modules it needs, we probably need a way to roll up one or more RPMs for the node_modules directory that `npm install` builds for the service.

[1] https://github.com/laverdet/node-fibers/

Comment 3

[James Socol [:jsocol, :james]]

(In reply to Les Orchard [:lorchard] from comment #2)
> Considering that and all the other pure-JS modules it needs, we probably
> need a way to roll up one or more RPMs for the node_modules directory that
> `npm install` builds for the service.

fpm [1] might be helpful here.

[1] https://github.com/jordansissel/fpm

Comment 4

[Luke Crouch [:groovecoder]]

Discussed using Amazon/Rackspace vm for MDNYC. This bug is much bigger.

Comment 5

[James Socol [:jsocol, :james]]

This is a blocker to getting rid of MindTouch, so it's critical that this is on IT's radar and we have a plan for when it can get done. We have a hard deadline on this transition.

Corey, how can we make sure this is a top priority for early Q2?

Comment 6

[Eric Shepherd [:sheppy]]

We need to get this moving. Getting MDN up on Kuma needs to happen soon, and this is a priority to make it happen.

Comment 7

[Jake Maul [:jakem]]

Copying in a lot of IT-relevant context from email.

Based on this, I think we need to spin up, say, 1 of these for dev and stage, and 2 for production, behind the new SCL3 Zeus backend LB.

These will interact with the new django-only dev/stage/prod environments.

The current deki-based stage env (sm-devmostage*) will migrate to SCL3 as-is and continue working normally until EOL. It will not use these KumaScript servers.

The current prod env (pm-dekiwiki*) will not migrate to SCL3. It will live in SJC1 until EOL, and will not use these KumaScript servers.


----------

> It seems to me what we really need to do here is finish out the bug to
> get the new dev/stage instances online, without dekiwiki, and with
> KumaScript (node.js stuff).

Yes, the new Kuma (Django) + KumaScript (node) system can and should be deployed without any MindTouch installation.

> What is needed for KumaScript to be working? Just a bunch of node.js
> packages installed, or... ? I really don't know anything about it. That
> bug is pretty vague on exactly what we (IT) need to do.

The bug I filed (bug 730705) is mostly questions, because I don't know if we have a Mozilla Way of launching node.js services yet. I tried to follow in footsteps of BrowserID and OpenBadges.org. So, hopefully nothing too exotic

> Some random questions that come to mind for me:
>
> 1) What is KumaScript? I mean, what's it for?

MindTouch offers a thing called DekiScript. It enables wiki page authors to write server-side scripting that generates dynamic content using a language based on Lua.

    http://developer.mindtouch.com/en/docs/DekiScript

KumaScript is a replacement for DekiScript based on node.js and server-side JavaScript. It lets trusted wiki authors write JS templates, which can be invoked via macros used by general authors.

    https://wiki.mozilla.org/MDN/Kuma/Scripting#Proposed_solution_.231
    https://github.com/mozilla/kumascript

> 2) Does it run on a separate domain from developer.mozilla.org? (maybe
> dumb question based on the answer to #1)

KumaScript is intended as an internal-facing HTTP-based service, like MySQL or redis or memcache. Only the Django app machines need to talk to it. It's definitely not meant to be public-facing. The Django app can be configured with whatever base URLs point at KumaScript internally.

> 3) Does this need to run on separate hardware, or right on the existing
> 3 nodes?

KumaScript can live on the same nodes or separate hardware. It can also live behind a private HTTP load balancer / caching proxy, if we want to scale that way.

> 4) Is this an separate app, like Redis/Celery/memcache/MySQL, or more
> like a vendor lib?

Separate app, like redis/memcache. Needs to run as a background daemon, talks HTTP on a configurable port (eg. 9080)

> 5) Any extra ports need opening to the outside world (or the inside one)?

No inbound public traffic needs to hit KumaScript. Internally, we need ports for Django to reach it.

We'll need outgoing ports so that KumaScript templates can make queries to things like Bugzilla, Socorro, the MDN wiki itself (ie. Django), and maybe some other data services.

> 6) Is this needed only once Mindtouch is gone? That is, can we set this
> up on the *new* dev/stage/prod nodes, and ignore the existing ones?

Yeah, new servers can stay clear of MindTouch and just run the Django + node.js combination.

The content migration script we're working on relies on Django and MindTouch DB tables - but not a full MindTouch installation. So, the only part of MindTouch needed on new servers is a read-only connection to a production MindTouch DB - or tables from an imported DB dump

That said, we *would* like to have both systems coexist for a bit of a honeymoon period. That would give MDN contributors a chance to give the new system a beating before we cut over entirely.

Comment 8

[Luke Crouch [:groovecoder]]

Jake, now that you've set up kumascript once, can you do it again for us on stage9? That way we can use stage9 for kumascript test day(s).

Comment 9

[Les Orchard [:lorchard]]

(In reply to Luke Crouch [:groovecoder] from comment #8)
> Jake, now that you've set up kumascript once, can you do it again for us on
> stage9? That way we can use stage9 for kumascript test day(s).

I think we talked briefly about this in Santa Cruz, and came to the conclusion that it wouldn't work on stage9. And that having both wikis in operation at once might be confusing.

But, that said, if stage9 is a possibility, that would be handy

Comment 10

[ali spivak]

Are we any closes to having a final decision or recommendation on this? 

We need a fully functional staging site so we can test a as-close-to-production version of kuma - including kumascript. 

MindTouch support ends on July 15 so we really, really need to get this decided and implemented ASAP.