768169 - New process for serving static content for Persona

Status: RESOLVED FIXED

(Cloud Services :: Operations: Miscellaneous, task)

Description

[Zachary Carter [:zaach]]

Please review the documentation on a new process for serving static content: https://etherpad.mozilla.org/static-process

Comment 1

[Lloyd Hilaiel [:lloyd]]

great writeup, zach!  This arch change has my r+ and full support.

I also wonder if along with this change we should change process names to reflect more clearly the responsibilities of the different processes?

browserid -> readapi
dbwriter -> writeapi

Comment 2

[Gregory Szorc [:gps]]

Not sure what the URL structure inside the static bit looks like. See http://gregoryszorc.com/notes/static-content-hosting.html for some tips on getting the most out of static content hosting.

Comment 3

[Lloyd Hilaiel [:lloyd]]

:gps this is a great reference, thanks!  I'll review in depth.

Currently you can check out our resource serving strategy by loading https://login.dev.anosrep.org

We built and use this thing: https://github.com/mozilla/connect-cachify

Comment 4

[Lloyd Hilaiel [:lloyd]]

bump!  where are we at with architecture review?  is there anything else you need from the dev team?

Comment 5

[Gene Wood [:gene]]

Pete, can you speak to this?

Comment 6

[Pete Fritchman [:petef]]

LGTM. One thing I'd like to see is a brief comment on why this content can't just be served from nginx (or some other static web server).

Comment 7

[Zachary Carter [:zaach]]

The static process will be serving semi-dynamic content (page templates for the Persona site) in addition to static resources at this time. But yes, static resources should be moved into a CDN/static server eventually.

Comment 8

[:Atoll]

Attachment: draft: add browserid-static process and router-metrics.json

first draft, incomplete

Comment 9

[:Atoll]

Attachment: draft: add browserid-static process and router-metrics.json (v2)

add daemontools run files

Comment 10

[:Atoll]

Attachment: draft: add browserid-static process and router-metrics.json (v3)

remove locale_directory config argument per changelog, still missing debug locale argument

Comment 11

[:Atoll]

Attachment: draft: add browserid-static process and router-metrics.json (v4)

remove "if stage" wrappers in various places, remove unrelated config change for parent bug

add svc_check_http!/static as a first guess at what the static monitoring path might be for nagios

add static bind_to 63200, static_url localhost:63200, location /static/__heartbeat__ -> localhost:63200/__heartbeat__

copy browserid 50001 rsbac rules to router 50008, static 50009

Comment 12

[:Atoll]

Attachment: draft: add browserid-static process and router-metrics.json (v5)

fix two unrelated rsbac typos (one functional, one cosmetic)

Comment 13

[:Atoll]

Attachment: draft: add browserid-static process and router-metrics.json (v6)

add daemontools run files for browserid-static service

remove duplicate router process and syntax-error close brace in webhead.pp

working on RSBAC issues with the new process

Comment 14

[:Atoll]

Attachment: puppet: add browserid-static process and router-metrics.json (v7)

changelog below for v6 is a complete re-do of the rsbac changes only.

add roles 50008 router, 50009 static

repair description for fd 50004 modules

repair role 50000 (uid browserid) access to nscd

clone role 50001 browserid to 50008 router, 50009 static

Comment 15

[:Atoll]

Attachment: sysadmins: add bidpush heartbeat check for static (v1)

requires "add browserid-static" patch (v7) for the nginx /static/__heartbeat__ redirect

confirmed working in staging

Comment 16

[James Bonacci [:jbonacci]]

QA will work with OPs to get this Verified in Stage once all the bits and pieces are done...

Comment 17

[:Atoll]

The puppet (v7) patch does not yet include the necessary graphs nor cepmon, and the monitoring component of the patch is untested. Work continues on those components.

Deployed train-2012.07.20 to stage with this bug's puppet (v7), sysadmins (v1) patches, and bug 776304's puppet (v1) patch.

Comment 18

[:Atoll]

Attachment: puppet: add browserid-static process and router-metrics.json (v8)

increase RAM allocation for BrowserID mysql databases from 256M to 50% of physical RAM

Comment 19

[:Atoll]

developed patch for this, but train has since derailed. returning this to the general pool for :gene to either triage+assign or close as desired.

Comment 20

[:Atoll]

this was eventually deployed in some fashion