708915 - Shared CDN property maintainability

Status: RESOLVED WONTFIX

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Jake Maul [:jakem]]

+++ This bug was initially created as a clone of Bug #708727 +++

I have set up a shared CDN property, cdn.mozilla.net. This uses subdirectories to silo projects from each other. De Todos Para Todos is in cdn.mozilla.net/dtpt, for example.

This is currently fronted by Cedexis, hosted on Edgecast, as a non-SSL-only CDN. Origin is cdn-origin.mozilla.net, served by PHX1 Zeus, on the genericrhel6 cluster. There is no staging or dev site, as it's all static content with no Rewrites or Redirects of any kind.

We need to develop a standardized way for content here to be updated.

At least one web dev has rejected the idea of a centralized github repo that multiple projects commit to, so that's out.

One possibility is to have separate git repos/submodules for content that goes here, and we just check them all out individually into the relevant project directory (cdn.mozilla.net/whatever). This should work well, but requires web dev to make sure all their CDN content is silo'd away properly so we can check out the appropriate content areas.

Another possibility is to develop/implement some sort of interface for them to manage their content themselves. videos.mozilla.org uses WebDAV in this way, for example. That would work.

Another possibility is to implement some form of synchronization whereby the proper directories within a project are sync'd over to the proper directory in cdn.mozilla.net's DocumentRoot. This seems rather non-universal, as each project may be different.


Other things to keep in mind:

1) At present this is local storage on the genericrhel6 nodes. This will need moved to NFS if it gets very big.

2) User-generated content and multi-hosting have already been brought up. At present these are not supported. They can both *become* supported if this gets backed by WOS and fronted by a processing script that fetches the proper content and returns it with the proper MIME type and cache headers. That seems like a good end goal to shoot for, and jsocol has already done some work on such a script/app (rtucker may have as well).

3) Non-user-generated content often may need to be deployed/updated at the same time the matching site is deployed/updated. So, whatever we do should ideally be tied into deploy jobs in some way, or at least doable simultaneously. This may be handled as an entirely separate site at first, and web dev will just need to tell us when we need to deploy something on cdn.mozilla.net as well as the site itself.

Comment 1

[Shyam Mani [:fox2mike]]

Giving this to jake for now.

Comment 2

[Jake Maul [:jakem]]

This is going nowhere fast. We can re-evaluate in the future. For the time being it's become a lot simpler for us to set up SSL-capable CDN properties, so having a shared one available has become a smaller concern.