921077 - Host Telemetry Dashboard site directly on S3

Status: RESOLVED FIXED

(Data Platform and Tools :: General, defect)

Description

[Mark Reid [:mreid]]

As I understand it, the telemetry dashboard is composed of a bunch of static html/js/css/data files.  As such, we should be able to host the site directly on Amazon S3 (possibly with the help of some DNS magic).

If we can compress things with a trick like this, so much the better:
http://stackoverflow.com/questions/5442011/serving-gzipped-css-and-javascript-from-amazon-cloudfront-via-s3

Comment 1

[Mark Reid [:mreid]]

The telemetry-dashboard code is in github here:
https://github.com/mozilla/telemetry-dashboard/

Comment 2

[Jonas Finnemann Jensen (:jonasfj)]

If/when using cloudfront we should watch out for latency issues... especially with data, we'll want to minimize this.
I'm currently in the processing of setting up a cron job that'll do a map/reduce job and upload .json data to a public S3 bucket... I have CORS enabled for this bucket too...

But the Javascript and HTML shouldn't be hosted in the same bucket, and Javascript + HTML should have decent CNAME...

Comment 3

[Benson Wong [:mostlygeek]]

:jonasfj, 
If the web app is static, how does it bust the browser cache to load the latest data?

Comment 4

[Jonas Finnemann Jensen (:jonasfj)]

@mostlygeek,

I'm not sure what we do now... I didn't setup the server, but from looking at it the server doesn't give any 
"Cache-Control" header. Which I guess is the thing to do, if you want the browser to always ask if there is
a new version.

We can configure S3 bucket CORS with a MaxAgeSeconds, but so far I've set it to 1 :)
Maybe we can set Expires for when do data updates in the future... I'm just saying we should watch out for
latency if we ever want to go for a cdn.

Comment 5

[Benson Wong [:mostlygeek]]

Maybe writing a simple index.json file, with a map {resource : changetime} might be useful. Then the web app can load up the index, and cache bust things with https://mybucket.s3/myobject.json?lastUpdateTime

Then you can leverage browser caching. 

Also if performance is what you want, you'll get further with gzipped content than a CDN. This is good anyways since a) less storage, b) less bandwidth, c) faster performance. 

see: http://stackoverflow.com/questions/3384508/using-gzip-with-html-pages-served-from-amazon-s3

Comment 6

[Jonas Finnemann Jensen (:jonasfj)]

Telemetry-dashboard static HTML is hosted at:
http://telemetry-dashboard-web.s3-website-us-east-1.amazonaws.com/

This is a public S3 bucket configured for static web hosting.

The telemetry aggregated JSON data is hosted at:
https://s3.amazonaws.com/telemetry-dashboard-data/v1/

This is a public S3 bucket, not configured for static web hosting.

Comment 7

[Jonas Finnemann Jensen (:jonasfj)]

@mostlygeek,
Sorry, did I forgot to mention that we need to setup CNAME for the webhosting S3 bucket.

That is we should figure out how to move telemetry-dash.mozilla.org to point to this bucket:
http://telemetry-dashboard-web.s3-website-us-east-1.amazonaws.com/

This is probably configured on AWS somehow...

Comment 8

[Jonas Finnemann Jensen (:jonasfj)]

DNS setup with done in bug 924551.
The currently deployment is still very fragile, and relies on branches from my personal repositories.
I'll be merging these into master, once the existing server is shutdown, which I'll check for now :)