Closed Bug 1431259 Opened 8 years ago Closed 7 years ago

Place a CDN in front of MDN (developer.mozilla.org)

Categories

(developer.mozilla.org Graveyard :: Performance, enhancement, P1)

Product:
developer.mozilla.org Graveyard
Component:
Performance
Platform:
All
Other
Type:
enhancement

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: rjohnson, Assigned: rjohnson)

References

Details

(Keywords: in-triage, Whiteboard: [specification][type:feature])

What problem would this feature solve? ====================================== A CDN in front of developer.mozilla.org will: * improve response times since endpoints will mostly be served from a cached response and because the cached response will come from a server that is geographically closer to the user * greatly reduce the load on MDN, which will in turn reduce downtime events due to scrapers and other issues * automatically provide a better level of protection against DDoS attacks (e.g., via AWS Shield) Who has this problem? ===================== All visitors to MDN How do you know that the users identified above have this problem? ================================================================== Although this change would benefit all users, it will most benefit users in locations a great distance from the west coast of the US, for example Bangladesh, who can experience slow page load times. How are the users identified above solving this problem now? ============================================================ They either tolerate slow load times or download a static copy (e.g., a tarball) of the site. Do you have any suggestions for solving the problem? Please explain in detail. ============================================================================== Place a CDN in front of MDN, at least for some endpoints. Is there anything else we should know? ====================================== No.
Assignee: nobody → rjohnson
Keywords: in-triage
Priority: -- → P1
Commit pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/7839d1966e26f79a3402ffad0f9187ae7b883ae0 bug 1431259: caching headers/tests for dashboards views (#4676) * bug 1431259: caching headers/tests for dashboards views * add "shared_cache_control" decorator as a thin wrapper around Django's "cache_control" to encapsulate default settings for shared caching * add tests for "shared_cache_control" decorator * delete custom "never_cache" decorator, and replace with Django's * delete tests for custom "never_cache" decorator * decorate the dashboards views and add tests * fix Cache-Control in test_ratelimit_429 * remove unnecessary assert for cache-control in response
Commit pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/4ba60584e82c5dcac2083ce44b167bc6b93953d7 bug 1431259: caching headers/tests for health views (#4677) * bug 1431259: caching headers/tests for health views * remove unnecessary assert of cache-control in response
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/1859bafd895d09bc75660dd70bc94078188aca83 bug 1431259: caching headers/tests for landing views https://github.com/mozilla/kuma/commit/e086fcb1930055da8ca3facb64f4dc50e86c8685 Merge pull request #4679 from escattone/cache-control-headers-landing-1431259 bug 1431259: caching headers/tests for landing views
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/1c97377665cb1a5ec870ad62628a602b509c0aec bug 1431259: caching headers/tests for search/events views https://github.com/mozilla/kuma/commit/ce52554811a53574a0495470aad58435949f4b26 Merge pull request #4680 from escattone/cache-control-headers-search-events-1431259 bug 1431259: caching headers/tests for search/events views
Commit pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/da9245976da5a404dcb96976942ff701600cf5ee bug 1431259: caching headers/tests for wiki misc/list views (#4687) * bug 1431259: caching headers/tests for wiki misc/list views * bug 1431259: adjust redirect_doc and localization_tags
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/7d5aadc47639d34508ec23ac513fa773cbd53bb6 bug 1431259: caching headers/tests for attachments views https://github.com/mozilla/kuma/commit/61aea62e532e391c5c8e26e0eaa5049d194ee0ad Merge pull request #4688 from escattone/cache-control-headers-attachments-1431259 bug 1431259: caching headers/tests for attachments views
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/93f295d7979e2f933540f968eb546634fd598218 bug 1431259: caching headers/tests for wiki code/legacy/akismet views https://github.com/mozilla/kuma/commit/2b3b8f3163a428a5186b403398198fec7b15c46f Merge pull request #4693 from escattone/cache-control-headers-wiki-code-legacy-1431259 bug 1431259: caching headers/tests for wiki code/legacy/akismet views
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/2de75d8ae501f5d543cf408cd2cb8d6aba3f7204 bug 1431259: caching headers/tests for wiki translate views https://github.com/mozilla/kuma/commit/7ad96618ac0f52f853e6f4d0256e951bbfb40c8f Merge pull request #4702 from escattone/cache-control-headers-wiki-translate-1431259 bug 1431259: caching headers/tests for wiki translate views
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/39623d48a54badcbe16b793071787398f6fea164 bug 1431259: caching headers/tests for wiki revision views https://github.com/mozilla/kuma/commit/1f43a1795f09ef5cd2040283d124b8803b1af383 Merge pull request #4704 from escattone/cache-control-headers-wiki-revision-1431259 bug 1431259: caching headers/tests for wiki revision views
Commit pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/277c2d9656720d62d23e502eaeefbbc3f0b8c520 bug 1431259: caching headers/tests for wiki create/edit/delete views (#4701) * bug 1431259: caching headers/tests for wiki create/edit/delete views * bug 1431259: address review feedback
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/71eb9464258c5396b8276c65e570de52e474f53c bug 1431259: caching headers/tests for version/set-language views https://github.com/mozilla/kuma/commit/dfe05c4830ef480a05330b510e5691682853d679 Merge pull request #4706 from escattone/cache-control-headers-version-1431259 bug 1431259: caching headers/tests for version/set-language views
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/4eb98f12aff800a82975baaf2bb758dd876ed9db bug 1431259: caching headers/tests for wiki feeds views https://github.com/mozilla/kuma/commit/a4726482a8749f3cfc6705cc135d433fe4394b63 Merge pull request #4707 from escattone/cache-control-headers-wiki-feeds-1431259 bug 1431259: caching headers/tests for wiki feeds views
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/b26f23974061b5db9376e810557d69d0ebc63437 bug 1431259: caching headers/tests for purge view https://github.com/mozilla/kuma/commit/e9f302f56c49ea06088e4b13d4864169d29e184d Merge pull request #4705 from escattone/cache-control-headers-purge-1431259 bug 1431259: caching headers/tests for purge view
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/13be42c6a49ddf60780e0353ae452590ba131d22 bug 1431259: caching headers/tests for various views https://github.com/mozilla/kuma/commit/b9f5b6a4945e71e0225ea3d69bb41738c3f4f3de Merge pull request #4717 from escattone/cache-control-headers-various-1431259 bug 1431259: caching headers/tests for various views
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/3ee1ab60ebeb2e293c258a7696c4b9c60625cce7 bug 1431259: caching headers/tests for wiki document views https://github.com/mozilla/kuma/commit/34f6a6d0d814ccbce7049c951547f87a6ce8eb18 Merge pull request #4719 from escattone/cache-control-headers-wiki-document-1431259 bug 1431259: caching headers/tests for wiki document views
Commits pushed to master at https://github.com/mdn/kumascript https://github.com/mdn/kumascript/commit/51f3aa559016a0c8df3cc5273f8b27fc1d0e753f bug 1431259: change stage domain to developer.allizom.org https://github.com/mdn/kumascript/commit/3b1be4064a0f22766a12a0bb094a0a5eb0310458 Merge pull request #662 from escattone/new-stage-domain-name-1431259 bug 1431259: change stage domain to developer.allizom.org
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/9d98c65d09f51c6d4434b8f589dfbfae1608ccc7 bug 1431259: change stage domain to developer.allizom.org https://github.com/mozilla/kuma/commit/9b5f82d9850762d49adecd5c1030ddc1c43cdaba Merge pull request #4725 from escattone/new-stage-domain-name-1431259 bug 1431259: change stage domain to developer.allizom.org
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/b8e44b2541768d466977768abbac08ca3d698985 bug 1431259: Don't cache the create page redirect Use a "no-cache" cache control header when redirecting to create a page, to avoid a loop where you have to wait for the cache to expire to see the new page. This also updates the shared_cache_control decorator to take no action if it detects the cache control header is set to no-cache. https://github.com/mozilla/kuma/commit/12514076caa1ccf09d2d6afff10e24f7d8de28d3 Merge pull request #4731 from jwhitlock/keep-no-cache-1431259 bug 1431259: Don't cache the "create page" redirect
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/50699bd1448a6a76bb27a76221e2ba19c9b02b0d bug 1431259: add cache control header to locale redirects https://github.com/mozilla/kuma/commit/019e4b13c22597f62fd78c78998a758581782a09 Merge pull request #4732 from escattone/cache-control-header-for-locale-redirects-1431259 bug 1431259: add cache control header to locale redirects
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/0011da2639808fd755937dfd16ffe03da0f65a49 bug 1431259: fix caching for miel endpoint https://github.com/mozilla/kuma/commit/7ac5b8a69c65195fd02f1603e19d58954477c388 Merge pull request #4733 from escattone/fix-miel-cache-control-1431259 bug 1431259: fix caching for miel endpoint
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/c9d2dacd7ea5f8a126521e7c65d1696ec25d067b bug 1431259: do not cache docs if logged-in user or ks errors https://github.com/mozilla/kuma/commit/f8ac57c7765a1fadc1aae4ddea2376625f37dd57 Merge pull request #4734 from escattone/cache-control-adjustments-for-docs-1431259 bug 1431259: do not cache docs if logged-in user or ks errors
Commit pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/195696d3101c9c71d7afe7a2e7c2be9c8077e8cf create/use test functions to encapsulate cache-control header assertions (#4736) * bug 1431259: encapsulate no/shared cache-control assertions * bug 1431259: ensure absence of s-maxage in assert_no_cache_header
See Also: → 1454574
We're going with CloudFront for the initial CDN, which uses the s-maxage header and requires some additional configuration, documented in the infra repo in a Terraform configuration: https://github.com/mozmeao/infra/blob/dc9bac03c3f7fbc887f0cef89f481ce59b2d3948/apps/mdn/mdn-aws/infra/mdn-cdn/mdn_cdn.tf CloudFront will handle all requests to developer.mozilla.org, serving some from the proxy cache and forwarding others to the backend web servers. We expect a faster experience for logged-out users (99% of MDN visitors), and a slightly slower experience for logged-in users (extra hop through CF servers, cache disabled). Because all requests are going through CloudFront, we're also going to switch the assets strategy. Instead of serving static assets from cdn.mdn.mozilla.net, we'll serve them from developer.mozilla.org. This should provide a modest performance increase, avoiding a follow-on DNS request and allowing follow-on requests through an HTTP/2 connection. CloudFront also likes to take care of compression, and only supports gzip, so there may be follow-on work to disable the GZip and Brotli middlewares in Kuma. The CDN switch is scheduled for today, April 17 2018, around 12:45 PM Pacific.
The CDN switch is scheduled for today, April 17 2018, around 10:45 AM Pacific. I was wrong about timezones.
Commit pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/8a9f4b51a22dbd597e17452c04ab4db31a6a7bce bug 1431259: remove compression middleware and its tests (#4740) * bug 1431259: remove compression middleware and its tests * bug 1431259: remove Brotli package and remove/simplify tests
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/9580b43bd8c6d25716d1285f88cd9db88bc76124 bug 1431259: add headless tests for cdn https://github.com/mozilla/kuma/commit/907c7ecb53371d67b446126c8540e50e842f56ea Merge pull request #4765 from escattone/cdn-headless-tests-1431259 bug 1431259: add headless tests for cdn
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/093e498a0ecf2c0f1bae107eb9f6e00b6a499d8d bug 1431259: fix headless tests with localhost https://github.com/mozilla/kuma/commit/b914ad986b88448a56f6eff01f6651d39ef9652e Merge pull request #4779 from escattone/fix-headless-tests-for-local-dev-1431259 bug 1431259: fix headless tests with localhost
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/5d202d6c8010e5a142877a10491589fad61e11c0 bug 1431259: Parametrize locale selection tests Use pytest.mark.parametrize to test the four cases in test_locale_selection_cached and _not_cached: - Accept-Language en-US - Accept-Language es - Accept-Language es, django-language cookie fr - Accept-Language es, django-language cookie fr, ?lang=de This will make it easier to debug failures when a document is available in English but not the other locales (the current sample database includes /en-US/Firefox, but not /es/Firefox, etc.) https://github.com/mozilla/kuma/commit/d5a21e62c743296e8d25eb764569cc847f4ad6ee Merge pull request #4788 from jwhitlock/parametrize-headless-locale-1431259 bug 1431259: Parametrize locale selection tests
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/0d5e5ae1c63e7229e9b7356cb32f999332c3f3ad bug 1431259: change cache TTL from 5min to 24hr (experiment) https://github.com/mozilla/kuma/commit/4cad652616ff4d4108b51d9003fa8cb8b0a7e2df Merge pull request #4845 from escattone/cdn-cache-ttl-experiment-1431259 bug 1431259: change cache TTL from 5min to 24hr (experiment)
See Also: → 1468414
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/cec2ac3e6f36c5f1834c0a3b5bbd4717509fc507 bug 1431259: change cache TTL from 24hr back to 5min https://github.com/mozilla/kuma/commit/8af2af07e1a1023ab67c7d6fe3810bd8e0b08dba Merge pull request #4855 from escattone/stop-cdn-cache-ttl-experiment-1431259 bug 1431259: change cache TTL from 24hr back to 5min
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/68fc6c036b476846ce01251c169f23b5dd7617cd bug 1431259: change cache TTL from 5min to 48hr (experiment) https://github.com/mozilla/kuma/commit/eb6cc1f053451b8c041586d634e9d84273eaa876 Merge pull request #4876 from escattone/cdn-cache-ttl-exp-2-1431259 bug 1431259: change cache TTL from 5min to 48hr (experiment)
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/e6b03736fa1a8e8cc78819f4511c19498d0e3b2b bug 1431259: reset cdn cache ttl to 5min https://github.com/mozilla/kuma/commit/40d6d9dcc16fcc242a069830948183525e7af951 Merge pull request #4885 from escattone/reset-cdn-cache-ttl-1431259 bug 1431259: reset cdn cache ttl to 5min
Product: developer.mozilla.org → developer.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.