1491329 - Migrate start.mozilla.org from IT to CloudOps

Status: RESOLVED WONTFIX

(Cloud Services :: Operations: Miscellaneous, task)

Description

[Mike Connor [:mconnor]]

Getting into ancient history here, but before Firefox 4 we had remote home pages, using the format <locale>.(start|start2|start3).mozilla.com.  For a variety of reasons, users ended up with these URLs as a user preference that's persisted across versions.  These are broken today (DNS points at SCL3, but the start.mozilla.org page is in AWS).  In July we had 50k unique daily users hiting start.m.o, today we're around, with two huge drops.

Atoll (cced) is looking into fixing the current DNS issues, and I'm filing a product bug to do a cleanup of these values in a future update, but we still shouldn't let this break without noticing.

Can we get some sort of basic health check that ensures this doesn't break again?

Comment 1

[:Atoll]

I've been digging into our DNS and it looks like the breakage affects these three domains:

*.start.mozilla.com
*.start2.mozilla.com
*.start3.mozilla.com

Comment 2

[Habibullah Pagarkar (use Slack)]

Summarizing a slack conversation with me, Mike Conner, Atoll and Brian Pitts:

This isn't a service that Cloud Ops maintains. IT currently hosts it and has identified that it’s an in-product dependency and IT is only hosting RewriteRules. It makes sense to transition it over to Cloud Ops. As of 2 pm CDT it looks like the emergency fix is on its way to production. Habib to set up a meeting the following week with Atoll with sufficient heads up to look at migrating this over to Cloud Ops.

Comment 3

[:Atoll]

(In reply to Atoll R. Soderberg  [:atoll, :rsoderberg] from comment #1)
> I've been digging into our DNS and it looks like the breakage affects these
> three domains:
> 
> *.start.mozilla.com
> *.start2.mozilla.com
> *.start3.mozilla.com

Additionally:

*.start-prod.mozilla.com

(In reply to Habibullah Pagarkar from comment #2)
> This isn't a service that Cloud Ops maintains. IT currently hosts it and has
> identified that it’s an in-product dependency and IT is only hosting
> RewriteRules. It makes sense to transition it over to Cloud Ops. As of 2 pm
> CDT it looks like the emergency fix is on its way to production. Habib to
> set up a meeting the following week with Atoll with sufficient heads up to
> look at migrating this over to Cloud Ops.

I intend next week to propose that we migrate two sets of "start page" services from IT to CloudOps:

1) start.mozilla.org

We operate a static website with static content called "start.mozilla.org", auto-updating from two repositories:

https://start.mozilla.org/ originates from https://github.com/mozilla/fx36start/

https://start.mozilla.org/locale/ originates from https://github.com/mozilla-l10n/fx36start-l10n/

There appear to be build scripts in use with these repositories, but I'll reserve that for the "do the technical work" phase of this effort.

2) start.mozilla.com, start2, start3, start-prod, CC-DD.start/2/3/-prod.mozilla.com

We operate a set of content-free Apache rewrites and redirects for URLs inbound to these sites to direct them to the appropriate start page.

2)

Comment 4

[Habibullah Pagarkar (use Slack)]

jbuck to discuss with mconnor if these pages can actually be loaded by Firefox 3.6 (a browser long since out of support) and if a redirect to another Mozilla owned property is sufficient.

Comment 5

[:Atoll]

All startpage endpoints end up at start.mozilla.org, in case that helps investigations any. HTTPS hosting of start.mozilla.com redirects is not the same as start.mozilla.org itself so definitely test both .com and .org.

Comment 6

[Brian Pitts]

Oops, didn't mean to resolve that, just change the title.

Comment 7

[Brian Pitts]

From Slack:

cshields 12:29 PM
We're doing to be picking up start page decom work soon that was dropped late last year. all of the old Pre-Firefox4 start page DNS entries will be going away.

gcox 12:33 PM
And, they have, as of 1730 UTC. *.start.mozilla.com *.start2.mozilla.com *.start3.mozilla.com *.start-prod.mozilla.com