682398 - Setup dev & stage site for mozilla.org

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[James Long (:jlongster)]

Our current trunk and staging environments for mozilla.org/firefox are not relevant anymore. We need to update them to match what's currently in production.

This means that the merged site should be running on www-trunk.stage.mozilla.com and www.stage.mozilla.com. The home page should be the mozilla.org homepage and /firefox should show the product pages previously on mozilla.com.

This needs to be done soon, as we can't really test our code fully until we deploy it live.

We can also use the newer URL formats such as www-dev.allizom.org and such. http://www-dev.allizom.org is current based off the "merge" branch in SVN and that branch will be deleted, it needs to run off trunk.

Comment 1

[James Long (:jlongster)]

Jake has worked closely with this so it might be appropriate for him to do it.

Comment 2

[Jake Maul [:jakem]]

I will move www-dev.allizom.org to the trunk branch, and it will stay as dev. This will thus be trunk for the root dir, and trunk for /org.

I will then create www.allizom.org right beside it, which will be stage. What branches should this pull from? tags/production doesn't make much sense, because then stage and prod would auto-update simultaneously, meaning stage has no value. 'trunk' is not great for the same reason. Is there a tags/staging?


Since prod already auto-updates, I have no problem making all 3 of these auto-update. It's just a question of which branches they pull from. On that note, it would probably be good if /org in production was not pulling from 'trunk'. :)

Comment 3

[mcbmoz]

This is great, do you think we'll need to change the workflow for the community portions of the site with this update? If so, let me know so I can make sure David Boswell can manage the education/communication part of that.

Comment 4

[Jake Maul [:jakem]]

I don't know how the community is involved with the maintenance of this site, so can't really comment on that.

If there are active participants though, it'll probably be a good idea to inform them of the change so they know where the "live" dev/stage sites are going to be in the future. That should head off various "why don't my changes appear on www-trunk.stage.mozilla.com anymore?!" and "stage doesn't look anything like prod" reports. :)

Comment 5

[James Long (:jlongster)]

(In reply to Jake Maul [:jakem] from comment #2)
> I will move www-dev.allizom.org to the trunk branch, and it will stay as
> dev. This will thus be trunk for the root dir, and trunk for /org.
> 
> I will then create www.allizom.org right beside it, which will be stage.
> What branches should this pull from? tags/production doesn't make much
> sense, because then stage and prod would auto-update simultaneously, meaning
> stage has no value. 'trunk' is not great for the same reason. Is there a
> tags/staging?

Yep, there is exactly that, it's "tags/stage".

> 
> 
> Since prod already auto-updates, I have no problem making all 3 of these
> auto-update. It's just a question of which branches they pull from. On that
> note, it would probably be good if /org in production was not pulling from
> 'trunk'. :)

Cool. Yes, mozilla.org is weird. But it'll take some time to transition everyone off that workflow unfortunately. Live updates is kind of like crack to people.

Comment 6

[James Long (:jlongster)]

(In reply to mcbmoz from comment #3)
> This is great, do you think we'll need to change the workflow for the
> community portions of the site with this update? If so, let me know so I can
> make sure David Boswell can manage the education/communication part of that.

Nothing changes on their end. This all just which domains host the sites. The SVN repos are all exactly the same so devs don't know the difference.

We will make changes in that regard soon, and we'll need a huge campaign to let everyone know they we're about to break their dev environment.

Comment 7

[David Boswell]

Based on James' note in comment #6 it sounds like there would be no workflow changes for the historical www.mozilla.org site as part of this bug.  If I understand that correctly, then that seems best in the short-term (even though we want to fix the live updates longer term).

As James also mentions in comment #5, transitioning the people (both employees and volunteers) who use the current live update system will take some time, so it would be good not to have that be a blocker for this bug.

Ideally we would change the current staging situation only for the pages that get moved to Bedrock.  All other pages would be migrated or archived and would no longer be an issue.  This would limit the amount of education we need to go through and limit the pushback of people addicted to the live updates.

Comment 8

[James Long (:jlongster)]

(In reply to David Boswell from comment #7)
> Ideally we would change the current staging situation only for the pages
> that get moved to Bedrock.  All other pages would be migrated or archived
> and would no longer be an issue.  This would limit the amount of education
> we need to go through and limit the pushback of people addicted to the live
> updates.

Agreed, but note for Jake: we can still setup these staging site for our mozilla.org/firefox workflow.

Comment 9

[James Long (:jlongster)]

Hey guys, any update on this?

Comment 10

[Jake Maul [:jakem]]

I believe the dev environment should be good to go. This is www-dev.allizom.org (same as the test site used for the merged branch, except I've "svn switch"ed it trunk instead of branch/merge). Please let me know if there are any problems with this.

Still working on stage.

Comment 11

[Jake Maul [:jakem]]

Stage (www.allizom.org) should be up shortly as well... this one required a DNS change. It will be operational as soon as the new record propagates out to you.

This pulls from tags/stage on the main tree, and trunk on the /org tree.

I have copied the config files (includes/config.inc.php and org/includes/config.inc.php) from www-dev, and changed the 2 relevant settings in it.

I have also made the proper change to the config file in org/thunderbird/includes/config.inc.php (this file is actually *in* SVN).

CDN URLs are disabled, both in dev and stage. This would take some time to enable without negatively interacting with prod's CDN in some way.


Note that I have had to disable the two RewriteMap lines in Apache's config, because the org-urls-301.txt and org-urls-410.txt files apparently don't exist in the tags/stage tag yet. There might be other things missing, I can't really tell... probably would be good to sync tags/stage with either tags/production or trunk before too long. Let me know when those files get put in there so I can enable them in the config.


dev and stage both auto-update as discussed. Note that at some point in the future stage and prod will probably *not* auto-update anymore, but will be pushed out in some more "normal" fashion. Ideally some process that is still relatively painless, but involves a bit more sign-off-edness from QA. Stage is intended to be not only "the next version to be deployed" but also a test *deployment* for that version. For now this is just "svn up", but may be more complicated with bedrock later on. That means the push process for stage and prod should be close to identical. This is to cut down on deployment-related breakage. However, this is a future thing and there will be lots of discussion before this happens (bedrock seems a likely time for this to happen).


At this point I believe this bug is completed. Please re-open if you discover anything broken/missing.

Comment 12

[James Long (:jlongster)]

Jake, the /firefox URLs aren't working on either dev or stage.

http://www-dev.allizom.org/en-US/firefox/fx/
http://www.allizom.org/en-US/firefox/fx/

The mozilla.com codebase should be checked out with a /org subfolder, so if anything I'd expect the .org URLs not to work. Not sure though, can you look into it?

Comment 13

[Anthony Ricaud (:rik)]

For trunk, this is related to my work on bug 680147.

Closing this for the moment. Will reopen if I'm wrong.

Comment 14

[Jake Maul [:jakem]]

To follow up on comment 12... comment 13 explains why www-dev (trunk) would be broken.

I believe stage is fixed now, but not of my doing. I think the issue was just that tags/stage hadn't been updated yet with any of the merged code, and was probably seriously confused. I'm guessing that's happened, because I'm seeing the "Good news your Fx is up to date" stuff on that page now. In fact I see that at both links at the moment.

Comment 15

[James Long (:jlongster)]

Anthony has been working on it as well, so I think we're good.

Jake: I just recreated tags/stage so it should have all the merge stuff now. Feel free to uncomment the RewriteMap directives.

Comment 16

[Jake Maul [:jakem]]

Yep, I see the files. Both RewriteMap directives are re-enabled. Thanks!

Comment 17

[James Long (:jlongster)]

Woot! Thanks Jake!