Closed Bug 658066 Opened 13 years ago Closed 13 years ago

Redirect aus2.mozilla.org to aus3.mozilla.org

Categories

(mozilla.org Graveyard :: Server Operations, task)

Product:
mozilla.org Graveyard
Component:
Server Operations
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: robert.strong.bugs, Assigned: oremj)

References

Details

Attachments

(1 file)

Test suite
28.22 KB, application/zip
Details 
Turns out that there is virtualization software that is restoring our Firefox preference files which then points our update url which was changed in Firefox 4 to aus3 back to aus2.
This will have the effect of serving aus* from a single colo - aus2 is current San Jose + Phoenix, aus3 is just San Jose.
Adding Anthony to give QA a heads up. Anthony, I'll also test the hell out of this with you if you are able to also test this from the QA side.

Nick, since the majority of users will hopefully be using aus3 I hope that this will just be a case for adding redundancy (if necessary) to aus3 sooner rather than later.
Yeah, it was all about redundancy. I just wanted to remind ourselves we'd be losing some.
aus3 should be in both locations.  From Argentina it resolves to a San Jose address.

This bug should involve setting up phx & redirect?
From a QA perspective, I've added a spotcheck of the AUS URLs to our testplan. This information is already reported in our automated tests so it's an easy spotcheck.

In my testing the last couple of days I can confirm that I've been seeing AUS3 URLs, not AUS2.
(In reply to comment #4)
> aus3 should be in both locations.  From Argentina it resolves to a San Jose
> address.
> 
> This bug should involve setting up phx & redirect?

Yes.  I don't recall the reasoning right now but we only setup aus2 in phx.

Will put this on Jeremy's list to setup.
Assignee: server-ops → jeremy.orem+bugs
aus3 is set up in phx @ 63.245.217.44 (can someone test this?)

Looks like aus2's dns balancing is done by 3crowd. It wouldn't take long to duplicate for aus3 after aus3 has been verified at that IP.
Can someone verify the set up in phx?
Nick, can you verify the set up or should this go to someone else?
Assignee: jeremy.orem+bugs → nrthomas
I'll look at the tests we ran when aus2 was enabled in PHX (bug 621075).
Any update on when this can be done?
Attached file Test suite — 
I can't find any differences between aus03.zlb.phx.mozilla.net (63.245.217.44) and aus3.mozilla.org (aus3.acelb.sj.mozilla.com), including ssl certificates, so I'm happy for us to go ahead and enable aus3.m.o in PHX. 

Tests:
$ ./test-all.sh 2>&1 | tee full.log | grep -v ^PASS

Inter-colo AUS3 tests
Doing content tests...
  Testing _main/all...
  Testing _main/fx3.0.19-mu
FAIL Content length check failed for Firefox|3.0.19|WINNT_x86-msvc|2010031422|hu|betatest|3 (main=42 test=911
  Testing _main/fx3.5.16...
  Testing _main/fx3.6.13...
  Testing _main/fx4.0.1...
  Testing _main/tb2.0.0.23...
Done.

Doing cck fallback tests...
Done.

Skipping forced check test - nothing throttled right now

Doing PPC blocking tests...
Done.
----------

I redid the one failure a hundred times and couldn't reproduce it, so I'm putting it down to a transient glitch. It was the only failure in over 5000 checks.
Rob, did you have any other tests/QA in mind before making aus2.m.o point to aus3.m.o ?
Assignee: nrthomas → jeremy.orem+bugs
I would like to verify that Firefox 4.0.1 with the app.update.url preference default value set to
https://aus2.mozilla.org/update/3/%PRODUCT%/%VERSION%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml

is able to update to Firefox 5.

I'd also like to verify Firefox 3.6.x is able to update to a newer Firefox 3.6.x

I can verify these though I don't know of a way these can be tested.
I'm not sure how we would implement the 'redirection' - eg an actual HTTP redirect or by just making aus2.m.o and aus3.m.o the same in DNS. In the latter case you can probably use the hosts file to set aus2.mozilla.org to either of the aus3.m.o IPs - 63.245.209.149 (SJC) or 63.245.217.44 (PHX).
Just checking, no action for me yet?
Not yet. I am going perform the checks I outlined in comment #14 hopefully today
I performed the checks I outlined in comment #14 and all looks good when using an http redirect. Note: this will not work with pointing aus2 to aus3 in DNS.
(In reply to Jeremy Orem [oremj@mozilla.com] from comment #16)
> Just checking, no action for me yet?
It looks like everything is good to go. Can you comment in this bug or ping me on irc ( rs: ) when this change is made so I can verify it immediately after it has been made?
fyi: I saw the following in the error console while testing the PHX server
63.245.217.44 : server does not support RFC 5746, see CVE-2009-3555
Just to confirm, are we just doing a straight 302 to aus2 e.g., Redirect / https://aus3.mozilla.org/
Yes, a Redirect /
Also, when do you want to make this change? I'd like to be available to verify all is well when it is done. Thanks
Want to ping me when you are on irc.mozilla? My nick is just oremj.
I just verified updates from 3.6.19 -> 3.6.20 -> 5.0.1 -> 6.0 and from 4.0 -> 6.0 using aus2 by adding the pref files that are re-added by the virtualization software. All of them updated properly... thanks!
I'm not sure if this affects metrics so cc'ing a couple of people from metrics.

Daniel and Blake, aus2 redirects to aus3 now to workaround a bug in virtualization software that causes pref files that point to aus2 to be left behind which causes updates to break. Since Firefox itself is running inside the virtualization software there is no known way as of yet to fix this permanently and I have asked kev to contact the creator(s) to a) get them to fix their software and b) if possible, provide us with a way to fix this.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
It looks like we forgot to add PHX to aus3.m.o, so I filed bug 683446.
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: