679474 - [One Mozilla] Record Page Load Time for www.firefox.com (pre-merge/post-merge)

Status: RESOLVED FIXED

(www.mozilla.org :: General, defect)

Description

[mcbmoz]

www.firefox.com is our top referring source for FX downloads from http://www.mozilla.com/firefox/new

The merge of mozilla.org and .com will add a layer of redirects (at least 2 I believe).

Can we get a record of page load time before the merge, so that we can monitor it post merge if conversion decreases (as a possible factor)?

****
Background

Google produced this presentation in June 2011. It's mostly technical and gives
recommendations and some code examples: http://pagespeed-velocity2011.appspot.com/#1

Among the relevant use cases they reference:

DoubleClick removed one client-side redirect
--1.5 second reduction in ad load time
--12% increase in click-through rate

Google Maps enabled HTML5 Application Cache
--3-second reduction in average page load time

Comment 1

[Corey Shields [:cshields]]

Yes, any redirects we add will slow the load down..  Even moreso than usual because you can not 'cache' a redirect and hits that would have otherwise been sent from cache are now going back to the webserver.

So, with the multiple layers in mind a simple load time does show the end user experience but can be a bit misleading as to what is going on behind the scenes.

It is something that we won't be able to avoid in the merge..  But hopefully over time we will see fewer and fewer hits to the old URLs.

Comment 2

[mcbmoz]

(In reply to Corey Shields [:cshields] from comment #1)
> Yes, any redirects we add will slow the load down..  Even moreso than usual
> because you can not 'cache' a redirect and hits that would have otherwise
> been sent from cache are now going back to the webserver.
> 
> So, with the multiple layers in mind a simple load time does show the end
> user experience but can be a bit misleading as to what is going on behind
> the scenes.
> 
> It is something that we won't be able to avoid in the merge..  But hopefully
> over time we will see fewer and fewer hits to the old URLs.

Cory,

Thanks for that reply. Is there a way we can lock in on pre-merge page load time and post-merge, just so we have it on the record somewhere and can watch it over time?

Also, is there a human-readable way to export the redirect paths, so we can also over time work towards simplifying them (I believe the SUMO team did this as well over the past few years).

~CB

Comment 3

[mcbmoz]

Hey Jake,

Is there any way to record this pre-merge?

~CB

Comment 4

[Steven Garrity [:sgarrity]]

Attachment: webpagetest.org page results from Aug 23

Perhaps not scientific, but I had webpagetest.org do 10 runs of performance tests on http://www.mozilla.com/

Comment 5

[Steven Garrity [:sgarrity]]

Attachment: webpagetest.org all-request results from Aug 23

Comment 6

[Corey Shields [:cshields]]

What exactly are you trying to measure?  Load times will differ across the board depending on where the test is taken from and what you are testing. 

There is no real "clean room" scenario for testing, but if we stick to one test (like the one that Steven just ran) then that would be our closest ability to having a before and after..

(as an aside, please don't reassign my bugs without reaching out and asking me first, thanks)

Comment 7

[mcbmoz]

(In reply to Corey Shields [:cshields] from comment #6)
> What exactly are you trying to measure?  Load times will differ across the
> board depending on where the test is taken from and what you are testing. 
> 
> There is no real "clean room" scenario for testing, but if we stick to one
> test (like the one that Steven just ran) then that would be our closest
> ability to having a before and after..
> 
> (as an aside, please don't reassign my bugs without reaching out and asking
> me first, thanks)

Cory,

My concern has just been capturing what thh redirects add to page load and if that corresponds with any decrease in download - one of many contributing factors.

What Stephen just did is great, foreheadsmack-worthy moment. 

Good to know knowing the workflow for assigning/reassigning. This is the first time I've run a project up to this point. 

~cb

Comment 8

[Corey Shields [:cshields]]

(In reply to mcbmoz from comment #7)
> My concern has just been capturing what thh redirects add to page load and
> if that corresponds with any decrease in download - one of many contributing
> factors.

That's fine, but in retrospect is not the time to be asking those questions.  What happens if the redirects add to the load time (which they do) and downloads decrease?  Do we roll back?  I'm not aware that we have even come up with such a contingency.

Just trying to get a feel for where this request fits within the whole picture and what that means for us before people start coming up with numbers and theories behind them.

Comment 9

[mcbmoz]

(In reply to Corey Shields [:cshields] from comment #8)
> (In reply to mcbmoz from comment #7)
> > My concern has just been capturing what thh redirects add to page load and
> > if that corresponds with any decrease in download - one of many contributing
> > factors.
> 
> That's fine, but in retrospect is not the time to be asking those questions.
> What happens if the redirects add to the load time (which they do) and
> downloads decrease?  Do we roll back?  I'm not aware that we have even come
> up with such a contingency.
> 
> Just trying to get a feel for where this request fits within the whole
> picture and what that means for us before people start coming up with
> numbers and theories behind them.

Hey Cory,

We had hoped to cover this in the Thursday merge meeting that I added you to the last few times, but based on workflow preference and availability I filed this bug last week and followed up via email and with an etherpad about what we might do and any last things we may have forgotten. It's a lot of communication outside of bugzilla, which I realize is your main workflow (which is why I pursued this in here too).

There are a lot of unknowns and we have discussed and made plans for what to do if things go wrong, see tracking bug 610724. We'll also be updating the header/footer with internal links post-merge and eliminating all our own referrals from other sites to point directly to mozilla.org/firefox. As the site gets reindexed we should see some leveling off, less people coming in through search to the former moz.com funnel and more going directly to moz.org/firefox -- we have a little design patch for moz.org's current home page just in case we see traffic hitting that page out of an older user pattern.

I pinged Jake on this today, just in case he was able to chime in on this given what he's been working on for the release tomorrow  - I think we are in the best possible shape we could be in. The last few weeks have been super busy for you guys and both Jeremy and Jake have been awesome (especially with babies coming in left and right!), but we made it and we're about to cross the finish line tomorrow.

I think we're good here - you?

~cb

Comment 10

[Jake Maul [:jakem]]

We use Gomez for a lot of our performance tracking things like this. Unfortunately, in the past we tracked www.mozilla.com, and as of yesterday we started tracking www.mozilla.org. As you know, these aren't the same code bases, or even the same content. (mozilla.com became mozilla.org/firefox... we don't track that)

The old mozilla.com loaded in about 3 seconds on average. This is a world-wide average, so it's anywhere from 0.5 seconds to 6 seconds, depending on region.

The new/merged mozilla.org (*not* /firefox) loads in about 5 seconds on average. Again, world-wide average.

I don't have good stats on mozilla.org/firefox at the moment, but I suspect it will be very similar to the original mozilla.com. It uses the same CDN, and is served from the same servers, and is (roughly) the same content.

Of course, traffic to mozilla.com will be a bit slower due to the redirect.

Traffic to mozilla.net will be slower still, because that has an extra redirect layer at the moment... mozilla.net -> .com -> .org/firefox.


The discrepancy between old .com and new .org appears to be:

1) www.mozilla.com was previously on a CDN. The CDN is still used for www.mozilla.org/firefox, but not www.mozilla.org. As far as I can see, mozilla.org in general just isn't on a CDN. We could work on setting this up... I would recommend toying with the code base to have a static_prefix option, like mozilla.org/firefox has.


2) There seems to be more elements on the current mozilla.org than there were on the old mozilla.com. This means more round-trips to the server, so more latency.

Comment 11

[John Slater]

Thanks Jake. Over time we'll definitely want to optimize the root mozilla.org homepage for performance as much as possible, but the key comparison for now is the former mozilla.com vs mozilla.org/firefox. Keep us posted!

Comment 12

[Steven Garrity [:sgarrity]]

Attachment: webpagetest.org page results from Aug 27

Comment 13

[Steven Garrity [:sgarrity]]

Attachment: webpagetest.org all-request results from Aug 27

Here are the same set of tests from webpagetest.org run on the post-merge site (same URL, http://www.mozilla.com/).

On average, the 10 runs take 13% longer, though I don't really consider this a fair comparison. The results would be affected by server load, network load, etc. so even time-of-day and day-of-week would pollute the data.

Looking at one of the runs, the .com -> .org redirect takes a total of 356ms (including DNS lookup, connection, response). That 356ms was about 13% of the total load time, so my totally unscientific conclusion is that the redirect adds about 350ms (or roughly 13%) to the mozilla.com load time.

Comment 14

[mcbmoz]

Steven & Jake,

Thanks so much for keeping an eye on this. Gomez seems to be noting some slowness in performance as well. So far, downloads haven't been affected. 

Adding Blake & Anurag from metrics to weigh in on anything else that they can see.

Best,
CB

Comment 15

[Anurag Phadke[:[e-mail]]]

@chrissie: the apache logs won't give us much input on the load times. Web-trends should have that info though..

Comment 16

[Jake Maul [:jakem]]

(In reply to Steven Garrity from comment #13)
> Looking at one of the runs, the .com -> .org redirect takes a total of 356ms
> (including DNS lookup, connection, response). That 356ms was about 13% of
> the total load time, so my totally unscientific conclusion is that the
> redirect adds about 350ms (or roughly 13%) to the mozilla.com load time.

This seems like a reasonable expectation to me. You have 1 round-trip for the DNS lookup, and another for the actual request. The first should ideally be from somewhere close to you, but no guarantees.

There may be some things we can do to improve this, but no obvious quick-fixes come to mind.

One thing that might help indirectly would be to work with Google (and other search providers) on various search terms... for instance, "mozilla" returns mozilla.org first, but "firefox" returns mozilla.com first.

Comment 17

[Corey Shields [:cshields]]

well past the merge..  closing this out.