Implement page content for Firefox download page dynamic content A/B test

VERIFIED FIXED

Status

defect
VERIFIED FIXED
11 years ago
7 years ago

People

(Reporter: drolnitzky, Assigned: wenzel)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

()

Attachments

(6 attachments)

Reporter

Description

11 years ago
Assets for the pages can be found here:
http://staging.theroyalorder.com/mozilla/FFDownload.zip

Mockups and further details can be accessed via grouphub.

There are 3 separate download pages plus 1 supplemental page (the "switch" page):
+ IE page
+ Firefox 2 and below page ("Upgrade page")
+ Firefox 3 and above page ("Customize/Do More Page")
+ Switch page (linked to from the IE page)
OS: Mac OS X → All
Hardware: x86 → All
Reporter

Updated

10 years ago
Blocks: 478236
Ryan is going to take a look and give an estimate for amount of work.  Still figuring out who is going to pick this up, however.

David - do you have an updated graphic for the workflow regarding the user-agent switching?  The last meeting we had left us in a confused state about:
1) do we take a sub-sample of overall traffic and only UA-switch on that sub-sample?
2) what are the overall percentages for each UA case
Assignee: morgamic → rdoherty
Reporter

Comment 2

10 years ago
Reporter

Comment 3

10 years ago
(In reply to comment #1)
> Ryan is going to take a look and give an estimate for amount of work.  Still
> figuring out who is going to pick this up, however.
> 
> David - do you have an updated graphic for the workflow regarding the
> user-agent switching?  The last meeting we had left us in a confused state
> about:
> 1) do we take a sub-sample of overall traffic and only UA-switch on that
> sub-sample?
> 2) what are the overall percentages for each UA case

I updated a flow diagram after the last meeting and had posted to basecamp.  I'll attach here for reference and add Ryan to the basecamp project as well.

It should be pretty straight-forward.  This assumes that we are taking 50% of all traffic directed to the Firefox download page for en-US and putting them into the "test" bucket. This large percentage will make for a faster test (large sample size, smaller margin of error), but if we aren't comfortable with 50%, the percentages on the flow diagram can be adjusted proportionately.
(In reply to comment #0)
> There are 3 separate download pages plus 1 supplemental page (the "switch"
> page):
> + IE page
> + Firefox 2 and below page ("Upgrade page")
> + Firefox 3 and above page ("Customize/Do More Page")
> + Switch page (linked to from the IE page)

Ok, I think I have the psds, copy and flow I need.

It's kinda hard to figure out where all the links go, none are labeled.

A rough estimate for these 4 pages (plus detection code) is week. Plus QA time.
Reporter

Comment 5

10 years ago
(In reply to comment #4)
> (In reply to comment #0)
> > There are 3 separate download pages plus 1 supplemental page (the "switch"
> > page):
> > + IE page
> > + Firefox 2 and below page ("Upgrade page")
> > + Firefox 3 and above page ("Customize/Do More Page")
> > + Switch page (linked to from the IE page)
> 
> Ok, I think I have the psds, copy and flow I need.
> 
> It's kinda hard to figure out where all the links go, none are labeled.

Added links and posted to basecamp.

> 
> A rough estimate for these 4 pages (plus detection code) is week. Plus QA time.
Fred is going to pick this up since Ryan is out for a week and this needs to be worked on during that time in order to have some data by end Q1.
Assignee: rdoherty → fwenzel
Fred - you've got mozilla.com already checked out, pretty similar drill to what you were ramping up for with the Fennec stuff.  I've added you to gropuhub, project page is here - https://mozilla.grouphub.com/projects/2811746/project/log

Can you take a look at the content and wrap your head around it?  Going to schedule a meeting with rolo next week to chat.

Ryan - thanks for taking a peek. :)
Assignee

Comment 8

10 years ago
All right!
Assignee

Comment 9

10 years ago
Rolo, is it possible to have the PSD files split up/exported into more usable chunks? For starters, the GIMP does not seem to get the colors straight, so if I use it to export things it won't do us any good.

What would help for the IE page for example is a PNG file of all the "pipes" in the background, the "bubble" background along with the "gauge", the awards background. The rest of the page seems to be the same as the current Firefox page.

For the "personal" page: The background, the add-ons guy, the "bullet point" pictures and the sidebar background stripes.

For the switch page: The background, each of the 5 illustrations, the sidebar background.

For the "upgrade" page: The background and the "top 5" box background.

Now of course, if you read some of this and think "but that's the same as page x on the current moz.com", please let me know as well, I may have overlooked that.
Reporter

Comment 10

10 years ago
I'll check with Royal Order on this.
Assignee

Comment 11

10 years ago
... though Ryan just mentioned I could use a free trial version of PS and cut
the layers out myself to get just the right pieces. Maybe I should try that first.
Reporter

Comment 12

10 years ago
Yeah, that would be preferrable if you can get your hands on a trial version.
Assignee

Comment 13

10 years ago
All right, will do. I'll get back to you if I have additional questions. Thanks!
Assignee

Comment 14

10 years ago
Another short question: The "personal" page is the one that's going to be shown to Fx3+ users. Still, it says "upgrade your firefox" on the bottom right. Is that even useful? I would assume the least of the people who see this page do *not* run the latest version?
Reporter

Comment 15

10 years ago
Yes, the upgrade your firefox should still be there - because if people don't have the latest version, they should still be encouraged to upgrade.  It's below the fold, so it's not the main message on this page, but we wanted to have it there anyway.
Assignee

Updated

10 years ago
Depends on: 482423
Assignee

Comment 16

10 years ago
Clouserw, is it acceptable if I commit my changes to SVN for QA to look at on stage? Or do you want me to attach patches and have them reviewed first.
Commit to trunk is fine. Use your own judgment if it will conflict with other not-live-yet changes on the same pages.
Assignee

Comment 18

10 years ago
David: The "switch" page has an FAQ. Do you have the final set of questions and answers for that somewhere?
Assignee

Comment 19

10 years ago
Another question: Does the new switch page replace this: http://www.mozilla.com/en-US/firefox/switch.html ? Or do you want me to preserve that old page and pick a different URL for the new one.
Reporter

Comment 20

10 years ago
Alix - do you have any clarity on the FAQ?

We can probably just use the /switch page url above for this project -- is there a way to archive that older content?  Maybe rename the current page switch-old?
Assignee

Comment 21

10 years ago
(In reply to comment #20)
> We can probably just use the /switch page url above for this project -- is
> there a way to archive that older content?  Maybe rename the current page
> switch-old?

I could do that, yes.

Same question for http://www.mozilla.com/en-US/firefox/upgrade.html and ...upgrade-2.html (curiously the pages seem identical). Do I replace upgrade.html with the new one?
Reporter

Comment 22

10 years ago
Do the same thing for that page (the original /upgrade page isn't in service I don't think).

Wil -- can you confirm that this is ok?
Assignee

Comment 23

10 years ago
Thanks for being so responsive, David. (Yet) another question: http://skitch.com/fredw/b8acw/personal-page-rough-background -- here's a screen shot where you can see that the background on the "personal" page has a very straight edge and does not blend smoothly into the background pattern at all -- unlike on the three other pages. I think this should be fixed.
Reporter

Comment 24

10 years ago
Definitely agree that it should be fixed.  Can you copy the background transition from the "customize" page?  Is there anything else you need to fix it?
Assignee

Comment 25

10 years ago
(In reply to comment #24)
> Definitely agree that it should be fixed.  Can you copy the background
> transition from the "customize" page?  Is there anything else you need to fix
> it?

Actually the customize page does not have a transition at all, its color matches the background tile so no transition is visible. So unfortunately I cannot easily fix it myself, sorry.

Second question (sorry for the bug spam): Does the "personal" page replace the current customize page, or is it unrelated?
Reporter

Comment 26

10 years ago
So to confirm, are you looking for better/new graphics files in order to fix the transition?

The "Personal" page does not replace the current customize page.  There are similarities, but it's different and unique.
Assignee

Comment 27

10 years ago
(In reply to comment #26)
> So to confirm, are you looking for better/new graphics files in order to fix
> the transition?

Yes, please. If you compare the other new pages with the "personal" page, you'll see the others blend into the tiled background (or at least connect to it smoothly). The same thing should happen on this page.

> The "Personal" page does not replace the current customize page.  There are
> similarities, but it's different and unique.

Thanks.
I just attached the switch page faq (rtf so you can see where links point to and pdf). Let me know if you need the content in a different format.
Reporter

Comment 31

10 years ago
Hey Fred -- here's an updated PSD which should help the blend issue from comment #27.

http://staging.theroyalorder.com/mozilla/moz_personal.psd.zip
Assignee

Comment 32

10 years ago
I committed what I have so far:
- https://www-trunk.stage.mozilla.com/en-US/firefox/ie.html
- https://www-trunk.stage.mozilla.com/en-US/firefox/switch.html
- https://www-trunk.stage.mozilla.com/en-US/firefox/personal.html

("personal" doesn't seem to have made it onto staging quite yet...)

The "switch" page contains Alix' new FAQ content now. Per comment 27, the "personal" page contains the "rough" background which I plan on swapping for the new version tomorrow.

The "upgrade" page is not done yet.

Stephen: Do you want to take a look at these pages in IE6/7 and see if everything looks all right?
Status: NEW → ASSIGNED
Assignee

Comment 33

10 years ago
All right, the background image on the "personal" page is updated and looks much better now.
Assignee

Comment 34

10 years ago
I added a new upgrade page as well:
- https://www-trunk.stage.mozilla.com/en-US/firefox/upgrade.html

CSS-wise I am done, unless something needs changed.

I am going to talk to clouserw and oremj about how to do the redirects properly for our setup.
Assignee

Comment 35

10 years ago
(In reply to comment #34)
> I am going to talk to clouserw and oremj about how to do the redirects properly
> for our setup.

Oremj will likely be able to assist, but we'll need to wait until Monday as he's on vacation.

Updated

10 years ago
Depends on: 483310

Updated

10 years ago
Depends on: 483320
Reporter

Comment 36

10 years ago
Hi Fred -- can we make sure that on the IE page (https://www-trunk.stage.mozilla.com/en-US/firefox/ie.html) that in the "What's so Great About Firefox" middle section, that we default to the "Firefox vs. IE" tab here (https://www-trunk.stage.mozilla.com/en-US/firefox/ie.html#feature-vsie)
Reporter

Comment 37

10 years ago
On the switch page, in box #4 ("Customization"), we should link back to the personalize test page.  Please link "thousands of ways" to: https://www-trunk.stage.mozilla.com/en-US/firefox/personal.html)

(if it's more appropriate to file a separate bug, I can do that as well).
Assignee

Comment 38

10 years ago
(In reply to comment #36)
> Hi Fred -- can we make sure that on the IE page
> (https://www-trunk.stage.mozilla.com/en-US/firefox/ie.html) that in the "What's
> so Great About Firefox" middle section, that we default to the "Firefox vs. IE"
> tab here
> (https://www-trunk.stage.mozilla.com/en-US/firefox/ie.html#feature-vsie)

Yes. I will see if we can actually let it default to this; otherwise we'll have to make sure to forward to the right URI (i.e., including the #feature-vsie) when detecting the user agent.

(In reply to comment #37)
> On the switch page, in box #4 ("Customization"), we should link back to the
> personalize test page.

Done: r23401.
Assignee

Updated

10 years ago
Depends on: 483763
Assignee

Comment 39

10 years ago
(In reply to comment #38)
> (In reply to comment #36)
> > Hi Fred -- can we make sure that on the IE page
> > (https://www-trunk.stage.mozilla.com/en-US/firefox/ie.html) that in the "What's
> > so Great About Firefox" middle section, that we default to the "Firefox vs. IE"
> > tab here
> > (https://www-trunk.stage.mozilla.com/en-US/firefox/ie.html#feature-vsie)
> 
> Yes. I will see if we can actually let it default to this

I wrote the code to allow it and it should work now (r23420).
Functionality in comment 37 and comment 39 has been verified; except for bug 483763, all dependent bugs have been verified fixed, too.

Krupa and I feel this is ready to go; Fred, want to mark it fixed?
Assignee

Comment 41

10 years ago
(In reply to comment #40)
> Krupa and I feel this is ready to go; Fred, want to mark it fixed?

Not quite, the redirect portion of the process is not implemented yet, but should be relatively simple. Oremj (CCed) is currently investigating what settings we need in order to forward users reliably by user agent.
Assignee

Comment 42

10 years ago
David, I have a question about the forwarding: The start point of the process diagram is "firefox.com", which I believe currently unconditionally forwards to http://www.mozilla.com/en-US/firefox/. Another URL with a similar forwarding pattern is getfirefox.com and there may be others as well.

Do you want the forwarding to happen only for people who specifically enter "firefox.com" into their browser, or do you want it to happen on http://www.mozilla.com/en-US/firefox/, no matter how people get there (i.e., also when going to mozilla.com, then clicking on projects, Firefox)?
Reporter

Comment 43

10 years ago
Hey Fred -- good question.  I would like the latter -- that is, no matter how people get to /firefox, they should be "eligible" as part of the test group.  So forward all possible users, regardless of whether they got their via a redirect (getfirefox.com, firefox.com, etc.).
Assignee

Comment 44

10 years ago
(In reply to comment #43)
> So forward all possible users, regardless of whether they got their via a
> redirect (getfirefox.com, firefox.com, etc.).

That sounds reasonable to me, will do.
Assignee

Comment 45

10 years ago
Here are some redirect rules for the central htaccess file.

Notes:
- the URI to be matched is quite complicated, to cover a trailing /, projects/ and index.html. If you think this could be simplified, please let me know. It's also repeated for each of the rules, but I don't know of a way to change that in .htaccess files. Is there one?
- The 50% matching is done by time (even seconds -> regular page, odd seconds -> UA matching)
- This requires the current index.html to be `svn mv`ed to firefox.html, so if you test this and get a 404 for that URL, that's why.
Attachment #368502 - Flags: review?(oremj)
Attachment #368502 - Flags: review?(morgamic)
Reporter

Comment 46

10 years ago
Hi all,

Is there an update here?  Have the redirect rules been put in place?  We need to launch this week (preferrably Tues/Wed) since it touches the download page and we need to have time to start it and then turn it off before the weekend.

-DR
(In reply to comment #46)
> Hi all,
> 
> Is there an update here?  Have the redirect rules been put in place?  We need
> to launch this week (preferrably Tues/Wed) since it touches the download page
> and we need to have time to start it and then turn it off before the weekend.
> 
> -DR

I'm guessing not, since there are two review requests pending.

Updated

10 years ago
Attachment #368502 - Flags: review?(oremj) → review+
Assignee

Comment 48

10 years ago
(In reply to comment #47)
> I'm guessing not, since there are two review requests pending.

Oremj just r+ed it, and I committed it (including moving the original firefox/index.html to firefox.html) to r23694.

Jeremy, when we push this, will you need to make special changes to Zeus so the rewrites are not cached?
Assignee

Comment 49

10 years ago
QA: Will you please check on www-trunk.stage if the rewrite rules behave according to the flow diagram (attachment 365290 [details])? If your tests pass, we are done code-wise and can push.

Jeremy or JSlater, will you please assist in pushing the changes then? I have not done it before. I am guessing, I'll have to push the changes to stage via kubla, and someone else will have to approve and take them from there to production?
(In reply to comment #48)
> (In reply to comment #47)
> > I'm guessing not, since there are two review requests pending.
> 
> Oremj just r+ed it, and I committed it (including moving the original
> firefox/index.html to firefox.html) to r23694.
> 
> Jeremy, when we push this, will you need to make special changes to Zeus so the
> rewrites are not cached?

Hmm, on second thought the no-cache headers should be set in the htaccess file.  I think you can set an environment variable in the rewrite rule and then set a conditional cache-control header based on that environment variable.

If that doesn't work I'll set up a no-cache rule on the lb.
Assignee

Comment 51

10 years ago
I tried out setting the no-cache headers in .htaccess and it turns out mod_rewrite's redirects remove these headers, so it does not seem to be possible to have mark a redirect with Cache-Control headers in .htaccess.

The question, of course, is: Are 302 (temporary redirect) headers *ever* cached? If not, we are golden anyway. We don't need the new pages not to be cached, we just don't want the redirect to be.

If yes, I can think of two options, which one do you prefer:
a) a setting in the cache
b) I move the 50/50 and user agent redirect magic out of .htaccess into PHP and set the no-cache headers there accordingly.
Comment on attachment 368502 [details] [diff] [review]
htaccess redirects

Are you saying this to me?
+RewriteCond %1 <3


If so, thanks.
The netscaler does cache 302s by default.  The main mozilla.com redirect (/ -> /locale) adds the no-cache headers.

I'll take a look and figure out if it is better to nocache it on the netscaler or apache.
They can be cached, but for our purposes here they aren't supposed to, are they?
Err, ok - thanks oremj.  <3 to you.
David: QA is blocked from testing this until we figure out the 302 caching.
Assignee

Comment 57

10 years ago
Oremj, please be so nice to let us know this morning how we should fix the redirect problem, so we can get the remaining code changes in and get the project on its way.

(In reply to comment #53)
> The netscaler does cache 302s by default.  The main mozilla.com redirect (/ ->
> /locale) adds the no-cache headers.

Unless I am mistaken, this happens in PHP (prefetch.php). We could do that as well (comment 51 b) if you want.
After playing with apache for a bit it turns out adding cache-control headers can't be done in / .htaccess file, but adding the following section to the vhost does work:

    <LocationMatch ^/en-US(/projects)?/firefox(/(index.html)?)?$>
        ExpiresActive off
        Header always set "Cache-Control" "no-store, no-cache, must-revalidate, post-check=0, pre-check=0, private, max-age=0"
        Header always set "Pragma" "no-cache"
    </LocationMatch>

Fred, does this solution work for you?
Assignee

Comment 59

10 years ago
(In reply to comment #58)
> Fred, does this solution work for you?

Thanks, Jeremy, yes this solution sure does work for me. To confirm, did you check if these headers are set as expected on the 302 redirect then?

Do you mind adding this directive to the www-trunk.stage vhost, so QA can check if it works properly with different UAs? I'll make sure to add a comment to the .htaccess file so we remember removing the vhost config when the test is over.

Because, if we end up doing a by-UA redirect *permanently*, we can probably just use the Vary: User-Agent header and we'll be fine.
Sample response:

< HTTP/1.1 302 Found
< Date: Tue, 24 Mar 2009 17:35:09 GMT
< Server: Apache/2.2.3 (Red Hat)
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0, private, max-age=0
< Pragma: no-cache
< Location: http://oremj-mozcom.khan.mozilla.org/en-US/firefox/firefox.html
< Content-Length: 342
< Connection: close
< Content-Type: text/html; charset=iso-8859-1


Unfortunately, the Vary header can't be used right now, because mozilla.com is still fronted by the netscalers.  We have to keep fronting it with the netscalers for now, because our Zeus cluster doesn't support global load balancing.
The directives have been added to www-trunk.stage
Assignee

Comment 62

10 years ago
(In reply to comment #60)
> Sample response:
> (...)

This is excellent.

> Unfortunately, the Vary header can't be used right now, because mozilla.com is
> still fronted by the netscalers.  We have to keep fronting it with the
> netscalers for now, because our Zeus cluster doesn't support global load
> balancing.

That's all right. We'll use it when it becomes available.
Assignee

Updated

10 years ago
Attachment #368502 - Flags: review?(morgamic)
Assignee

Comment 63

10 years ago
Stephen: Can you check if that works now? I seem to have trouble getting
www-trunk to forward me to a UA-specific page *at all* now. :(
Assignee

Comment 64

10 years ago
(In reply to comment #63)
> Stephen: Can you check if that works now? I seem to have trouble getting
> www-trunk to forward me to a UA-specific page *at all* now. :(

Followup: On my dev copy, both the redirects and oremj's header settings from comment 58 work fine.

On www-trunk, the NetScaler still seems to cache the 302 redirect, in spite of the no-cache headers being set correctly. Oremj, if you want to check it out, the page is: https://www-trunk.stage.mozilla.com/en-US/firefox/

It currently forwards every user to ie.html. Instead, it should observe the redirect rules and, for example, never forward Opera users to anything but firefox.html, and MSIE users 50% of the time to firefox.html, the other half of the time to ie.html.
Reporter

Comment 65

10 years ago
Wanted to make comment here on timing and address a conflict with Funnelcake on Thursday.  I'd like to propose the following:

+ Launch the test this evening during regularly scheduled IT downtime (7 pm PST).
+ Pause the test at 11:59 on Wednesday evening.  Start Funnelcake and run it until 12:01 AM Thursday morning.  Resume this test at 12:01 AM Thursday morning.
+ Make a determination (based on the data) whether to continue running the test Friday and through the weekend. 
+ Implement the IE page permanently on en-US if it statistically outperforms the current (control) page, prior to 4/1 *if* we have a valid test result.

Stephen has told me that if we can work out the caching issue early this afternoon, that he can QA before IT downtime at 7 tonight. 

Are there any issues with the above plan?  If we are able to have a valid test, and assuming the IE page outperforms the control, can we push this page live for *all* IE users prior to 4/1?
Assignee

Comment 66

10 years ago
- Launching the test tonight is fine by me when QA gives the green light.
- pausing it on Wednesday night, we'd need additional code that makes this happen. You want it to only be paused for 2 minutes?
- For putting the IE page in place permanently in case of success, we obviously need to change the code a little too but that's not a problem.
Assignee

Comment 67

10 years ago
Oremj: If I push the changes from trunk to staging via kubla and tag them all accordingly, will you be able to push them live in tonight's maintenance window?

Besides the htaccess changes, there are quite a few HTML/CSS and image files.
a=stephend for the functionality spec'd and tested in
https://bugzilla.mozilla.org/attachment.cgi?id=365290.

Tested (from /firefox):

* Firefox 2.0.0.20 -- 50% of the time I got the upgrade.html page
* Firefox 3.0.7 -- 50% of the time I got the personal.html page
* Firefox 3.5 candidates ("Shiretoko" in the user-agent string) -- always got
the control page, since it didn't have "Firefox" in the user-agent string and
we're sniffing for that)
* IE 6 / 7 / 8 -- 50% of the time, got the ie.html page
** switch.html was successfully linked from the ie.html page
* Safari 3.2.2/3.2.1 (Mac/Windows) -- always got the control page
* Google Chrome -- always got the control page
* Opera 9.64 -- always got the control page

Can be marked fixed -- verified on my end; when deployed tonight, I'll do a
post-launch check, too.
(In reply to comment #67)
> Oremj: If I push the changes from trunk to staging via kubla and tag them all
> accordingly, will you be able to push them live in tonight's maintenance
> window?
> 
> Besides the htaccess changes, there are quite a few HTML/CSS and image files.

Yeah, if you tag them all in kubla and file a push bug I can do that tonight.
Assignee

Comment 70

10 years ago
Rolo: Please confirm if you need a "pause" to be included in the code and how long. If I don't get that code in now, this won't automatically happen and I don't think we'll be able to push that change tomorrow outside a push window.

Also, by "pause" we mean "redirect everybody, regardless of user agent, to the control page", right?

As for pushing it live for *all* IE users before 4/1, as I said it's not a problem code-wise, we'll just need to schedule another push window with IT, probably.
Regarding comments 65 and 66, can you make sure the switch page (switch.html)
remains there permanently? It's launched as part of the test but is meant to be
a permanent landing page on mozilla.com. Thanks!
Reporter

Comment 72

10 years ago
Fred -- the pause would be for the entire day on Thursday (the length of the Funnelcake program).  So technically pause would start at 11:59 pm on Wednesday evening and not restart again until 12:00 AM on Friday.

And by "pause", you are correct--the normal website behavior would be available (everyone gets the normal Firefox page, aka "control" page).
Assignee

Comment 73

10 years ago
This additional rule will cease serving redirects for the entire day of 3/26.

Jeremy: Is that okay? Note that the old behavior (= no external redirect at all) is achieved through a rewrite rule. That implies, for that entire day, the Firefox page will not be cached. (Though its pictures, CSS, etc. will).
Attachment #369173 - Flags: review?
Assignee

Comment 74

10 years ago
(In reply to comment #73)
> This additional rule will cease serving redirects for the entire day of 3/26.

Alix: The new pages won't go anywhere, they'll stay available via their complete URL.
Why not push this after funnelcake to avoid committing one-off hacks?
Assignee

Comment 76

10 years ago
(In reply to comment #75)
> Why not push this after funnelcake to avoid committing one-off hacks?

I am fine either way. I guess David just wants the additional data from Wednesday.

David, you can have the final call here. Either I commit the "pause" code now, then prepare the push for tonight, or we run funnelcake without any of this first, and schedule a push for this after Thursday.
I think we've met our objectives for the quarter by having this code complete and ready to go, and it seems like everyone's happy with how things have turned out -- don't think anybody is going to complain if we schedule around funnelcake.  Have you guys discussed this with Dave Bottoms?
Also, how set in stone is funnelcake's schedule?
Assignee

Comment 79

10 years ago
I'm going to have to go for tonight, so I should either prepare the push now and file a bug for IT, or we need to reschedule.

Updated

10 years ago
Attachment #369173 - Flags: review? → review-
Comment on attachment 369173 [details] [diff] [review]
Funnelcake "pause" rule

For this pause it is probably better just to do a redirect.
I sent an email out about this -- we should resolve the scheduling conflict sans hack if we can.  Check email.
Reporter

Comment 82

10 years ago
Fred -- given that you are on the East Coast, how long do you have until we must have a go/no go on this decision?
Assignee

Comment 83

10 years ago
I am in Europe since the beginning of the month, sorry. But I can have morgamic and oremj push this accordingly, if necessary.

Jeremy/Mike: We'd need to add/commit the "pause" code to .htaccess (including the R=temp that oremj asked for), then push this from trunk to stage to prod on kubla. All relevant files are tagged as "dyn-landing" in kubla.
Sounds good - thanks Fred!  Go to bed, dude.  We'll figure it out from here. :)
Reporter

Comment 85

10 years ago
Per my email, here's the approach we should take:

Let's go ahead and start the test tonight, *without* the on/off timing rule.

We'll plan on shutting this off late on Wednesday.  Depending on how quantity looks, we'll either restart sometime on Friday and run it through the weekend, or determine that our test is done.  This will avoid adding in the hack.

*However* -- I want to make sure that should we need more quantity, that we can turn this on again sometime on Friday during the day.  Since we're running the test against a control, it doesn't matter when we run it (weekend or weekday) -- it just matters that we have a control vs. a test group and that we have enough quantity.
I've pushed what Fred tagged to production and added the directives from comment 58 to the virtual host.
http://www.mozilla.com/en-US/firefox/firefox.html is a 404; is that just a not-yet-synched-to-the-webheads thing?
(In reply to comment #87)
> http://www.mozilla.com/en-US/firefox/firefox.html is a 404; is that just a
> not-yet-synched-to-the-webheads thing?

That's now fixed, and I've tested post-push, and this is looking good.
Reporter

Comment 89

10 years ago
Awesome guys -- thanks for turning this around today.  Pages look good.
Woot!  Thanks for helping me talk through my questions and good job on the launch.  :P
Assignee

Comment 91

10 years ago
Oh yes, it works great for me. <burns>Eeeexcellent.</burns>
Assignee

Comment 92

10 years ago
I just committed the .htaccess changes (incl. external redirect as oremj requested) to r23779.

I pushed the change live via kubla.

Stephen: If you want to make sure the pause works as expected, all day Thursday everybody is supposed to be sent to .../firefox.html unconditionally. Starting Friday, the 50/50 and UA-based rules will be effective again.
Assignee

Comment 93

10 years ago
(In reply to comment #92)
> I pushed the change live via kubla.

Actually, it said it failed to commit the prod push. Oremj, can you do it?
Assignee

Updated

10 years ago
Depends on: 485256

Comment 94

10 years ago
Do we have an ETA for when this is being turned off (or paused) today?  I think Morgamic had suggested 7:00 PM PDT.
Assignee

Comment 95

10 years ago
Since the original code from attachment 369173 [details] [diff] [review] did not change, the pause is 24 hours, from 12am to 12am.
Reporter

Comment 96

10 years ago
Update: let's plan on continuing to run this after the funnelcake period ends tonight at midnight.  We'll assess tomorrow, but probably let it run over the weekend.  We have most of the data we need, but want to verify a couple things with a larger sample size.
Assignee

Comment 97

10 years ago
All right. The ruleset will automatically reactivate itself at midnight tonight.
We're ready to push this live; Alix and I sat down today and tested the following:

Windows XP
- Opera 9.6.4: default + auto
- Safari 3.2.2: default + auto
- Firefox 3.0.7: default + auto
- IE 6: windows + auto
- IE 7: windows + auto
- IE 8: windows + auto
- Fx 2: default + auto
- Chrome: default + auto

Windows Vista
- IE 7: windows + auto
- IE 8: Windows + auto
- Opera: default + auto
- Safari: default + auto
- Firefox 2:  default + auto
- Firefox 3: default + auto

Windows 2000
IE 6: default + auto
Fx 3: default + auto

Windows 2003 server
Fx3: default + auto
IE6: Windows + auto

Windows 98
IE6: warning + no dwn
Opera: warning + no dwn

Win 95, Me, NT4: can't test - no VM

Linux (Ubuntu 8.1)
- Firefox 3.0.7: default + auto
- Opera 9.6.4: default + auto

Mac 10.4:
Firefox : mac + auto
Safari: mac + auto
Opera: mac + auto

Mac 10.5:
Safari: mac + auto
Opera: mac + auto
Firefox: mac + auto

Comment 99

10 years ago
um, are you referring to /firefox or download.html?
Reporter

Comment 100

10 years ago
This should probably be in a new bug, no?
(In reply to comment #99)
> um, are you referring to /firefox or download.html?

Wrong bug, sorry -- meant to comment over in bug 482230.
/firefox redirection/A/B testing is verified to be back in and working, again.
Reporter

Comment 103

10 years ago
OK, the 50% traffic test should be taken down (the results are in).

Here's what needs to happen:

- Remove 50/50 redirect when users hit en-US/firefox
- IE page (ended up winning over the control), needs to be the default page for all IE users inbound to /firefox for en-US (will work on getting the page localized shortly).
- Any non Firefox, non IE user needs to see the current firefox download page (/firefox)
- Detection for the /personalize and the /updated pages needs to remain

Jeremy/Fred -- how soon can you remove the 50/50 redirect?
Fred, let me know when the code has been backed out in production and I'll wipe out those no-cache headers.
Reporter

Comment 105

10 years ago
To clarify, we are removing the *firefox/firefox.html page which is currently
rotating with /ie.html as the control page in the experiment.

The appropriate behavior should be (again, for en-US only for now until we can
localize):

User has IE: gets the ie.html page
User has Firefox version below 3.0: gets the /upgrade.html page
User has Firefox version 3.0 or above: gets the /personal.html page
User has a browser that is not IE or Firefox: gets the current download page
(en-US/firefox)
Time is running out if we're to get to this before our SFx push tonight (just making note of that; I think we just told the people helping us test SFx affiliate-point counting to find the green "Download Firefox - Free" button, instead).

Updated

10 years ago
Attachment #371603 - Attachment mime type: application/octet-stream → text/plain

Updated

10 years ago
Attachment #371603 - Attachment is patch: true

Updated

10 years ago
Attachment #371603 - Flags: review?(oremj) → review-
Comment on attachment 371603 [details] [diff] [review]
removes 1-day workaround and 50% sampling

Do we still want the UA forwarding?
Comment on attachment 371603 [details] [diff] [review]
removes 1-day workaround and 50% sampling

Nevermind. I see comment 105 specifies the behavior.  This bug is too long!
Attachment #371603 - Flags: review- → review+
For some reason, even on production, I stopped getting sent to http://www.mozilla.com/en-US/firefox/personal.html when using a Firefox 3+ browser; instead, I always get sent to http://www.mozilla.com/en-US/firefox/firefox.html.

(David and I sat down and tested this before we pushed to production, so it's regressed since then.)
Reporter

Comment 111

10 years ago
Yep, it's not working.  It's directing me (using Firefox 3+) to firefox/firefox.html (which was the "control" page for the test) instead of the /personal page.
Assignee

Comment 112

10 years ago
Mike and Jeremy: Thanks for picking this up while I was out.

Now, oremj, did you remove the no-cache headers? If so, it's getting cached and that's the reason why it is not working properly now. If you removed them, will you please put them back into place and flush (this part of) the cache?

Once m.com is behind the Zeus cache, we'll be able to use Vary: User-Agent, but for now, the no-cache rules will need to stay in place.

Also, once this is done, I agree with Jeremy that the bug is getting too long and we should close it out and file smaller issues as they arise.
This is still in place and was never removed (at least by me):
    <LocationMatch ^/en-US(/projects)?/firefox(/(index.html)?)?$>
        ExpiresActive off        Header always set "Cache-Control" "no-store, no-cache, must-revalidate, 
post-check=0, pre-check=0, private, max-age=0"
        Header always set "Pragma" "no-cache"
    </LocationMatch>
Hrm, should we remove or would that incorrectly cache redirects despite different UA strings?
Since UA based redirection is still in place we'll need to keep those headers.  Is everything working properly right now?
Reporter

Comment 116

10 years ago
From my unscientific tests, everything looks like it is working as it should: IE visitors see the ie.html page, firefox 3 users see the personalize.html page, and those below firefox 3 see the upgrade page.
Firefox 3.1/3.5+ users (Shiretoko nightly candidates) see http://www.mozilla.com/en-US/firefox/firefox.html 100% of the time when hitting /firefox, but the 3.0/3.0.0.x case is as David says in comment 116; they see http://www.mozilla.com/en-US/firefox/personal.html.

Is the first behavior OK, David?  I think we agreed we didn't care about Shiretoko (it doesn't have "firefox" in its user-agent string), so it seems we're all set here.
Reporter

Comment 118

10 years ago
Yep, I'm cool with the behavior above specifically for nightly/shiretoko users.
I think this should be FIXED then.  Thanks everyone.
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Assignee

Comment 120

10 years ago
(In reply to comment #119)
> I think this should be FIXED then.  Thanks everyone.

I agree. Thanks Mike and all involved.
Rockin'.
Status: RESOLVED → VERIFIED
Component: www.mozilla.org/firefox → www.mozilla.org
Component: www.mozilla.org → General
Product: Websites → www.mozilla.org
You need to log in before you can comment on or make changes to this bug.