Closed Bug 651230 Opened 13 years ago Closed 11 years ago

Warning for non-supported Firefox versions

Categories

(support.mozilla.org :: Knowledge Base Software, task, P4)

Product:
support.mozilla.org
Component:
Knowledge Base Software
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
2013Q2

People

(Reporter: jsocol, Assigned: rehandalal+mozilla)

References

Details

(Whiteboard: u=user c=general p=1 s=2013.9)

Attachments

(1 file)

Firefox out of date warning banners - 01
1.13 MB, image/png
Details 
Once we're rolling out versions every three months, we'd like to display some warning to unsupported Firefox versions automatically. This would probably be site-wide. The warning would encourage users to upgrade to the latest and greatest.

This may not actually need to get finished in Q2. Early Q3 might be fine.
Target Milestone: 2011Q2 → 2011Q3
Whiteboard: [ux-wanted]
Can we use the Army of Awesome dropdown banner with different copy and styling? 
http://people.mozilla.com/~chowse/drop/sumo/customer-care/v1/top.html
Target Milestone: 2011Q3 → 2011Q4
I'm not sure that's a good idea. It obscures the search field and many people will miss the text that says "click here to close". I'd much rather use a variant that moves down the whole page, like on stack-overflow: http://stackoverflow.com/
Cleaning up 2011Q4
Target Milestone: 2011Q4 → ---
Bug necromancy!! 

The Release Management team is looking for ways to update more and more people to the latest version. Considering that we have about 6M visits and about 5M unique visitors a month with old versions of the browser...we could drive some of those updates ourselves.

Verdi mentioned that we even have some designs from this idea started.
I think all context before comment 4 is irrelevant (redesigns, yay). Except maybe comment 0.
Bram, how do you feel about http://people.mozilla.com/~chowse/drop/sumo/customer-care/v1/top.html minus the old design?
Priority: -- → P4
Whiteboard: [ux-wanted] → u=user c=general p= s=2013.backlog
Target Milestone: --- → 2013Q2
Since upgrade is a critical thing that we want to encourage every user to do, I prefer to make it a horizontal bar that moves the whole page down.

This will make the notification area feel a lot more permanent, though still dismissable, if the user really wants it to.

As far as implementation goes:
* We need to make this bar attractive, because upgrade is so important. Having an actionable download button will be seen more than a blue link, for example.
* I worry that this will break Tabzilla, because it will sit on top of it. Will this be a problem?

Lastly, the header approach might not be the right one at all. If that is the case, a less intrusive alternative might be to put the warning message on the header navigation area and highlight the area.

Something like this:
http://cl.ly/image/2w2m3H1x1s2a
(In reply to Bram Pitoyo [:bram] from comment #7)
> a less intrusive alternative might be to put the warning message
> on the header navigation area and highlight the area.
> 
> Something like this:
> http://cl.ly/image/2w2m3H1x1s2a

That doesn't seem much like a warning to me. It could be mistaken for part of the navigation.
We *may* want to try the less intrusive version first and see how much traffic it generates. If the conversion rate is too low (visits from old browsers who click on it is below a reasonable 10%) we can be more aggressive. 

We want users to upgrade, but we don't want to piss them off or lose them to another browser.
Here’s a more ‘intrusive’ version of a horizontal warning banner that moves the whole page down. I can design a bigger button or word the message differently, but these should give you an idea of its versatility.

The good news is that we have plenty of options to choose from. My advice is to start with something that will take the shortest time to implement and examine the result before trying something else.
This is great Bram.

The orange panel is the most "in your face" and hard to miss. If the implementation of this is easy, I would suggest going with the Light Yellow version of Comment 10 and make it more obvious and intrusive (Orange) depending of the conversion rate.

We should track the Upgrade Button for Analytics too (it's an external link). We need to attach the following event:

_gaq.push(['_trackEvent', 'External Links', 'Firefox Upgrades']);
Ricky, can you estimate the version from comment #10? It would push the page down.

Maybe we can use the same banner with a white background for important messages, for example when we want to inform a large portion of our user base about a recent issue.
Whiteboard: u=user c=general p= s=2013.backlog → u=user c=general p= s=2013.9
The UI is easy. What logic will we be following to to know if the browser is up to date? We can only determine the major versions. How do I know which versions are up to date? What am I comparing to?
Flags: needinfo?(a.topal)
We should start with pre Firefox 4 builds and expand from there.

What to other people think?
How about all Pre-Firefox 12 builds (so 11 and down). After Firefox 12 we stopped supporting several OS's, but users below 12 can still update to 12 (and if they can update further, they should receive a prompt to update to Firefox.latest).
Ricky, major version should be fine, we'll miss a few who are not on the current chemspill release, but I don't see that as a big issue. 

Tyler, so we should advertise Firefox 12 for everyone pre 12 and then ask them again to update to Firefox.latest?

I'd much rather ask them to update directly to the latest version (I guess we'd loose some old Mac OS and Win2000 users?)
Flags: needinfo?(a.topal)
I'd suggest just pointing users to https://support.mozilla.org/en-US/kb/update-firefox-latest-version, and letting them use the built in updater. This will update them to Firefox 12, and if they can go beyond that, they will get another prompt to update to Firefox.latest. Our wording should be along the lines of:

"Your Firefox appears to be outdated. Please update Firefox using these simple steps...LINK". 

Perhaps creating a really simple stub article that simplifies the information in the above article to encourage users to update.

In fact, we could even expand this to all Versions of Firefox 17 and lower if we aren't concerned about older OS's (Kadir, do you know how many win200 and Mac 10.4 users we get daily?). We can't do 17 or higher until the next ESR since there is no way to tell ESR and non-ESR from the UA.
To summarize:

* We are going with the yellowish option from Comment 10
* We will display this to Firefox users on version < 12 (make the 12 a setting)
* If the user hides it by clicking the X, we'll persist that to localstorage so we never show it again to them

Sounds like it should be 1pt.
Whiteboard: u=user c=general p= s=2013.9 → u=user c=general p=1 s=2013.9
* Would the information provided from the about:support plugin be able to give us more specific details? We could detect chemspills, for example. It is possible that many users have this. (As a side note, we should track who has the addon installed).

* If a user has version 13, shouldn't we recommend they upgrade? I know that some OSes can only go up to 12, but if they are past 12 already, they should be able to upgrade all the way, right?
Some data to bring light:

Firefox 4 and below: 10% of the visits
Firefox 5 to Firefox 12: 5% of the visits

The biggest group of the Fx5toFx12 is Firefox 10, ESR seems to haven't updated to 17 in many cases so we should be cautious about this.

I stick to my recommendation of doing 4 and below and see what's the conversion rate and the "complain rate".

Works?
>Would the information provided from the about:support plugin be able to give us more >specific details? We could detect chemspills, for example. It is possible that many users >have this. (As a side note, we should track who has the addon installed).

It would, but it's only accessible to us after a user goes through the AAQ. For just general users visiting the site who haven't installed the add-on, no. And we'd have to ask for permission to get that anyway. I don't think it would be worth the effort (until we get it built in-product as an easily accessible API, then we can ask for permission to give personalized support options on the site based on their detailed information. Hmmm, that's an interesting idea /me wheels start turning)

>If a user has version 13, shouldn't we recommend they upgrade? I know that some OSes can >only go up to 12, but if they are past 12 already, they should be able to upgrade all >the way, right?

No, we did drop support for 10.5 at 16. And ESR is currently 17, so we can't show the prompt to 17 users.

>The biggest group of the Fx5toFx12 is Firefox 10, ESR seems to haven't updated to 17 in >many cases so we should be cautious about this.

ESR 10 is no longer supported, so users of that version should be receiving update prompts. I wouldn't be concerned about ESR 10 users seeing an out of date prompt. They should take it up with their IT staff.

Below Firefox 4 users are likely either stuck there (PPC, etc.) or don't want to update (dislike change). I don't see us having much success with those users as compared to 4 and above (who often don't update because the updater is busted, etc.)
Tyler, you are right that ESR 10 is no longer supported. Unfortunately those users can't update to 17 themselves and they need to wait for their IT department to do so, so we are in the same position as the pre Firefox 4 users who are stuck. Because this pre 4 are way more at risk, I prefer to experiment the impact of this bar with them than with versions that are more up to date before rolling this out more aggressively.
I don't think we have to worry about the people who can't update (Windows & Mac). They accounted for about 155,000 visits (.66%) last month. In contrast there were about 5,050,000 Windows users (21.4%) not on the latest version that could be and about 272,000 Mac users (1.15%) that could be. Altogether that's about 22.5% of our users. I'd rather target all of them than worry about annoying .66%.
What’s your thought on the link that this banner should have? It could either say “Download” or be a download button that goes to the Firefox download page, or it could be a link to view a SUMO article for upgrading Firefox.

Or we could display both: a download button, followed by a link labeled “view instructions”.

I think that the more direct and actionable we can make it, the more user will succeed in actually updating. If every user has to pass through an article first in order to download Firefox, less of them would do it than if we’d download the relevant Firefox version directly upon clicking the “Download” button.
Assignee: nobody → rdalal
Verdi, what's your proposal then? To show the message to anyone in less than 20?

Bram, I will go with a link to the download page: 

http://www.mozilla.org/en-US/firefox/new/

If we could dynamically update the link to point to the latest build, we should go with that.

In any case, we need to track the links, to do so we should use Google Analytics. The right format for the link is: (using the previous link)

<a href="http://www.mozilla.org/en-US/firefox/new/" onClick="gaq.push(['_trackEvent', 'Outbound Links', 'Outdated Download Banner'); return false;">

We may want to add a delay on this link so the information that GA captures is more precise, in that case please follow the instructions explained here:

http://support.google.com/analytics/answer/1136920
>Bram, I will go with a link to the download page: 

I wouldn't point to the download unless we can UA sniff to now show this to unsupported OS's, as those users will download the new version of Firefox, install it, and it will break the working version of Firefox they have installed. Using the built in updater won't allow them to update to an unsupported version of Firefox, so that avoida this issue.
Right, when I say dynamically update the link I meant: if we can reuse the code that the download page uses to show the button, we should use that. Otherwise let's point to the Download page. IIRC the conversion rate of that particular page is quite high.
We have discussed duplicating the logic that links to the correct build of Firefox before, and decided that it is not appropriate to do. The best we can do is link to the download page.
The WebProd team is currently working to add conditional messaging to the /new page that handles all platforms and Fx browser versions that are viewing this page.

We will first implement appropriate messaging and download (or update) button for all desktop browser versions and unsupported platforms. Next we will address FxOS and Android users. I'll keep you posted on when it is implemented. Visual design for first step will be completed this week.
Wondering if we should make this bug dependent of that particular development or we can just go ahead and implement it in any case. 

Tyler, you seem to have a strong argument against promoting the "update" for users who can't update so we don't break their Fx installation. What's your take?
Flags: needinfo?(tdowner)
(In reply to Ibai Garcia [:ibai] from comment #25)
> Verdi, what's your proposal then? To show the message to anyone in less than
> 20?

I think we should show it everyone not on the latest but we should be two weeks behind the release schedule. In other words Fx 20 users won't see the message until two weeks after 21 is out. Most people will update by then. If they haven't, they need help/a push. 

I think we should link to the download page just like we do on /products/firefox - that's the easiest and most straight-forward approach. On the other hand, sending people to the update instructions - https://support.mozilla.org/kb/update-firefox-latest-version#w_how-do-i-manually-check-for-updates - means getting them to read and follow instructions which, for some users, will only partially update them.
Can we implement the banner with some dead simple logic for now and then make it something more complicated?

It would be great if we can use this bug to just implement the UI and show it for Firefox version < X. Then we can open a new bug to define more complicated logic.

Otherwise, we'll have to drop this to the backlog until it gets spec'd out fully.
If we are going to ignore the number of users on unsupported OS's (which it sounds like Verdi is saying is  pretty small percentage) then I just want to make sure that's clear. Linking directly to the download page will probably break Firefox for these users, while linking to an article about how to update (it doesn't have to be https://support.mozilla.org/kb/update-firefox-latest-version#w_how-do-i-manually-check-for-updates, we can make something lighter) won't. but it will also result in a lower conversion rate as well.
Flags: needinfo?(tdowner)
And this is why I proposed to go with a small sample first and move from there. Basically we want to test how efficient this is before putting much effort on it as Ricky mentions. 

If we don't want to risk people who are stuck in old machines, can we try Firefox 12 to Firefox 18? From there we can roll it out to older versions if the conversion is good.

The download page at moz.org has a conversion rate of 66%. Adding an article in between will make this way lower, diminishing the purpose of this banner.
I would run from Firefox 12 to Firefox 16 (this represents a sizable portion of our users on older versions) and avoid the conflict of presenting this warning to ESR users as well. We could also include Firefox 18 and 19 (I think we shouldn't show this prompt to ESR at all). We could also show to all 3.6 users and lower, those users are getting prompts from Facebook and other sites and we should show them prompts to update as well. Then we can start showing to 4-11 once we get a good strategy.
What Tyler says.
So to clarify. We're showing this to 

FF <= 3.6 or (FF >= 12 and FF <= 16)

Correct?
Rehan, let's avoid pre 3.6 releases for now, so:

(FF >= 12 and FF <= 16)

Works?

Ibai
Well, we can go ahead with this, but since we can't tell who is an ESR user and who is just on an outdated version I'd just show this to all of them >12. If some ESR users update to the latest version: Well, even better ;)

Tyler, what's your take?

Rehan, to make sure this doesn't fall off the sprint, please go ahead and implement comment #38 if the timing becomes an issue.
Kadir, the problem is that ESR users in a corporate environment can't update by themselves.
Yeah, but the worst that is going to happen for them is that they download a version which they can't install. It won't break their Firefox or anything. They don't have the rights to install any executable.
in Re to ESR 10:
I'm fine with showing this prompt to ESR 10 since it is out of date and not supported anymore. As kadir says in comment 41, a user with non-install rights won't be able to install a newer version anyway (not changing anything for them) nd for those users who either can install or installed ESR by mistake (like home users) this is a good prompt to see. Business users who see this will complain to their IT, and their IT may then decide to update them.

Also, SUMO isn't really for ESR users, they should be going to their internal IT departments anyway, or through the EWG mailing list (http://www.mozilla.org/en-US/firefox/organizations/faq/). 

So basically I see no loss from showing this prompt to ESR 10 users, and a great gain. and if we don't show to 10, we will lose any home users on non-ESR 10.
Right, they can't install the update. They will invest some minutes to download and install the software to find out that it's not possible and it could result in a call to the IT department.

Even if this only happens in a dozen of cases, it's a dozen of frustrated people.

In the other hand, we can test this banner with version 12 to 16 that we know can be in 20 without problems. Users in 17 to 19 are in "non safe versions" but if they haven updated yet, they are probably happy campers in the version they are.

My question is, what's the benefits of showing this initial iteration to version 17 to 19? Is it worth the false positives of 17 ESR?
I just spoke with Tyler and it seems that there's a misunderstanding that made me think that we wanted to discard pre 3.6 releases.

Rehan, here is the plan:

Go for 

FF <= 3.6 or (FF >= 12 and FF <= 16)

As you said before.

We will measure the conversion rate of this feature. If everything looks good (good conversion rate and no complains) we then will roll it out to 

FF != 17

Unless we can identify ESR builds and in that case we can do 

FF != 17 ESR

I hope this makes sense. We basically don't want to create a conflict with IT departments who have deployed FF and have an Executive yelling at them because their browser is out of date...when it's actually not.
My argument is that a handful of people who ask their IT departments why SUMO is telling them that they are on an outdated version is way better than leaving hundreds of thousands of people on outdated and insecure versions of Firefox, when we could help them secure their browser. And who knows, some IT departments might even update Firefox at that point. 

Leaving aside for the moment the argument, that people in corporations with IT departments won't have much need for SUMO anyway. They'd just tell IT that something is not working. 

For sake of moving this along, let's go with <=16, if that gets us closer to the  deployment of this feature to a part of our userbase.
I think there's a lot of hand-waving about enterprise users here and a total lack of data.

My vote is we implement it in such a way that the SUMO team maintains which versions get the notice and which don't. We could (ab)use the showfor data (something we already maintain) to know the total number of things released. Then we have a form available to SUMO team where they can check off which versions get the notice and which don't. Then SUMO team can deal with it and evolve the released versions list as the discover things about their user base and it doesn't tie up development resources twiddling with numbers and maintaining it.
(In reply to Will Kahn-Greene [:willkg] from comment #46)
> My vote is we implement it in such a way that the SUMO team maintains which
> versions get the notice and which don't. We could (ab)use the showfor data
> (something we already maintain) to know the total number of things released.

If we do this, we might as well do Bug 768244 with it. That's the right long term solution.

I guess what I am trying to do is split this into two bugs:
* Implement the UI with some dead simple logic (this bug)
* Implement the logic, which probably requires implementing Bug 768244 if we want to give full control to non-devs.

Sound like a plan? Let's go forward with implementing the UI and show it to users as spec'd in Comment 44: FF <= 3.6 or (FF >= 12 and FF <= 16). Data can be collected and then we'll know if it even makes sense to spend time on the ideal solution where changes can happen without devs touching anything.
Agree with 46 and 47.

Kadir, 17 ESR is still up to date so we can't tell them otherwise. 

If you are talking about the gap from 4.0 to 11 there are 2 groups there that we need to be cautious with: 10 ESR and old systems. My take is that we should nail down the implementation of this alert before targeting those cases. We don't know how this banner converts or annoys users. Let's go with safe groups and then roll it out to the rest of the audience.
Ibai, I know that 17 ESR is up to date, but 17 itself is not. So the question is whether we want to annoy ESR users by making them download an update they can't use or leave 17-19 users on an insecure version without warning them.

As I said in comment #45, if there is no consensus on that, let's just move on with:

FF <= 3.6 or (FF >= 12 and FF <= 16)
Landed:
https://github.com/mozilla/kitsune/commit/cdac73c1d6f40dd096573a469fff3bfe12758a4b

Pushed to prod by me, right now.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Rehan, I can't measure the impact of this since the product landing page sends the most people to the mozilla.org download page, but it has the Firefox download button with the same url as the warning banner. 

Did you have time to check if the #desktop in http://www.mozilla.org/firefox#desktop makes any difference?

If not, could you please change the url of the link here to http://www.mozilla.org/firefox?from=sumo_warning
Kadir, we are tracking all the exiting links as Events. That way we can track these clicks from our profiles instead of the profiles of a different properties.

I'm not sure why we didn't follow the convention proposed in Comment 11.

Creating a new bug to tag that link with those parameters.
Depends on: 878882
Should we begin discussing expanding this to users of Firefox 4-11 and perhaps 18-20?
Some data to keep the conversation going:

Firefox 3.* users convert at a 10% rate while Firefox 12-16 users convert to a rate closer to 5%

Hypothesis... 3.* users don't know they are out of date and they are stuck in an old version while newer Firefox users are not updating even though they know they are out of date.

The rough calculation I've done actually shows how the newer version users have, the worse they convert: The CTR decreases linearly going from almost 6% to almost 4%.

Maybe we should trigger this for 4-11 but not 18-20?
(In reply to Ibai Garcia [:ibai] from comment #56)

Hi Ibai, your finding is interesting: the newer the version, the worse the conversion rate. I have no theory to offer, other than a hypothesis that people who have recently upgraded are more likely to assume that their Firefox is already secure or on the latest version. Either they’re not aware of our rapid-release cycle, or they had downloaded the version that doesn’t have silent update built in.

Another possible theory is that they either:
* Are not informed or possess enough technical knowledge to seek information
* Are informed, but hate/fear change

This theory is truer the older a person’s Firefox version is.

For example, if someone is still running version 3.6, it’s very possible that he’s still running it because he doesn’t know of any newer development. The reason being, if he’s informed enough, he would have probably downloaded an upgrade a while back.

Alternatively, he may be informed but refused to upgrade due to fear of UI change or add-ons breakage.


Another possible group of people that are not upgrading are those who are using a version of Firefox given and set up by the company with the update flag turned of. This Firefox may be kept on the old version due to the company’s internet access policy, for example. I can’t ascertain if these numbers also populate the calculation you have performed.

It might be helpful to set up an IT outreach initiative to talk to companies and help their IT departments deploy the latest LTR.
Ok, so we are still only showing this to users on 3.6 and below, and Firefox 12-16, correct? We should start to expand this (even a low CTR is better than no CTR). I'm proposing the following:

<=16 (ESR 10 is no longer supported)

18-23 (ESR 17 is still supported and ESR 24 is coming online)

We have run several studies of uses and find that roughly 80% of them want to be up to date, but either assume they are already updated automatically (but aren't due to a failure in the updater, user permissions, or something else), or just turned updates off at some point in the past or don't use Firefox enough to have it check for updates.

I haven't seen any evidence this banner annoys users, and honestly, users on very old systems have no right to be annoyed at getting warned they have an out of date and unsupported version of the software when visiting said software's support site. We don't have to roll out to the most recent versions (so we don't have to warn on 22 and above if you'd like) but we should implement a policy of continually updating this list as new versions roll out. If needed we can file a new bug.
Tyler, thanks for highlighting this. Please open another bug and we can discuss there what's the right strategy (i.e. show the banner everyone who is outdated for more than 2 releases, etc).
Blocks: 922742
Reopening this bug:

Checking the traffic generated by this bug I found out that the Event Tracking (discussed in Comment 25) https://bugzilla.mozilla.org/show_bug.cgi?id=651230#c25 was never implemented and we can't track who clicks on that link.

Please, incorporate the event to the link.

Thanks,
Ibai
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
So, it should show up as "External Links" | "Top Banner - Firefox Upgrades" in GA.
The naming was different...and it confused me.

Nothing to see here...keep moving :D
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: