956879 - Crash classification for out-of-date Firefox

Status: RESOLVED FIXED

(Socorro :: Backend, task)

Description

[Benjamin Smedberg]

I discovered that 2.2% of our crashes from last week were from Firefox 12.0, and several thousand more were from even older versions: 4.0, 4.0.1, 3.6.28.

When somebody submits a Firefox crash from an old version, the email response should direct them to upgrade their Firefox, with suitably scary warnings about how they are at risk. For now, this should be anything prior to Firefox 17, which still has some ESR users. We'll bump it up to ESR24 in a few months.

I realize that some users might be using old PPC macs or even windows 98 and might therefore not be able to upgrade, and I'm open to suggestions for how to deal with those sub-populations, but I don't want to delay this on those minor details.

mverdi, is there an existing SUMO article we can use as the text of this email? This will use the same mechanism as bug 931907 support classifications.

Lars can you provide the classifier code? For now this is Firefox-only.

Comment 1

[Verdi [:verdi] Best to slack me]

Here's what I understand:
*Windows XP SP1 & Windows 2000 are stuck on 12
*Mac 10.4 is stuck on 3.6.28
*Mac 10.5 is stuck on 16
*There are people stuck on random versions because something when wrong with an auto-update and they now have to download a new copy in order to update.
*Some people have turned auto-updates off.

We have specific sumo articles for people on old OSs/versions:
https://support.mozilla.org/kb/firefox-36-no-longer-supported
https://support.mozilla.org/kb/firefox-no-longer-works-some-versions-windows-xp
https://support.mozilla.org/kb/firefox-no-longer-works-windows-2000
https://support.mozilla.org/kb/firefox-no-longer-works-mac-os-10-4-or-powerpc
https://support.mozilla.org/kb/firefox-no-longer-works-mac-os-x-10-5

This article has the steps for updating firefox https://support.mozilla.org/kb/update-firefox-latest-version

If you are on Windows and Fx 11 or older and try to update (via about Firefox) you will only update to 12. Then, if your version of Windows supports a newer version we'll auto-update you if you have updates turned on. If not, you're now stuck on 12.

Are you going to link to an article or include the steps? The steps are slightly different for different operating systems. Also, it's possible (maybe likely in these cases?) that the update won't work for the reasons above.

Maybe the best thing is to:
1. Direct people who we know can't update to one of those specific sumo articles
2. Direct others to download a new version via this url https://www.mozilla.org/firefox/all/ (http://www.mozilla.org/en-US/firefox/new/ hides the download button for Fx users).

Comment 2

[K Lars Lohn [:lars] [:klohn]]

I'll gladly write the classifier.  I can trigger it on multiple fields, so that old FF versions where there is no upgrade path can be spared the classification and upgrade nag.  

Question regarding the threshold of "older than 17".  Of course this will be a moving target.  Is there an algorithmic way that I can determine what that threshold should be?  Or is it something we want to just have as a configuration parameter or even hard coded?

proposed rule:
    if FF:
        if FF.version is less than THRESHOLD:
            if (Windows XP SP1 or Windows 2000) and FF.version == 12:
                 skip classification
            else if (Mac OS 10.4 or lower) and FF.version == 3.6.28:
                 skip classification
            else if Mac OS 10.5 and FF.version == 16:
                 skip classification
            else:
                 apply classification 
            
 
Alternatively, I could split this into several classifications where I differentiate between the cases for each of the five support articles listed above.

Comment 3

[Benjamin Smedberg]

Given the information Verdi provided, I think the classifier should be something like:

if FF and version < THRESHOLD:
  if os==mac and (arch==ppc or osversion <= 10.4):
    classify as https://support.mozilla.org/kb/firefox-no-longer-works-mac-os-10-4-or-powerpc
  elif os==mac and osversion == 10.5:
    classify as https://support.mozilla.org/kb/firefox-no-longer-works-mac-os-x-10-5
  elif os==win and osversion == win2k:
    classify as https://support.mozilla.org/en-US/kb/firefox-no-longer-works-windows-2000
  elif os==win and osversion < winxpsp2:
    classify as https://support.mozilla.org/en-US/kb/firefox-no-longer-works-some-versions-windows-xp
  else:
    classify as https://support.mozilla.org/kb/update-firefox-latest-version

We should let people know that they are crashing because they have an old/unsupported version of Firefox even if there is nothing we can really do for them.

As for the question of THRESHOLD, I'd really love to be very specific about it: a week after is unthrottled to everyone, we immediately start showing the email to old users. But that sounds complicated, so for now I'm going with "a few months after the ESR expires". So it will be 17 for now, then 24, and change about twice a year along with the ESR schedule. I don't have an algorithmic way to do it, so a config parameter makes sense to me.

Comment 4

[K Lars Lohn [:lars] [:klohn]]

in your pseudo-code in Comment #3, do you want me to take it literally and use the complete URL as the classification?

Comment 5

[Benjamin Smedberg]

No, it should classify to some intermediate string (you can make it up). Basically for all of these support classifiers there are going to be two separate steps:

* classify to some programmatic string
* map programmatic strings to actions that various subsystems should take: the email subsystem right now, and in the future the builtin Firefox support subsystem

Comment 6

[K Lars Lohn [:lars] [:klohn]]

Attachment: PR for out-of-date classifier

@bsmedberg - you may do an exhaustive review if you wish or a general one if your timing is tight.  In case of a general one, toss it back to me and I'll get one of the others to do the nit picky details.

Comment 7

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/3028a4e52175527d81ec250c9266eac163e1a302
Merge pull request #1839 from twobraids/out-of-date

fixes Bug 956879 - added out-of-date support classifier