1246269 - Old Thunderbird versions are not updating. Help | About | Check for updates returns "Thunderbird is up to date". Log shows "Server certificate has expired"

Status: NEW

(Thunderbird :: General, defect)

Description

[Wayne Mery (:wsmwk)]

rsx11m alerted me to a version 9 user who was unable to update. Yesterday I installed 9.0.1, 11.0.1, 12.0.1 - all report "Thunderbird is up to date". Rail reports the update rules are good - >3.6.13 and <10.0 should update to version 10, and thence to version 12.

My install of 12.0.1 shows...

prefs
app.update.url https://aus2.mozillamessaging.com/update/3/%PRODUCT%/%VERSION%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml
app.update.url.details http://www.mozillamessaging.com/%LOCALE%/%APP%/releases/

log
AUS:SVC UpdateManager:_loadXMLFileIntoArray: XML file does not exist
AUS:SVC getLocale - getting locale from file: resource://app/update.locale, locale: en-US
AUS:SVC Checker:getUpdateURL - update URL: https://aus2.mozillamessaging.com/update/3/Thunderbird/12.0.1/20120428123112/WINNT_x86-msvc/en-US/release/Windows_NT%206.1.1.0%20(x64)/default/default/update.xml?force=1
AUS:SVC gCanCheckForUpdates - able to check for updates
AUS:SVC Checker:checkForUpdates - sending request to: https://aus2.mozillamessaging.com/update/3/Thunderbird/12.0.1/20120428123112/WINNT_x86-msvc/en-US/release/Windows_NT%206.1.1.0%20(x64)/default/default/update.xml?force=1
AUS:SVC Checker:onError - request.status: 2153390069
AUS:SVC getStatusTextFromCode - transfer error: Server certificate has expired (please adjust your system clock to the correct date and time if it is incorrect), code: 2153390069
AUS:SVC UpdateService:removeDownloadListener - no downloader!

Comment 1

[rsx11m]

aus2.mozillamessaging.com:443 - The certificate expired on 02/09/2015 15:55:26 GMT.
Nevertheless, we should get some useful error message rather than no updates available.

Comment 2

[Kent James (:rkent)]

There should be reasonable limits to how far back we support product, even updates. For code, we don't consider compatibility with systems that old.

Comment 3

[rsx11m]

Fair enough. I'm only wondering if the current version would behave identical (i.e., just claim "no updates" despite actually running into an error with the certificates), and whether or not the certificate on aus2.mozillamessaging.com:443 (independently from the version which tries to access it) needs to be renewed to avoid such incidences in the future (if it's still used).

Comment 4

[Kent James (:rkent)]

You could argue that a status of "no updates found" should not be reported to the user as "Thunderbird is up to date" but perhaps "No updates found for Thunderbird". That wording copies Firefox though. I don't think that you should have error reporting by default though, as there will be too many times when there are errors, and we usually just want to ignore them and try again later.

Regarding the certificates, I vaguely recall some IRC discussion about whether these old update servers were still in use, and whether their certificates should be renewed. I may have recommended that we not bother to update the old certificates.

Comment 5

[Wayne Mery (:wsmwk)]

So this is partly a result of bug 1151131. I would have liked to see more input there before coming to a decision considering no metrics were discussed. I'll do some checking.

Comment 6

[:Atoll]

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #5)
> So this is partly a result of bug 1151131. I would have liked to see more
> input there before coming to a decision considering no metrics were
> discussed. I'll do some checking.

The certificate expired in February 2015 when we couldn't find anyone to claim ownership of the site. We only received final approval (based on internal discussions with AUS and Thunderbird folks) to decom the aus/aus2.mome sites in December 2015. We then took no action whatsoever, because we've seen up to 12 weeks between responses to questions when discussing these sites. So, we'll continue to hold, but please advise us within 2-4 weeks on how to proceed :)

(Setting needinfo for :wsmwk as current torch bearer for this issue, please do not clear without an actionable plan or a next-step person who will be responsible for an actionable plan. We've been trying to get a decision here for years and too many people have commented and then refused to participate.)

Comment 7

[Kent James (:rkent)]

Per bug 1151131 comment 7 aus2 was last used in Thunderbird version 9, so 10 and later still work. I really don't think it is realistic to expect Mozilla to maintain 4 year old infrastructure to support people who are not updating.

Comment 8

[Wayne Mery (:wsmwk)]

(In reply to Richard Soderberg [:atoll] from comment #6)
> (In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #5)
> > So this is partly a result of bug 1151131. I would have liked to see more
> > input there before coming to a decision considering no metrics were
> > discussed. I'll do some checking.
> 
> The certificate expired in February 2015 when we couldn't find anyone to
> claim ownership of the site. We only received final approval (based on
> internal discussions with AUS and Thunderbird folks) to decom the
> aus/aus2.mome sites in December 2015 [bug 1151131]. We then took no action whatsoever,
> because we've seen up to 12 weeks between responses to questions when
> discussing these sites. So, we'll continue to hold, but please advise us
> within 2-4 weeks on how to proceed :)
> 
> (Setting needinfo for :wsmwk as current torch bearer for this issue, please
> do not clear without an actionable plan or a next-step person who will be
> responsible for an actionable plan. We've been trying to get a decision here
> for years and too many people have commented and then refused to
> participate.)

Sorry for the difficulties, and thanks for the patience. Please know I'm not attempting to apportion blame. 

I'm new to this topic, so I'm green compared to Kent. I don't pretend to speak for all of Thunderbird, just offering my opinions - perhaps strong on some points.  Perhaps I should comment in bug 1151131 but for now I'll stay here...


(In reply to Kent James (:rkent) from comment #7)
> Per bug 1151131 comment 7 aus2 was last used in Thunderbird version 9, so 10
> and later still work. I really don't think it is realistic to expect Mozilla
> to maintain 4 year old infrastructure to support people who are not updating.

My concerns about the infrastructure hinge somewhat on Kent's point above.  I suspect the implied simplicity of "people who are not updating" is actually not simple.  Of course we know some reasons users *choose* to stay on an old release - restrictions on old OS[1], addon compat, displeasure with changes to look and feel, bugs in new version, etc.  Beyond that, I suspect we have a poor understanding of users' situations where they have *not* intentionally chosen to stay on an old release, or wouldn't stay there if they new better - as a result of update bugs, firewalls/proxies, user been given bad advice, poor corporate dictates, etc.  We don't know what percentages of our population is in which categories - we lack data. [2] and [3] provide some historical data references.  I no data so far for present day.

On principal, I dislike the idea of stranding a large number of users who don't know better. I also agree we should not needlessly keep old infrastracture.  But can't we do better than only killing the old AUS (if that's what we decide to do)?  So I'm wondering what do are numbers look like, are the numbers worth the effort, (a broader question) what do we want our general user experience to be in this area, and are the considerations if our future is outside Mozilla?

Whittling it down for now to two questions (even though there are more):

1. Who can tell us what Firefox's approach and philosophy has been over the past few years with regard to retiring old AUS, and to what extent users can get stranded as a result?  

2. On the general subject of users on old releases - I've actually been mulling it over for a few months because I noticed we have a high percentage of users on very old beta releases (I think it was around 20%). So my question is, should we inform users with an appeal like Firefox had done by telling users via start page or some other mechanism "Your Thunderbird is out of date"?  The majority might actually be appreciative.  (http://mzl.la/1VBEI75 is a sampling of bugs showing what Firefox has done, and not done, in the past few years) 

(I'm intentionally leaving my NI active)


[1] OS example Mac OS X 10.5 is no longer supported in TB17

[2] https://bugzilla.mozilla.org/show_bug.cgi?id=754773#c0 has data of what Firefox found with regard to Firefox betas - 83% did NOT know they were on an oudated release!

[3] I did a small sample of Kent's 2014 ADI data. This is by no means a statistical study - just some reference data. I looked at EN,DE,JP locales: 
* 2013-08-19 - Version 17 had been out for one month, and we were entering the ESR era. ~87% on v17, ~4.5% on v3.1 and v3.0, ~2% on v2, ~1.5% still on the immediate two prior releases v15 and v16, 0.5% on *future* betas
* 2012-08-18 - version 14 had been out for about one month, and were were in the fast release era. Similar statistics for older v3* and v2 versions and beta. v14 at 70% and the prior 4 versions totalling about 19%

Comment 9

[Matt]

I think we need to do something to move these people.  Eventually they do decide to update and it is difficult at times.  Due to code for conversions of profile data having been removed or just because it is a huge jump,  they often appear in Support forums.

What I see now are folk using windows restore points to back out of an update that caused them difficulties and staying there.

Comment 10

[Kent James (:rkent)]

Attachment: ThunderbirdPre10.png

Here's data on Thunderbird usage for versions 3 - 9. Total usage is about 160K ADI peak, compared to 10,000K ADI peak for all of Thunderbird. So this represents about 1.6% of users.

It would be good if we had some sort of official policy on what versions we still support update from. If I had to guess, I wouls say that today update from Thunderbird 17 or later should be OK from a code perspective. Even version 10 would make me nervous.

If someone wanted to take this on, what would make sense would be a support note for updating old versions, that would recommend users update manually first to the last 10.0 version, then to Thunderbird 17, then to later versions. That concept needs testing. But someone would have to care enough to put some effort into this. I don't care enough. I think that we should support update now from 17.0, but once 45.0 is released we should drop all aupport of 17 or older, including updates.

Comment 11

[Matt]

  Perhaps the idea might be to place an unsupported warning on https://live.mozillamessaging.com/thunderbird/start for versions prior to say 17.

BTW in support we generally recommend.

2.XX (yes there are still folks using V1 and before .8 is the earliest I have seen in support in the last year.)
3.0 or 3.1 to pick up the conversion of the password store
TB 11 to pickup the tabs on top conversions. 
TB 34 for the conversion of localstore.RDF to XULStore.json.

I do not know if these are exhaustive, or all necessary,  but I have not yet had a failure,  other than the  < 3 to 3 passwords lost issue,  and that occurred at the time as an upgrade bug anyway.

Comment 12

[Magnus Melin [:mkmelin]]

Matt, those migrations happen even if you don't do upgrades step-by-step.

I think the best plan would be to make the start page show an intrusive warning with instructions (i.e. download link) on how to upgrade, in case your installation is really old.

Comment 13

[Wayne Mery (:wsmwk)]

(In reply to Wayne Mery (:wsmwk) from comment #8)
> (In reply to Richard Soderberg [:atoll] from comment #6)
> > (In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #5)
> > > So this is partly a result of bug 1151131. I would have liked to see more
> > > input there before coming to a decision considering no metrics were
> > > discussed. I'll do some checking.
> > 
> > The certificate expired in February 2015 when we couldn't find anyone to
> > claim ownership of the site. We only received final approval (based on
> > internal discussions with AUS and Thunderbird folks) to decom the
> > aus/aus2.mome sites in December 2015 [bug 1151131]. We then took no action whatsoever,
> > because we've seen up to 12 weeks between responses to questions when
> > discussing these sites. So, we'll continue to hold, but please advise us
> > within 2-4 weeks on how to proceed :)
> ...
> 1. Who can tell us what Firefox's approach and philosophy has been over the
> past few years with regard to retiring old AUS, and to what extent users can
> get stranded as a result?  

atoll ^

Comment 14

[:Atoll]

I couldn't say, sorry. That's a question for the owner of Firefox Product Updates (whatever it's formally called, anyways). I would suggest discussing that with rkent and whomever runs the updates servers these days.

Comment 15

[Rob Lemley [:rjl]]

As of February 2020, Thunderbird < 36 is unable to update. See https://mozilla-balrog.readthedocs.io/en/latest/client_domains.html and bug 1612595.
Short version is there's some certificate pinning requirements involving a SHA-1 certificate from Thawte and it's not possible to renew or anything.