1170331 - Constantly losing smtp password

Status: RESOLVED DUPLICATE of bug 441889

(Thunderbird :: Security, defect)

Description

[Todd]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Firefox/38.0
Build ID: 20150513174244

Steps to reproduce:

th 31.7.0
Windows 7 pro 64 bit

Dear Thunderbird developers,

Would one of our intrepid heroes please fix this for me.  It is driving my customer AROUND THE BEND!

I have a customer with five computers that use MAPI and Thunderbird sending eMail to its customers from a program called Domico 2000 (I can duplicate without this program).   These eMails are *critical* to the business.

Problem: it randomly asks the operator for their smtp password:
   "Enter your password for [e-mail] on smtpout.secureserver.net
(This is Go Daddy's eMail server.)

When I check "saved Passwords", the thing is empty.

I can duplicate the issue by right clicking on a file on the desktop and selecting Sent To, Mail Precipitant (MAPI)

POP3 is mis-configured purposefully so they will not receive email.  And told not the get mail on a timer or on start up.

Many thanks
-T

Comment 1

[Todd]

Just got a report from another customer report of this.  She is using XP-Pro-SP3 and iMap with zoho mail and no MAPI (just eMail)

Comment 2

[Todd]

Hi All,

I am moving this up to critical as it is costing my customer money (their customers are not receiving their mailings).

This morning, one of the MAPI customers I had erased Keys on was being prompted for her password, again.  On investigation, her "saved passwords" was empty.  Re-entering the password repopulated "saved passwords".  Exited and restarted Thunderbird and it stayed.  It will now stay for a day or two.

On the second customer with imap and no MAPI, her "saved passwords" should have had three imap and three smtp saved passwords.  One of her smtp's was missing.

Please fix this quickly!

Many thanks,
-T

Comment 3

[Todd]

Another one of the IMAP facilities just called in.

I went into all five facilities and re-erased key3.db and erased signons.sqlite.  Then restarted Thunderbird and re-enter the password.

Comment 4

[Kent James (:rkent)]

I understand that this is a serious issue for you, but whatever you are seeing is not being reported elsewhere.

It appears to me your issue is related to sending using MAPI. ("I can duplicate the issue by right clicking on a file on the desktop and selecting Sent To, Mail Precipitant (MAPI)")  It has been years since any MAPI issues have been fixed in Thunderbird, this is ancient code that very few current developers have any experience with. My main recommendation would be that you do not encourage your users to rely on MAPI for mission critical functions.

I realize that is not what you want to hear. Looking at the history of MAPI bugs, that last fix of any consequence was done by Neil several years ago. He is still an active participant in the Mozilla Mailnews area. I've cc'd him on this.

Perhaps he can respond. But if this is as critical to you as you say, my recommendation to you would be that you try to hire Neil as a consultant to examine this issue in your case. This is my personal recommendation, not anything official from Mozilla or the Thunderbird team.

Comment 5

[Todd]

(In reply to Kent James (:rkent) from comment #4)
> I understand that this is a serious issue for you, but whatever you are
> seeing is not being reported elsewhere.

Maybe not to you.  But other are complaining about it (Jan 2015):
http://forums-test.mozillazine.org/viewtopic.php?f=39&t=2902343

This is where I got the idea to erased key3.db and signons.sqlite

Here is an older one from Linux (no MAPI here):
http://askubuntu.com/questions/172193/how-do-i-stop-thunderbird-from-forgetting-my-passwords

And I found a few other with DuckDuckGo

> 
> It appears to me your issue is related to sending using MAPI. ("I can
> duplicate the issue by right clicking on a file on the desktop and selecting
> Sent To, Mail Precipitant (MAPI)")  It has been years since any MAPI issues
> have been fixed in Thunderbird, this is ancient code that very few current
> developers have any experience with. My main recommendation would be that
> you do not encourage your users to rely on MAPI for mission critical
> functions.

It happens more often with MAPI, but if you read my second post, it is also happening where no MAPI is involved.

By the way, the customer has no choice but to use MAPI.  I have asked the vendor several times to please do it themselves, but they ignore me.  They ignore me when I tag them for security issues too.  It is what it is.  (One of the reasons why I love Open Source -- you ACTUALLY fix things.)

If the customer goes back to using Outlook, then they introduce all kinds of security issues.  Not to mention they lose a bunch of functionality.   And they lose access to you guys -- please don't under estimate the importance of the service you provide.

> I realize that is not what you want to hear. Looking at the history of MAPI
> bugs, that last fix of any consequence was done by Neil several years ago.
> He is still an active participant in the Mozilla Mailnews area. I've cc'd
> him on this.
> 
> Perhaps he can respond. But if this is as critical to you as you say, my
> recommendation to you would be that you try to hire Neil as a consultant to
> examine this issue in your case. This is my personal recommendation, not
> anything official from Mozilla or the Thunderbird team.

I would appreciate you letting me know if he responds.

Many thanks,
-T

Comment 6

[Matt]

Lets look at this objectively.

MAPI is no longer included in Windows.  The download from Microsoft does not work with Windows 8 See  https://www.microsoft.com/en-us/download/details.aspx?id=1004

So we are talking about a legacy protocol that will not run only on Windows and not on windows 8 or 10.

Given the process that I see occurring in the MAPI connection.  I see no correlation between creating a mail (and sending a mail)using mapi and the password manager.  Conceivable the store could become corrupt.  But as the MozillaZine thread suggests.  Deleting the store will fix that.

I think the issue may well be external to Thunderbird.  Located perhaps in an anti virus program or suite.  Norton's has it's password vault which is know to have issues with Thunderbird on occasion.  People change their password and never get prompted for a new one.
What anti virus program is in use.  
Is it set to scan mail?  What if it is turned off?  
Does the problem occur in windows safe mode?  Thunderbird safe mode? Both?

Comment 7

[Todd]

Attachment: Send to in Windows 10

My Bad.  I got MAPI mixed up with something else.  I have a support ticket into the vendor to find out how exactly he does the call.  I believe they are using the same mechanism ad "Sendto:" but will find out shortly.

Comment 8

[Todd]

(In reply to Matt from comment #6)
> Lets look at this objectively.
> 
> MAPI is no longer included in Windows.  The download from Microsoft does not
> work with Windows 8 See 
> https://www.microsoft.com/en-us/download/details.aspx?id=1004
> 
> So we are talking about a legacy protocol that will not run only on Windows
> and not on windows 8 or 10.

My Bad.  They are not using MAPI.  I will get back when they get back to me.


> 
> Given the process that I see occurring in the MAPI connection.  I see no
> correlation between creating a mail (and sending a mail)using mapi and the
> password manager.  Conceivable the store could become corrupt.  But as the
> MozillaZine thread suggests.  Deleting the store will fix that.
> 
> I think the issue may well be external to Thunderbird. 

Happens in Windows, Ubuntu, Scientific Linux 6.x.  (My base OS, and yes it does happen to me occasionally, but it doesn't bother me, so I haven't complained about it.  I know my passwords by heart.)

> Located perhaps in
> an anti virus program or suite.  
> Norton's has it's password vault which is
> know to have issues with Thunderbird on occasion.  People change their
> password and never get prompted for a new one.
> What anti virus program is in use.  

I sell Kaspersky, so most all my Windows customer are on various versions.  The two customers complaining here are on Kaspersky End Point Security kes10winsp1_en_aes56-10.2.2.10535.exe

I have so many customer on Thunderbird and Kaspersky, if is was an issue, I would get my ears ringed every 10 minutes or so.  And it doesn't explain Ubuntu and Scientific Linux


> Is it set to scan mail?  

Not outbound

> What if it is turned off?  

Since this takes several days to occur and it is a high security situation, this is something that can not be tested.

> Does the problem occur in windows safe mode?

The calling program doesn't work in safe mode.

> Thunderbird safe mode? Both?

Haven't tried TH in safe mode.  This would be pretty much impossible to test, as I have no control over the workstations in this respect.

Also, Ubuntu and Scientific Linux don't have anti viruses running on them and have a rescue but not a safe mode.

What I think is going on is that one or both of the databases are being corrupted.  I think what would fix this would be a journal so that a corrupted database could be corrected on the fly.

I will get back on how the vendor calls me back on the calling method.

Also, if it helps, the sendto customer usually notices this when they are doing their close outs of their database and all their daily mail gets sent.  Perhaps, Thunderbird's password databases are not too tolerant of having to share a busy CPU?

Comment 9

[Todd]

Hi Guys,

They are actually using "Simple MAPI".  Sort of:

http://en.wikipedia.org/wiki/Messaging_Application_Programming_Interface
     "Simple MAPI is a subset of 12 functions which enable developers to add
     basic messaging functionality"
And it is still active in all versions of Windows.

They are using the "sendmail.dll":
http://dll.paretologic.com/detail.php/sendmail

But this doesn't explain Ubuntu or Scientific Linux.  

I still think that the passwords database are being corrupted by a busy CPU.

Comment 10

[Matt]

I have at this point only the most cursory look at sendmail  but it looks like an end to end mailer that connects directly to the SMTP server... the mail client should not be involved.

However,  try creating an exception in Kaspersky for the Thunderbird profile folder.  Allow scheduled scans.  Hopefully when Thunderbird is closed.  But disable any real time scanning.  Also ensure that the option Tools menu (alt+T) > options > Security > anti virus is turned on. (this ensures incoming mail is written to a temp file on arrival so the anti virus can scan it.  So the need to scan the mail store is far less.

The reality is it is your windows users that have a problem.  Not Linux users as the calling software is windows. Linux users might have an issue.  But similar symptoms does not mean the same problem.  What is the calling software?  I have knocked around retail and legal systems for a long time and am more than familiar with dodgy code from expensive line of business applications.  I am also more than familiar with their often total ignorance of the need to system security.

Thunderbird has files that often measure in the Gb range and a scan takes around 10 minutes.  Anti virus scanning as they are updated can cause all sorts of timing and contention issues that result in totally bizarre outcomes, so preventing on access scanning is a good idea regardless of any obvious issues..

Comment 11

[Todd]

(In reply to Matt from comment #10)

I have countless Thunderbirds installed at customer sites across two counties.

> I have at this point only the most cursory look at sendmail  but it looks
> like an end to end mailer that connects directly to the SMTP server... the
> mail client should not be involved.

It connects to whatever the registry points at.  Check out
   HKEY_CLASSES_ROOT\.mapimail
   and
   HKEY_CLASSES_ROOT\CLSID\{9E56BE60-C50F-11CF-9A2C-00A0C90A90CE}

I am not sure this is the say they are doing it anyway, as sendmail.dll does seem to be stand alone.  Anyway, they really do want Thunderbird, as they live off their Sent folder.

> 
> However,  try creating an exception in Kaspersky for the Thunderbird profile
> folder.  Allow scheduled scans.  Hopefully when Thunderbird is closed.  But
> disable any real time scanning.  Also ensure that the option Tools menu
> (alt+T) > options > Security > anti virus is turned on. (this ensures
> incoming mail is written to a temp file on arrival so the anti virus can
> scan it.  So the need to scan the mail store is far less.
> 
> The reality is it is your windows users that have a problem.  Not Linux
> users as the calling software is windows. Linux users might have an issue. 

One customer with five computers and sendmail is using Windows 7

The other is using XP and iMap.  She does not use Sendmail.  (And, I am not sure the first customer is either at this point.)

The symptom is exactly the same between Linux and Windows.  The Show Passwords entry is missing.

> But similar symptoms does not mean the same problem.

When the symptom is exactly the same, it is rare that it is not the same problem.  Although I have seen some pretty bizarre stuff.  (I only get called in when the customer gives up, so bizarre is my world.)

>  What is the calling
> software? 

Will answer that one off line

> I have knocked around retail and legal systems for a long time
> and am more than familiar with dodgy code from expensive line of business
> applications.  I am also more than familiar with their often total ignorance
> of the need to system security.

Sometimes pay software is impossible to get bugs fixed.  Open Source can be similarly frustrating at times too (Open Office, Wine).  Fortunately, Mozilla is very responsive.
 
> Thunderbird has files that often measure in the Gb range and a scan takes
> around 10 minutes.  Anti virus scanning as they are updated can cause all
> sorts of timing and contention issues that result in totally bizarre
> outcomes, so preventing on access scanning is a good idea regardless of any
> obvious issues..

The AV is now scanning outgoing eMail.  I don't see where excluding the profile would have any effect.  But ...  (And it won't help the Linux side.)

No one yet has complained after I re-erased key3.db and erased signons.sqlite.  Just erasing key3.db did not keep the problem from coming back.

I still think some kind of journal is called for on the passwords databases.  

When this happens to a lot of my customers, "some" on them throw a fit over not having a password.  "I have never had to enter a password".  I tell them nicely that they did once and they pressed the Remember button.   So, this is a frequent occurrence.  The two customers I am reporting this to your are only the ones that have tasked me to figure out why this keeps happening.

Comment 12

[Todd]

Extra information:

1) On the customer with five computers, there have been several instances when they get prompted for the SMTP password, enter it, check off remember, send the eMail successfully, and have to repeat the process every new eMail they send.  In other words, the password won't save.  Erasing key3.db and signons.sqlite "seems" (watch the "weasel" word) to fix it.

2) I have just sent an eMail to the customer with five computers if Thunderbird is running when they do their morning reboots.  (They leave their computers running all night so they can communicate with a web server.)

Comment 13

[Wayne Mery (:wsmwk)]

What is the short, 3 sentence version of what this is, current status, and whether it is likely, or not likely, to be a problem fixable in Thunderbird?

Comment 14

[Todd]

Since re-erased key3.db and erased signons.sqlite, the problem has not come back.  I do believe this was caused by a corruption from an upgrade.

Comment 15

[Wayne Mery (:wsmwk)]

(In reply to Todd from comment #14)
> Since re-erased key3.db and erased signons.sqlite, the problem has not come
> back.  I do believe this was caused by a corruption from an upgrade.

perhaps an example of bug 441889

Comment 16

[Todd]

IT'S BACK !!!!

Yesterday the customer (Thunderbird 38.3.0 on Windows 7 Pro x64, SP1) called me with her SMTP password being prompted for again.  With Thunderbird off, I removed key3.db and signons.sqlite.  Then reentered her password with "remember".  It worked for a "day".  Now it is back.  (signons.sqlite was not recreated.)

This is driving my poor customer crazy at five different locations.

Comment 17

[Todd]

And this day, the fix only worked for an hour.

Would this have anything to do with POP/IMAP being turned off (do not download)?

Comment 18

[Javi Rueda :javirid]

(In reply to Todd from comment #16)
> Yesterday the customer (Thunderbird 38.3.0 on Windows 7 Pro x64, SP1) called
> me with her SMTP password being prompted for again.  With Thunderbird off, I
> removed key3.db and signons.sqlite.  Then reentered her password with
> "remember".  It worked for a "day".  Now it is back.  (signons.sqlite was
> not recreated.)

Hi Todd. I am very sorry about the problem your customers are experiencing with Thunderbird.

I am very worried that, since Mozilla 32, the correct file to be removed was changed from signons.sqlite to logins.json. See my own bug 441889 comment 28.

I really hope that helps in solving the problem.

However, I think this bug is not a duplicate of that. I have not found any reference to "user changed his/her password on server" here.

Comment 19

[Todd]

Thank you.  So far key3.db and logins.json is holding.

Comment 20

[Peter Langston]

This has been happening to me, too.

Specifically:
    I used to enter my email password once upon invoking Thunderbird and then Thunderbird would accept new incoming email and send outgoing email using that password until I Quit Thunderbird.  When I next ran Thunderbird I would have to enter my password again.  This is perfect behavior.
    Now I have to enter my password when I launch Thunderbird AND every time I send a piece of email.  This sucks the mop.
    I know that I can let Thunderbird save my password and then anyone can run Thunderbird on my machine and read my incoming email without a password.  NOT an acceptable work-around!

I'm running Thunderbird 38.3.0 on a MacBook Pro running OS x 10.10.5

Best,
    Peter

Comment 21

[Todd]

(In reply to Peter Langston from comment #20)
> This has been happening to me, too.
> 
> Specifically:
>     I used to enter my email password once upon invoking Thunderbird and
> then Thunderbird would accept new incoming email and send outgoing email
> using that password until I Quit Thunderbird.  When I next ran Thunderbird I
> would have to enter my password again.  This is perfect behavior.
>     Now I have to enter my password when I launch Thunderbird AND every time
> I send a piece of email.  This sucks the mop.
>     I know that I can let Thunderbird save my password and then anyone can
> run Thunderbird on my machine and read my incoming email without a password.
> NOT an acceptable work-around!
> 
> I'm running Thunderbird 38.3.0 on a MacBook Pro running OS x 10.10.5
> 
> Best,
>     Peter


Hi Peter,

This my notes on how to work around the issue:

1) exit Thunderbird

2) Remove key3.db and logins.json from the user's Thunderbird profile directory

3) Restart Thunderbird and re-enter passwords

4) optional: exit Thunderbird and set key3.db and logins.json to Read Only

I have had great luck with #4 above.

-T