508381 - Multiple failed login attempts when IMAP password is changed, even after clear of saved password

Status: RESOLVED FIXED

(MailNews Core :: Networking: IMAP, defect)

Description

[Mahesh Asolkar]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2a1pre) Gecko/20090804 Minefield/3.6a1pre
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.3pre) Gecko/20090803 Lightning/1.0pre Shredder/3.0b4pre

Today, after changing my email password in MS exchange, I spent a few hours with IT because Thunderbird was sending saved password (old password) multiple times.

I had cleared the 'Saved Passwords', but 'Check for new messages at startup' was checked. So may be Thunderbird was sending empty (or worse cached) passwords (?)

The result was that there were a bunch of failed login attempts, which caused my account to be locked repeatedly (every time I started Thunderbird).

Reproducible: Always

Steps to Reproduce:
1. On an IMAP system that locks your account after a few incorrect passwords, create an account with 'Check for new messages at startup' checked.
2. Change the password on the email system
3. Start Thunderbird
4. Try the same after clearing the 'Saved Passwords'
Actual Results:  
Account was getting locked due to multiple failed login attempts

Expected Results:  
1. If 'Saved Passwords' are cleared and 'Check for new messages at startup' is checked, Thunderbird should not attempt any logins until a password is entered in the password prompt (which, by the way, appears as expected).
2. If one login attempt using 'Saved Password' fails, Thunderbird should not attempt any logins until a password is entered in the password prompt. 

For lack of understanding of inner workings of Thunderbird (and IMAP) most of what I said above may be inaccurate but I am just trying to read the symptoms.

Eventually, I had to un-check 'Check for new messages at startup' and start Thunderbird. This kept Thunderbird from logging in to my Exchange server before I had a chance to update my password in Thunderbird.

Comment 1

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #0)
> Today, after changing my email password in MS exchange, (snip)

(Q1) When you(or administrator) changed password?
     (a) While your Thunderbird is not active.
     (b) While your Thunderbird is active and is logged in using old password.

> I had cleared the 'Saved Passwords', (snip)

(Q2) If (Q1)=(b), did you restart Tb after clear the 'Saved Passwords'?

(Q3) Do you enable 'Check for new messages every NNN minutes'?

(Q4) Does you Exchange server supports IDLE command?
     Do you enable "Use IDLE command if server supports it"? 

Possibly side effect of some changes in bug 422814 in order to avoid frequently reported issue such as bug 423354.
Before that change, password was cleared in order to avoid problem like yours, then phenomenon like bug 423354 frequently reported, thus change was made.

Comment 2

[Ludovic Hirlimann [:Usul]]

Nikolay can you confirm ?

Comment 3

[Mahesh Asolkar]

(In reply to comment #1)
> (In reply to comment #0)
> > Today, after changing my email password in MS exchange, (snip)
> 
> (Q1) When you(or administrator) changed password?
>      (a) While your Thunderbird is not active.
>      (b) While your Thunderbird is active and is logged in using old password.

We use a tool to change password. It usually takes up to an hour for the password to propagate to all unix/Exchange systems. I had Thunderbird running when I used the password tool. But I closed in after some time. I only noticed the reported behavior the next morning when I started Thunderbird. I am not certain if the new password had taken effect in the Exchange server while Thunderbird was running with the old password.

But when I started Thunderbird next morning, I had not cleared 'Saved Passwords'. So Thunderbird did start with old password, when the server had the new password.

I cleared 'Saved Passwords' and restarted Thunderbird during the multiple iterations of locking/unlocking account that I went through with the IT.

> 
> > I had cleared the 'Saved Passwords', (snip)
> 
> (Q2) If (Q1)=(b), did you restart Tb after clear the 'Saved Passwords'?
> 

Yes. (Also more above)

> (Q3) Do you enable 'Check for new messages every NNN minutes'?
> 

Yes.

> (Q4) Does you Exchange server supports IDLE command?
>      Do you enable "Use IDLE command if server supports it"? 
> 

Yes (to the best of my knowledge). And I do have "Use IDLE command if server supports it" enabled.

Comment 4

[Nikolay Shopik]

>We use a tool to change password. It usually takes up to an hour for the
>password to propagate to all unix/Exchange systems
While this little off-topic of this bug, but why you don't use Kerberos? It's seems fit more better here.

Ludo, no I can't confirm that yet

Comment 5

[Mahesh Asolkar]

(In reply to comment #4)
> >We use a tool to change password. It usually takes up to an hour for the
> >password to propagate to all unix/Exchange systems
> While this little off-topic of this bug, but why you don't use Kerberos? It's
> seems fit more better here.

IT/administrators alone know. I am a mere user.

Comment 6

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #3)
> We use a tool to change password. It usually takes up to an hour for the
> password to propagate to all unix/Exchange systems. I had Thunderbird running
> when I used the password tool. But I closed in after some time. I only noticed
> the reported behavior the next morning when I started Thunderbird. I am not
> certain if the new password had taken effect in the Exchange server while
> Thunderbird was running with the old password.

Question about next part.
> I only noticed the reported behavior the next morning when I started Thunderbird.

If password is changed at server and if your account is not locked at the next morning, first login fails because of old password Tb knows. In this case, password is prompted, and if you enter new password properly, login should successful.

What happened in the next morning was following, wasn't it? (I'm suspecting 1-a.)
  1-a) Your account was already locked at the next morning.
  1-b) You typed wrong password multiple times at the next morning,
       then account was locked.
  2) Because account was already locked, "clear password", restart of Tb after
     "clear password" won't resolve problem until unlock of your account. 

Possible reason of 1-a) which I'm suspecting.
 - While you were using Tb with old password, new password arrived at server.
 - When next mail check by 'Check for new messages every NNN minutes', login
   failed due to old password, and Tb silently terminated the mail checking.
 - Mail check by 'Check for new messages every NNN minutes' was executed multiple
   times using old password, because current Tb doesn't clear password in such
   situation(this is the change I said in previous comment).
 - Due to repeated attempt of login with old password, server locked your account.

Did you see new mail arrival around 5:00 PM?

Comment 7

[Mahesh Asolkar]

1-b did not happen.

1-a happened once. The first time next morning, and I think your reasoning for 1-a may be accurate. But after that, I sat with IT and did multiple iterations where I would have them unlock the account. Wait for a while. Then start Thunderbird (with 'Check for new messages at startup' checked and 'Saved Passwords' cleared) and just watch. Immediately (within seconds, without even entering anything in the password prompt) the account would see multiple authentication attempts. They would all fail and the account would lock.

I have multiple folders in my IMAP account - about 20 I subscribe to. I have mail.check_all_imap_folders_for_new set to true.

When 'Check for new messages at startup' was checked and 'Saved Passwords' cleared, the behavior was almost like multiple threads started authentication attempts. While one waited for the password prompt to return, rest went on with bad password and made the authentication requests anyway.

Again, last paragraph is just my interpretation of the symptoms, may be way off.

What's so unique about 5:00PM?

Comment 8

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #7)
> What's so unique about 5:00PM?

I imaged 9:00 AM to 5:00 PM worker, and I assumed people usually checks not-processed-mails before end of daily Job. If you didn't see new mail at 4:00 PM but you saw new mail at 5:00 PM just before terminate Tb, problem like "unlock of account due to Tb's behaviour" should be ruled out from phenomenon of comment #0.
That was question for it.

> Then start Thunderbird (with 'Check for new messages at startup' checked and 'Saved Passwords' cleared) and just watch.

Procedure was next?
  A-1. Clear password, then terminate Tb
  A-2. Restart Tb with 'Check for new messages at startup' checked.
Or next?
  B-1. Terminate Tb
  B-2. Restart Tb with 'Check for new messages at startup' checked,
       and cleared saved password.

Comment 9

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #7)
> Immediately (within seconds, without even entering anything in the password prompt) the account would see multiple authentication attempts.
> They would all fail and the account would lock.
> 
> I have multiple folders in my IMAP account - about 20 I subscribe to.
> I have mail.check_all_imap_folders_for_new set to true.

Your interpretation was correct.
  - Check for new messages at startup = checked
    Three IMAP folders : Check for new messages at startup = checked
    Password is saved.
  - Tb logged-in to Gmail IMAP server via three connections concurrently
    just after restart.
  - When password was change at server (change via Gmail Web Interface),
    all concurrent attempt of login failed.

If "number of mail folders to be checked for new mail at startup" exceeds server side limit of "contiguous invalid login attempts", account lock may happen.
If number of folders is not so many, and if server won't lock account, operation to use new password was very simple.
  Multiple logins fail due to wrong password
  => A dialog of Retry/New Password/Cancel => Reply "New Password"
  => password dialog appears => Enter new password => login is executed
  => Second of Retry/New Password/Cancel => Reply "Retry" => login is executed 
     
I think next recovery procedure is required, if account lock happned.
(A)
  1. Clear saved password, and terminate Tb
  2. Unlock account
  3. Restart Tb, and enter proper password
(B) (procedure you did to recover from the problem.)
  1. Uncheck "check for new messages at startup", and terminate Tb
  2. Unlock account
  3. Restart Tb, click a folder, enter proper password

You are saying that (A) didn't work? Tb attempted login even though password was already cleared?

Comment 10

[Mahesh Asolkar]

(In reply to comment #9)
> 
> I think next recovery procedure is required, if account lock happned.
> (A)
>   1. Clear saved password, and terminate Tb
>   2. Unlock account
>   3. Restart Tb, and enter proper password
> (B) (procedure you did to recover from the problem.)
>   1. Uncheck "check for new messages at startup", and terminate Tb
>   2. Unlock account
>   3. Restart Tb, click a folder, enter proper password
> 
> You are saying that (A) didn't work? Tb attempted login even though password
> was already cleared?

Exactly right. I first followed (A) - logically, I thought that would work. But it did not. Mere starting of Tb would lock the account. Then I followed (B) and it worked.

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #10)
> Exactly right. I first followed (A) - logically, I thought that would work. But
> it did not. Mere starting of Tb would lock the account. Then I followed (B) and
> it worked.

I could see next log just after restart of Tb(2009/8/01 build) with procedure of (A). It's same log as one obtained when old(wrong) password is saved.  
Tb3 sends null as password? Password is not cleanly removed by password manager?
Can you check it with server side log?

> Logging suppressed for this command (it probably contained authentication information)
> 2 NO [ALERT] Invalid credentials (Failure) 
> Logging suppressed for this command (it probably contained authentication information)
> 3 NO [ALERT] Invalid credentials (Failure) 
> Logging suppressed for this command (it probably contained authentication information)
> 4 NO [ALERT] Invalid credentials (Failure) 
> Logging suppressed for this command (it probably contained authentication information)
> 5 NO [ALERT] Invalid credentials (Failure)

Comment 12

[WADA:World Anti-bad-Duping Agency]

There was no rows for "imap://..." in signons.sqlite (checked with Fx 3.5+SQLite Manager. Only two smtp:// was seen in databade.)
Possibly null or garbage is sent as password.

Comment 13

[WADA:World Anti-bad-Duping Agency]

Quick summary: There at least three problems.
(1) New mail check at startup.
    Tb shouldn't attempt parallel connection unless at least one session is
    successfully connected after startup.
(2) After removal of password.
    Tb shouldn't attempt login when password is not saved.
(3) Possibility of account lock should be ZERO, when password is changed
    at server while Tb is running with old password.
    But it's impossible as far as complaint like bug 423354 should be cared for.
    "Password clear(or not-clear) upon each connection error" should be optional?

Confirming based on my some duplication test results.

Comment 14

[WADA:World Anti-bad-Duping Agency]

For above problem of (2).

(i) Logic after login failure looks next. Dialog of "Retry/New Password/Cancel" seems to be issued at here.
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#8021
Logic after successful login is done in else block(saves good password).
(ii) Logic before login looks "simplle call of password.get()", and logic for checking return code or returned value is not seen in the module.
(iii) If password manager simply returns null with some return code for "entry is not defined", problem like (2) can occur.

Comment 15

[Mark Banner (:standard8)]

I'm a bit concerned about this, but I haven't got my head around it all yet, so putting it on the proposed blocking list for now.

Comment 16

[David :Bienvenu]

I'm very concerned about this, so I'm plussing it - we've had bugs like this in the past and it's very painful for the users affected.

Comment 17

[WADA:World Anti-bad-Duping Agency]

David, should we open separate bug for each problem in comment #13?

Comment 18

[David :Bienvenu]

I'm not really sure there are three separate issues - basically, if we don't have a password, we should prompt for one, or if the password fails.

Comment 19

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #18)

Problem (1) looks different issue from problem (2) for me.
Even if password is cleared by first login failure and even if "password clear" is processed properly before attempt of login, all other concurrent login requests are already scheduled and executed using old(wrong) password, and many of them fails before password clear. It may cause account lock.

Multiple login requests(multiple sessions with IMAP) are serialized?

Comment 20

[Mahesh Asolkar]

If there is a possibility of simultaneous login attempts - as in case of having multiple IMAP folders - how about making one solitary login attempt. If it succeeds, then let the rest go simultaneously. If the solitary login attempt fails, then do whatever needs to be done to refresh password information. Then follow the same procedure all over again.

Comment 21

[David :Bienvenu]

there are a couple levels of password caching - one is in the password manager; an other is in the server object. The latter doesn't survive a restart. It sounds like we're perhaps not clearing or marking invalid the latter in the case of a logon failure, so we send it again the next time around. We want to keep it around to some extent so we can pre-fill the password dialog with it, but we don't want to send it to the server w/o the user's OK.

Comment 22

[Mark Banner (:standard8)]

Attachment: Proposed fix

There's two issues that I've found.

First is the okayValue isn't initialised if we don't display a prompt. This means that the nsImapIncomingServer::PromptForPassword function can return wrong values depending on what happens.

Second, nsImapProtocol::GetPassword doesn't handle cases where PromptForPassword (and nsMsgIncomingServer::GetPasswordWithUI) returns an error value of rv - the typical case being where it needs to prompt for a password and it can't get one because we haven't got a message window.

Coupled with those, I've decided just to remove the okayValue parameter - it doesn't add any value to what we get in rv.

This works on my test case of removing the password and starting up Thunderbird - I can see it trying but failing to get the password and hence aborting the connections (why it is trying to connect to folders other than just inbox I'm not quite sure).

I probably need to do some more testing, but if you could take an initial look at this at least, that would be good.

Comment 23

[Mahesh Asolkar]

(In reply to comment #22)

> (why it is trying to connect to folders other than just inbox I'm
> not quite sure).
> 

In my case, I thought it was because I have multiple folders in my IMAP account - about 20 I subscribe to - and I have mail.check_all_imap_folders_for_new set to true.

Comment 24

[David :Bienvenu]

yes, this sounds reasonable. I'll test it a bit over the weekend.

Comment 25

[Mark Banner (:standard8)]

Try server builds:

http://s3.mozillamessaging.com/build/try-server/2009-09-04_18:[e-mail]-st8-imap-login/[e-mail]
http://s3.mozillamessaging.com/build/try-server/2009-09-04_18:[e-mail]-st8-imap-login/[e-mail]
http://s3.mozillamessaging.com/build/try-server/2009-09-04_18:[e-mail]-st8-imap-login/[e-mail]2

Comment 26

[Mark Banner (:standard8)]

Checked in: http://hg.mozilla.org/comm-central/rev/bb4b3b275c93

Comment 27

[WADA:World Anti-bad-Duping Agency]

Checked with next build.
> Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.4pre) Gecko/20090907 Shredder/3.0b4pre

(1) Password change while Tb is inactive, restart Tb.
    Only single login attempt was seen in IMAP log.
    Dialog for "Retry, New Password, Cancel" was displayed.
(2) Password change while Tb is running.
    Dialog for "Retry, New Password, Cancel" appeared upon connection request
    by such as "check new messages every NN minutes".
In any case, all of "Retry", "New Password", "Cancel" worked well.  
This bug's problems look to have been resolved.

(Off-Topic)
I didn't check test case of "Leave dialog with not-responded status for long long time". I hope no memory leak or no resource shortage in such situation.
All subsequent connection requests are rejected when the dialog exists? Or queued?