204371 - SMTP fail with repeat password dialog (CRAM-MD5 authentication refused)

Status: RESOLVED INCOMPLETE

(MailNews Core :: Networking: SMTP, defect)

Description

[noririty]

Steps to Reproduce:
1. Add a smtp account
2. Try to send a mail

Actual Results:  
Repeat PASS command did not succeed dialog box

Expected Results:  
Let me send my mail

 - Occurs On
        2003-04-16-09-trunk
        2003-05-02-11-trunk

 - Doesn't Occur On
        2003-04-16-04-trunk (Last good build)

Comment 1

[noririty]

Attachment: succeeded

SMTP Protocol Log (succeeded w/2003-04-16-04)

Comment 2

[noririty]

Attachment: failed

SMTP Protocol Log (failed w/2003-04-16-09)

Comment 3

[Christian Eyrich]

Why sending with 2003-04-16-04 is no problem is easy. The server doesn't submit
a known AUTH mechanism (to this time we only supported PLAIN and LOGIN), so we
don't use one.

Why we don't fall back after fail with CRAM-MD5 with 2003-04-16-09 and on is
also clear. Because you checked "Use name and password" Mozilla tries to use
authentication anyways and since CRAM-MD5 is the only mechanism it can only fall
back to this.
To send mails over this server, just uncheck "Use name and password" in your
SMTP configuration.

The interesting question is, why the authentication fails at all. I don't know -
the only anomaly I can see is, the decoded challenge from the server is
something like <1364370800.11847752@ᷮmf.home.ne.jp> - the character after the @
has the decimal code 201 (0xc9).
What could be the problem with this? base64 decode is not, maybe MSGCramMD5()
has a problem also I can't see one. David?

Or the server sends this character by error but compares our answer to the
correct value.

I removed blocking of 202011 because there's no dependency.

Comment 4

[David :Bienvenu]

MSGCramMD5 won't have a problem with that, but the underlying PSM hash code
might (I don't know if it does or not)

Comment 5

[Christian Eyrich]

Sorry, I misread the decoded challenge. It's
<1364370800.11847752@?w.mf.home.ne.jp> whereby the character after the @ has the
*octal* code 201 (0x81).

I now checked Mailnews' HMAC-Hash answer to the challenge from the second log
against the HMAC-Hash calculated by some external hash calculators. And it
matches, so we can be sure de- and encoding by Mailnews works absolutely ok.

That draws us back to the question what else can be wrong.
Noririty, could you check sending mails using this server with an other CRAM-MD5
enabled mailer?

Comment 6

[Matteo Settenvini]

Possibly related to bug 205003, also if that bug is for POP3 networking.

Comment 7

[Christian Eyrich]

Related in so far as CRAM-MD5 might be generally broken in some servers, yes.

But I didn't get an answer from the reporter if he could reproduce the failure
with another client, so I can't say what side to blame.

In general not working CRAM-MD5 is no big problem in SMTP because we do fallback
here. In this case it's only a problem because there's nothing to fallback to.
So 1. only few users will ever notice a problem here and 2. we can't do anything
against it.

Comment 8

[Philip Wilk]

Possibly bug 155172 is a duplicate or related. I agree this is a major problem.

Comment 9

[Philip Wilk]

see also bug 154099

Comment 10

[Christian Eyrich]

No, neither bug 155172 nor bug 154099 are related to this one.
The first one not because it's not the problem that the user won't be asked if
authentication failed. And the later not because the auth failure *is* recognized.

By the way, both bugs were reported long time before the last working (in terms
of this bug) build (which is as the reporter stated are 2003041604).

Comment 11

[(not reading, please use [e-mail] instead)]

taking, but this might be a better one for [e-mail].

didn't bienvenu change things so we don't do CRAM-MD5, unless the user enabled
an explicit hidden pref for it?

Comment 12

[David :Bienvenu]

[e-mail] made that change - I just checked it in.

Comment 13

[Christian Eyrich]

No, no one made any change. This pref was for POP3. Since we do fallback in
SMTP, we don't have a problem if one mechanism fails.

As I wrote in comment 3 the problem here is, that the user checked "Use name and
password", but the only mechanism (CRAM-MD5) fails. Here we really can't do
anything besides not to fail.
But the reporter doesn't answer to all my requests on testing with other clients
and I can't see any failure in our answer (see my comment 5).

Comment 14

[toofan]

Attachment: Failed Password Authentication

I have also faced this problem within my corporate network. I can't say for
sure what server we are using. The attached log file shows the details.

Comment 15

[toofan]

Attachment: Sylpheed Log

I have attached a log that I got from trying to use Sylpheed-Claws with the
same SMTP server. This also gave the same error.

Comment 16

[toofan]

Attachment: Mozilla 1.3.1 SMTP Log

I reverted to Mozilla 1.3.1, and found out that it goes through because it does
not seem to use authentication, even though "Use User-id/Password" is checked.
The log is attached - there was probably a bug in 1.3.1, which allowed it to go
through in such a case.

When I went back to 1.4.rc-1, and unchecked this option, the send worked.

There seems to be something wrong with our SMTP server, in this case.

Comment 17

[Christian Eyrich]

So there seem to be faulty SMTP servers with CRAM problems like the POP3 servers
we know of (e.g. bug 205003). I guess now, in noririty's case it's the servers
problem too.

There might be much more but since we do fall back if PLAIN or LOGIN have also
been advertised, much of them won't be noticed by the user.

Any suggestions for a workaround, David, Seth? Or close this bug as INVALID?

Comment 18

[David :Bienvenu]

Can we just add a pref and some UI to use secure authentication like we did for
incoming servers, and default it to true for smtp servers?

Comment 19

[Christian Eyrich]

Hm, what should that bring?
The current checkbox "Use username and password" can be seen as such a pref. So
I don't think we need another one. The problem is not we're hung as soon as
CRAM-MD5 fails - we're only if there's no weaker meachanism advertised too.

Currently Mozilla states the password is wrong while it isn't (resp. it might be
or not - we don't know for sure).


BTW: We're a little bit inconsistent. If the pref is enabled and no known
mechanism advertised, we try to send the mail(s) without login in (HELO style).
If we have a know mechanism but fail with it, we alert the user.
I'd vote to never try without pw if the option is enabled.

Comment 20

[toofan]

I would like to suggest that, if the "Username and Password" option is set, and
the authentication does not go through 3 times (3 is a magic number), it would
be nice to bring up a dialog box saying, "Would you like to disable SMTP server
authentication and try again ? This works with some SMTP servers. <Yes> <No>
<Cancel>". The dialog box could have a "Don't ask me this question again" option
box.

I would also like to second Christian's opinion. I have not verified what he
said, but if the server has no known authentication method, then it may be
better to display a message like "The SMTP server does not support
authentication. Would you like to disable SMTP server authentication and try
again ? <Yes> <No> <Cancel>". Again, with the "Don't ask me ..." option.

The problem is that 1.3.x's behaviour was slightly different from 1.4's
behaviour (but I believe that 1.4's behaviour is correct, since 1.3.x gives you
the false sense of security). The bigger problem is that Netscape and Microsoft
Outlook behave like 1.3.x - they try without authentication. So people might
wonder why 1.4 behaves differently from Netscape, Moz 1.3 and Outlook.

Comment 21

[Christian Eyrich]

What do you mean with "3 times". Should we try to authenticate 3 times using the
same mechanism without prompting the user, 3 times all server supported
mechanisms one after another (so 3 "rounds") without prompting the user. Or one
"round" as now, then prompting the user for wrong password like now, trying
another "round" with the new entered, prompting the user again like now, then
another "round" and then saying "Authentication failed 3 times. Would you like
to disable SMTP server authentication and try again? This may work with some
SMTP servers. <Yes> <No>"?

> but if the server has no known authentication method, then it may be
> better to display a message like "The SMTP server does not support
> authentication. Would you like to disable SMTP server authentication and try
> again? <Yes> <No>". Again, with the "Don't ask me ..." option.

I think I like that.

> The bigger problem is that Netscape and Microsoft Outlook behave like 1.3.x -
> they try without authentication.

Oh, yes? I thought Outlook gives up if you say (secure password authentication)
and the server doesn't support LOGIN (M$ idea of a secure login).

Comment 22

[Roberto Avanzi]

I have encountered this bug with 1.4rc2 and 1.4rc3, not with the 
following version

Mozilla 1.4a

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4a) Gecko/20030402

Comment 23

[Christian Eyrich]

*** Bug 211238 has been marked as a duplicate of this bug. ***

Comment 24

[Jonas Bandi]

The following remedy does not work for me:

To send mails over this server, just uncheck "Use name and password" in your
SMTP configuration.

"Use name and password" is not checked in my SMTP-Preferences, but mozilla does
ask for a password anyway, and consequently fails to deliver.

Comment 25

[Christian Eyrich]

Sure you unchecked "Use name and password" in the right servers config? It's
quite strange if it's off and Mozilla asks anyway.

Switching the use of authentication off does not help in any case (resp. in few
cases I think). Only if the provider allows unauthenticated logins.

If you're sure you unchecked in the right config, a log could be interesting to
see what mechanisms the server provides.
Ah, and please write what version you're using.

Comment 26

[Jonas Bandi]

Problem solved by creating a new profile in 1.4.

It seems to be a problem with installation/migration of profiles from 1.3 to 1.4.
In 1.3 everything worked fine. After a default installation of 1.4 (the profile
was automatically migrated, as always ...) this bug occured. In this (migrated)
profile the checkbox "Use name and password" does not work: the password gets
asked in both cases, and consequently sending fails.
Now I created a new account, and sending works, when the checkbox is deactivated.
But in the old (migrated) profile the checkbox does still not work (error is
reproduceable).

Comment 27

[Christian Eyrich]

Ah, ok.

What if you set mail.smtpserver.smtp*.auth_method (* for your smtps number) to 0
by hand in your migrated profile (when moz shut down)?

And you're getting login failures if you're trying to login with name and
password. Is this because no login required or a real login failure?
So a log would nevertheless be good.

Comment 28

[Asa Dotzler [:asa]]

jbandi, please don't set flags if you don't know how. Thanks. 

Comment 29

[Zarco Zwier]

> What if you set mail.smtpserver.smtp*.auth_method (* for your smtps number) 
to 0 by hand in your migrated profile (when moz shut down)?

Hey man, this seems to have worked for me :)

Comment 30

[Wayne Mery (:wsmwk)]

(In reply to comment #26)
> Problem solved by creating a new profile in 1.4.
> 
> It seems to be a problem with installation/migration of profiles from 1.3 to 1.4.
> In 1.3 everything worked fine. After a default installation of 1.4 (the profile
> was automatically migrated, as always ...) this bug occured. In this (migrated)
> profile the checkbox "Use name and password" does not work: the password gets
> asked in both cases, and consequently sending fails.
> Now I created a new account, and sending works, when the checkbox is deactivated.
> But in the old (migrated) profile the checkbox does still not work (error is
> reproduceable).

so is it correct that this not (strictly speaking) a networking problem, but rather a migration or a profile problem?

Comment 31

[Wayne Mery (:wsmwk)]

Christian in comment #5
> Sorry, I misread the decoded challenge. It's
> <1364370800.11847752@?w.mf.home.ne.jp> whereby the character after the @ has the
> *octal* code 201 (0x81).
> 
> I now checked Mailnews' HMAC-Hash answer to the challenge from the second log
> against the HMAC-Hash calculated by some external hash calculators. And it
> matches, so we can be sure de- and encoding by Mailnews works absolutely ok.
> 
> That draws us back to the question what else can be wrong.
> Noririty, could you check sending mails using this server with an other CRAM-MD5
> enabled mailer?

reporter noririty is no longer available, so his/her comments will not be forthcoming

do we need a bug to implement comment 21 about the lack of an authentication method?

Comment 32

[Wayne Mery (:wsmwk)]

Version 91 has all new smtp backend code. If you can still reproduce this issue, please file a new bug report https://bugzilla.mozilla.org/enter_bug.cgi?product=MailNews%20Core&component=Networking%3A%20SMTP