1.37 KB, patch
|Details | Diff | Splinter Review|
2.28 KB, text/plain
2.32 KB, text/plain
1.10 KB, patch
|Details | Diff | Splinter Review|
Using a Thunderbird 126.96.36.199 profile, which has STARTTLS selected for secure connection, and where useSecAuth is not set, migrating to Thunderbird 3.0pre (20091015 nightly, Win32) results in mail.smtpserver.smtp#.useSecAuth = true. Thus, the initial attempt to send a message fails, I had to manually clear the "use secure authentication" checkbox. Proposal: If secure authentication is enabled, and useSecAuth not set in the previous profile, initialize mail.smtpserver.smtp#.useSecAuth as false given that a secure mechanism is already available. Not many servers support secure authentication and secure connection at the same time, and it doesn't provide additional security anyway, thus this should be a safe approach.
The same happens when migrating a SeaMonkey 1.1.18 profile to SM 2.0 RC1.
The case in bug 523153 is interesting as neither encryption nor secure authentication was set in the 2.0 profile. While that combination in general should be discouraged (in my example security was still maintained by the connection encryption), the question is if secure authentication should be switched on when migrating the profile. As with bug 491202, a fallback likely would have to be provided if the initial attempt using that option fails.
I'll look at this - I vaguely remember Magnus having some experience with bugs in this area, but I could be wrong.
use secure auth is completely orthogonal to the connection security, as far as the code is concerned (i.e., you can use secure auth with secure connections, non secure auth with insecure connections, whatever - we don't limit you to certain combinations.) I'm not able to reproduce this here. Is it the act of "migrating" that causes the useSecAuth to get set to true, or did you try to send an e-mail first? There is a per server pref that controls whether we'll try secure auth, if the server advertises it (trySecAuth) which defaults to true. If we detect that the server advertises secure auth, we will set useSecAuth to true, but you're saying your server doesn't support secure auth?
(the try secure auth stuff is not new to 3.0 - it's been there for a long time, I believe).
I don't recall exactly if I checked the pref before or after trying to send a message, but it was definitely set afterwards. The trySecAuth pref is (now) false for that server (default is true) along with useSecAuth, but it might have been set after unchecking the box. In the 2.0 profile, no specific settings are present for either trySecAuth (true by default) or useSecAuth (false by default). The server supports secure connection but not secure authentication, it advertises GSSAPI though in addition to AUTH and LOGIN.
I'm pretty sure we consider GSSAPI to be secure authentication.
Ok, still wondering then why 2.0 wasn't bothered... ovidiu, can you check what your SMTP server advertises?
This code changed quite a bit from 2.0 - see bug 311657
So, looking at bug 394043, I seem to run into a case where GSSAPI is advertised but no credentials are available (or possible). "The huge number of servers that incorrectly offer GSSAPI means that this is the only way to do this without introducing new-UI to allow the user to request GSSAPI." So, if there are frequent cases where GSSAPI is advertised but not configured, wouldn't this be an exception to the rule of not falling back to non-secure authentication? This appears different from, let's say, CRAM-MD5 which should be supported by the server if advertised.
That's a reasonable analysis - do you think you could generate a patch?
Sorry, I'm not familiar enough with that part...
d'oh, that makes two of us - what if I or Christian were to generate a patch for you to try?
The same most likely applies to the IMAP side of that server as well, that's bug 491202 I've filed before. Patch would be good, sure. :-)
(In reply to comment #9) > ovidiu, can you check what your SMTP server advertises? sorry, how do I do that?
(In reply to comment #16) > sorry, how do I do that? telnet smtp.server.com 25 ehlo server.com And you'll got everything what it support
250-mail6.dr.myx.net 250-PIPELINING 250-SIZE 5520000 250-ETRN 250-AUTH PLAIN LOGIN GSSAPI DIGEST-MD5 CRAM-MD5 250-AUTH=PLAIN LOGIN GSSAPI DIGEST-MD5 CRAM-MD5 250 8BITMIME
(In reply to comment #18) > 250-AUTH PLAIN LOGIN GSSAPI DIGEST-MD5 CRAM-MD5 > 250-AUTH=PLAIN LOGIN GSSAPI DIGEST-MD5 CRAM-MD5 Hmm, that's clearly different from what I see as you also have DIGEST-MD5 and CRAM-MD5 which should work without special configuration, if I'm not mistaken. Also, the second line with AUTH=... looks strange, I don't have that one.
AUTH= for broken clients like OE, who don't understand AUTH w/o "="
I'm not sure that falling back to insecure authentication in the GSSAPI case is the right thing to do. I don't think it's really ever the right thing to do, if use secure auth is checked (and we don't know if the user or probing checked in). I think we should ignore GSSAPI when doing probing for secure auth, unfortunately, in the smtp protocol code. The autoconfig probing, on the other hand, could treat GSSAPI as unreliable, and fallback to probing w/o GSSAPI, and not set use sec auth if verify logon failed. Not without pain, of course...
Yes, your patch did the job! I've created a fresh comm-1.9.1 build with a fresh 188.8.131.52 profile, showing trySecAuth=true and useSecAuth=false. Migrating to 3.0pre without the patch left the defaults intact until I tried to send a message, then trySecAuth=false and useSecAuth=true for that server. Recovering the 184.108.40.206 profile and adding the patch to 3.0pre made that trySecAuth=false and useSecAuth=false as intended, and the connection no longer failed. While this covers my case, that may not resolve ovidiu's problem. Since his server proposes MD5 authentications in addition to GSSAPI, I'd expect that box nevertheless to be marked with useSecAuth=true in that case. Looking at nsSmtpProtocol.cpp lines 951-959, GSSAPI is tried first if set and only then the CRAM_MD5 version. Maybe that's part of the problem there.
maybe CRAM_MD5 also fails in his case?
Comment on attachment 407762 [details] [diff] [review] possible fix >+ // Don't trust GSSAPI since the server can advertise it >+ // but the client may not be set up for it. >+ m_prefUseSecAuth = TestFlag(SMTP_AUTH_SEC_ENABLED & >+ ~SMTP_AUTH_GSSAPI_ENABLED); I read the code as "If the server advertises GSSAPI then don't try secure auth". But the comment isn't clear as to what's supposed to happen in the case where the server advertises multiple secure auth mechanisms.
The intention is "don't use secure auth if *only* gssap is enabled, but no other secure auth mechanisms"
(In reply to comment #24) > maybe CRAM_MD5 also fails in his case? ovidiu, can you create an SMTP log for your case?
Comment on attachment 407762 [details] [diff] [review] possible fix >+ m_prefUseSecAuth = TestFlag(SMTP_AUTH_SEC_ENABLED & >+ ~SMTP_AUTH_GSSAPI_ENABLED); Ah, so SMTP_AUTH_SEC_ENABLED is SMTP_AUTH_GSSAPI_ENABLED | SMTP_AUTH_*_ENABLED?
Yes, if * is the other secure auth mechanisms other than gssapi.
(In reply to comment #27) > ovidiu, can you create an SMTP log for your case? how?
that logon looked like it worked.
Did you run this with "Use secure authentication" checked to produce the error?
Comment on attachment 407762 [details] [diff] [review] possible fix >+ m_prefUseSecAuth = TestFlag(SMTP_AUTH_SEC_ENABLED & >+ ~SMTP_AUTH_GSSAPI_ENABLED); OK, so the name SMTP_AUTH_SEC_ENABLED confused me. Maybe it should be SMTP_AUTH_ANY_SEC_ENABLED? (Same for INSEC of course.)
yes it did, with it [.]unchecked, the fault was that initially tb3 [v]checked it on upgrade. *should I do log with checked (not working)? *is there a file to del or something so that I force it to "re assume" what to do as if it's fist time over tb2 profile? the use case was:(explanation Bug 523153) *by upgrading (operate tb3 over the old tb2 profile) --the auth was defaultly [v]checked resulting in unable to send msg. --Simply operating tb2 over it did sent, before I discovered what happend or do any changes. --Swithcing tb2-3 several times kept this behavior, tb2 ok vs 3 not ok [in tb2 I don't have that checkbox, in tb3 I have it]. *I did [.]uncheck it and worked since, made the sceen capture, and this log with working one. It did not revert to faulty [v]checked ever since, by using 2 and 3 alternatively or so, it was just the first time operating 3 over the profile that really assumed the wrong variant.
(In reply to comment #36) > yes it did, with it [.]unchecked, > the fault was that initially tb3 [v]checked it on upgrade. > *should I do log with checked (not working)? Yes please, the idea being to see if it tries *-MD5 and if they fail too. Currently, only the successful plain login is shown in the log. > *is there a file to del or something so that I force it to "re assume" what to > do as if it's fist time over tb2 profile? I've made a backup of the 2.0 profile before each migration attempt, after that simply throw away the migrated TB3 profile and copy your TB2 backup in place for the next attempt. For now though, simply testing what happens if the checkbox is ticked in normal operation would be sufficient for the log.
That log unfortunately doesn't tell which authentication method was used, but there are two failed attempts, which may relate to the DIGEST-MD5 and CRAM-MD5 methods. Would GSSAPI even try to authenticate if no Kerberos is configured?
Comment on attachment 407762 [details] [diff] [review] possible fix From rsx11m's comments this certainly seems to be an improvement.
Comment on attachment 407762 [details] [diff] [review] possible fix patch checked into trunk and 3.0 branch
yes, we would try gssapi authentication even if Kerberos is not configured because we only find out that it's not configured by trying to do the authentication. But we would fall back to trying cram-md5/digest-md5. It may be that the server is advertising cram-md5 but not implementing it correctly. We've certainly had experience of that. I could try to generate a try server build where smtp logging also showed the authentication protocol, so we could get a better idea of what's failing with ovidiu's authentication. We could also turn off this upgrade feature by flipping the pref that controls trying secure auth. I agree that falling back to insecure auth when the connection is secure would be OK, but making that change to the code at this point is scary. Or we could just relnote this.
Now that I think about it, if Kerberos isn't configured on the client, we would try gssapi to some extent, but we wouldn't send any gssapi protocol to the server, so it wouldn't show up in the log. So this could simply be CRAM-MD5 and DIGEST-MD5 failing. An other possibility would be to only do the secure auth probing if the connection was not secure, though that's not going to help 2.0 users who were doing insecure authentication over non-encrypted connections...Ludovic and anyone else with an opinion, do we think this secure auth probing is worth the extra support issues it's going to raise for migration from 2.0 to 3.0? Should we add something to the FAQ and What's new page as well?
> Ludovic and > anyone else with an opinion, do we think this secure auth probing is worth the > extra support issues it's going to raise for migration from 2.0 to 3.0? David, can you summarize what's changed between Tb2 and Tb3 at a high-level and exactly what our options are? I don't have quite enough context to comment based only on the low-level details.
See bug 311657 for the details. TB 2 had no UI for whether to use secure authentication mechanisms or not. We would try the most secure auth mechanisms advertised and if those failed, fall back to insecure authentication. There was a hidden per server pref, trySecAuth, where you could turn off even trying secure auth, basically because of broken/mis-configured servers, iirc, that would drop the connection if we tried secure auth mechanisms. TB 3 adds a UI for the user to specify secure authentication mechanisms only, and does not fall back to insecure mechanisms if the corresponding per-server pref, useSecAuth, is set. It also doesn't try secure auth if that pref is not set. This matches what we do for other kinds of servers. TB 3.0 does a one-time "migration" of the smtp server secure auth setting from existing pre 3.0 profiles as follows: The first time you attempt to authenticate an smtp server with 3.0, we check if trySecAuth is set. If so, we check if the server advertises secure auth, and if so, we set trySecAuth to false (making it a one time probing), and set useSecAuth to true. The patch in this bug removed GSSAPI from the secure auth types we detect when probing, since just because the server advertises it does not mean the client can use it. The remaining issue seems to be that some servers advertise CRAM-MD5 and DIGEST-MD5, but logon attempts fail. However, because the server advertises them, we set the server to be a useSecAuth server. The user then can't send mail with 3.0, whereas they could with 2.0. The workaround for those users to go into account settings and turn off the use secure auth checkbox, and they're good to go from now on. I have no idea how common those servers are, other than the feedback we've gotten from the betas (the code has behaved this way on the trunk since 06/2007). I suspect most of the people who've had this issue were bit by the GSSAPI issue, since fixed. Note that autoconfig does not have this issue as much because it falls back to insecure auth, as long as the connection is secure. Our options are: 1. Relnote/faq/what's new along with support education and vigilance 2. Allow fallback to insecure auth if the connection is secure (a bit scary to change the code at this point, and probably not too helpful to that many existing users since 2.0 defaulted to insecure connections anyway) 3. Somehow notice that the authentication failed in a probing session and set useSecAuth back to false. I'm a little scared to change this code too much since it can't be unit-tested because I don't think fake server supports secure authentication mechanisms or secure connections.
(In reply to comment #45) > I have no idea how common those servers are, other than > the feedback we've gotten from the betas FWIW mine is a crappy old server and has not evolve long time now as much as they do not intend to promote this service in the future. (though it was a very popular one in this country years ago, cannot say today) I was wondering how much it is used and how marginal the use case, but I cannot decide that
When starting Seamonkey 2 using the profile of Seamonkey 1.1.8 (pointing the profile.ini to it) the smtp serversettings are automatically set to "use name and password". Even if there is no name set and the old prefs.js contains: user_pref("mail.smtpserver.smtp1.auth_method", 0);
Comment #47 was also independently filed as bug 524868. If you want to split that off I'll confirm the other bug, or we can consolidate the cases here.
clearing off the blocking list.
Another perspective on the issue is given in bug 527078, for users connecting to the same server from a "trusted" or an "untrusted" network, thus changing the requirements for connection security and authentication. For those users, a single combination of "STARTTLS if available" and "User name and password" was catching both cases and configured/used as necessary. Per bug 350314, this ambiguity was discouraged, thus those users now would have to define two SMTP servers and switch between them, depending on their location. All those cases point to the same underlying issue, the behavior is now more strict, the user may not understand the error messages received, and it may be inconvenient when having to switch server definitions with different settings. On the other hand, there is the question of whether or not it's acceptable to fall back to a less secure combination for convenience, thus introducing an ambiguity whether or not the current connection can be considered secure.
I installed Sea Monkey 2.0 on my Windows XP system on 2009/11/20 upgrading from 1.1.18. I experienced this bug but could not follow the advice offered by the error message display and on the seamonkey forum as the use secure authentication check box was not checked so I could not clear it. Unless one has to do something like checking this box it, saving the information restarting seamonkey and then unchecking the box and restaring sea monkey again. Luckily uninstalling 2.0 and using the pre-install restore point seems to resurrect 1.1.18 without problems.
Re David Bienvenu's comment #45. "The workaround for those users to go into account settings and turn off the use secure auth checkbox, and they're good to go from now on." This workaround was not possible as the appropriate Use secure authentication box was not checked in the first place if my memory serves me correctly. I have already reverted to 1.1.18 so cannot go back to look. My mail server for both ingoing and outgoing is mail.zip.com.au. My settings for this server in Sea Monkey 1.1.18 were use secure connection = never and use secure auth - unchecked.
(In reply to comment #52) > This workaround was not possible as the appropriate Use secure authentication > box was not checked in the first place if my memory serves me correctly. The box is initially unchecked before the first attempt to send a message is made (that's the trySecAuth status). Once the probing resulted in a secure authentication, the box is checked. Unchecking it then should resolve the issue, unless trySecAuth is magically set again somehow. > My mail server for both ingoing and outgoing is mail.zip.com.au. Connecting to that server on ports 25 and 587 doesn't show any AUTH response at all, thus "use name and password" should be unchecked as well (bug 524868).
(In Reply to #53) Maybe it is not the same bug 522633 then. I made several failing attempts to send a message and after all of them the use secure auth box remained unchecked.
small comment on this: I observe a very similar problem with the beta 4 (shipped with opensuse 11.2) and with the final build (X11; U; Linux i686; de; rv:220.127.116.11) Gecko/20091130 SUSE/3.0.0-12.1 Thunderbird/3.0. A migrated profile for t-online.de (the biggest provider here in germany) works with plain unencrypted smtp on TB2 and fails on TB3. Error message complains about a secure smtp connection try which fails as the server doesn't support it (which is correct, as I can see with netcat). But still, the UI settings indicate no security (STARTTLS or TLS/SSL are not set). I will post additional info on the server capabilities and prefs settings later ... From my point of view this is a severe showstopper, as users can not send mail after upgrading.
Is this big itself fixed noe? Perhaps, if the core issue or one of multiple issues is fixed here, mark is closed and e.g. use bug 524868 for the other issue?
The issue for which I had opened the bug is resolved for me, but it was my impression that David wanted to investigate further. Including this one, I'm counting at least four pending bugs on bug 311657 (see bug 524868 comment #12 for references to the other ones). Thus, I'd agree that consolidating those to a few core problems yet open may be a good idea.
I could not go deeper into my issue yet as I am still waiting for a prefs.js from the machine where it happened. TB 3 seems to ask for STARTTLS after migration without proper reason. If this is true, I would better assign my issue to ID 534158, I assume ...
I have upgraded from Thunderbird 2 to Thunderbird 3 and cannot send emails because the upgraded software will not connect to the SMPT outgoing server as described above. My settings do not request any form of "authentication"....obviously the upgraded software is causing the problem. I need to send emails. Is there a convenient fix, or do I need to change to different email software?
Here are some more data: The server capability ------------------------------------ netcat mailto.t-online.de 25 220 fwd08.aul.t-online.de T-Online ESMTP receiver fmsad1025 ready. EHLO 250-fwd08.aul.t-online.de ready. 250-SIZE 52428800 250-8BITMIME 250-ENHANCEDSTATUSCODES 250 HELP 250-2.0.0 Supported commands: 250-2.0.0 HELO EHLO MAIL RCPT DATA RSET NOOP QUIT HELP 250 2.0.0 See RFC 821, RFC 1123, RFC 1869, RFC 1870. ------------------------------------ The smtp part of the prefs.js AFTER migration from TB 2.x to 3.x : ------------------------------------ user_pref("mail.smtp.defaultserver", "smtp1"); user_pref("mail.smtpserver.smtp1.auth_method", 1); user_pref("mail.smtpserver.smtp1.description", "t-online"); user_pref("mail.smtpserver.smtp1.hostname", "mailto.t-online.de"); user_pref("mail.smtpserver.smtp1.port", 25); user_pref("mail.smtpserver.smtp1.trySecAuth", false); user_pref("mail.smtpserver.smtp1.try_ssl", 3); user_pref("mail.smtpserver.smtp1.useSecAuth", false); user_pref("mail.smtpserver.smtp1.username", "firstname.lastname@example.org"); ------------------------------------ (unfortunately I have no copy of the settings before migration) Bye, Thomas
(In reply to comment #60) > I have upgraded from Thunderbird 2 to Thunderbird 3 and cannot send emails > because the upgraded software will not connect to the SMPT outgoing server as > described above. My settings do not request any form of > "authentication"....obviously the upgraded software is causing the problem. > > I need to send emails. > > Is there a convenient fix, or do I need to change to different email software ? artmount: My guess is that: Uncheck 'Use name and password' for that server will help full details: http://gsfn.us/t/nvmd If that doesn't work: Please create a new support request at: http://getsatisfaction.com/mozilla_messaging/topics/new
> (comment #60) My settings do not request any form of "authentication".... If you get an error message referring to SMTP-AUTH not being supported by your server, that's case #1 in bug 524868. This specific bug here is on *secure* authentication broken upon migration, not authentication being required after the upgrade per se. According to your comment #61, this part seems to have worked as intended (both trySecAuth and useSecAuth are false).
Created attachment 420904 [details] [diff] [review] turn off sec auth probing This patch would turn off upgrade smtp secure auth probing, but would allow us to turn it back on later, if we wanted. I'm basically putting this up for consideration, as an option, for 3.01 or 3.02...
Just to understand the implications of attachment 420904 [details] [diff] [review] correctly, this means that secure authentication for SMTP is never probed for, thus the checkbox is *not* set even if the server would support it, and any secure mechanism wouldn't be used unless the user explicitly requests it? Based on a respective discussion in the IMAP counterpart (bug 491202), this may result in a combination where neither secure connection nor authentication is configured, thus exposing the password. Is there any other safeguard against this case? In line with the IMAP-autoconfig implementation, rather than just probing the AUTH response and flipping on the authentication checkbox just based on that, it could be checked if secure authentication goes through first with the given credentials before falling back to unsecure authentication. Based on duplicates reported, no authentication at all covered by bug 524868 seems to be the more significant issue anyway. The GSSAPI issue was a specific case, but then there are cases like comment #18 where no secure authentication seems to work (which would be covered by a procedure specified in the previous paragraph here).
Some interesting observation: The server in bug 540473 reports CRAM-MD5 as the only secure authentication method, which apparently failed. That's more specific than bug 523153 or bug 539944, which had multiple authentication methods advertised in the EHLO response. All have different providers, so that's suspicious if something more is wrong with secure authentication, causing those failures...
In reply to comment #c68 from rsx11m: Some providers do not offer secure smtp methods like STARTTLS/SMTP with TLS on the same server address but on another server with a different DNS name. I would prefer some kind of prompt to the user, before any changes to the server settings are executed. Changes to incoming or outcoming server connections are extremely delicate and will cause big annoyance especially for normal (inexperienced) users. Yesterday i had the third issue caused by this automatic profile migration ... I think that this issue qualifies for a very fast patch, even before tb 3.1
The underlying issues with secure authentication may already have existed in the prior TB 2.0 releases, which however just resulted in it falling back to an unsecure LOGIN/PLAIN authentication, thus exposing credentials if combined with an insecure connection. TB 3.0 exposes those now after the new checkbox and the trySecAuth mechanism was introduced. Ben's work in bug 525238 should help the diagnosis here by being able to pick a specific flavor of secure authentication and test them one-by-one.
Can anyone here give me access to an SMTP server that has this issue? We'd like to work on a fix that turns secure authentication and then if authentication fails, turns it back off...
As per discussion yesterday, at the very least landing the "turn off sec auth probing" patch needs to block 3.1.
A kind soul did give me access to an smtp server with this issue.
I've turned off secure auth probing for 3.02 and on the 3.1 trunk for now. I'm going to try to reproduce the issue with the server I got access to, but I need to be on a network that allows access to port 25 - I'll try to do that this afternoon.
I've been looking at the imap issue for the server Ovidiu gave me access to. The first thing I tried was STARTTLS access on port 143 with this server. We fail to connect because STARTTLS resets the connection during the negotiation. We essentially handle this error silently, because we think we're going to retry, but we don't retry in the case of an error before or in logon...I think if we retried, we'd then error out and tell the user, so I think I have a fix for that case, at least.
We're resetting the blocking flag for 3.1 on this bug and instead setting the wanted-thunderbird+ flag. We have too many blocking-3.1 bugs, to the point where it doesn't mean much, and managing the list is making it hard to actually work on closing bugs, which helps no one. Thunderbird 3.1's primary purpose is to allow us to offer a prompted major update to Thunderbird 2 users, to ensure their continued ability to safely use Thunderbird. Thunderbird 2 is built on an outdated version of Gecko, and our long-term ability to maintain the users' safety for Thunderbird 2 users is limited. If you think this bug meets the requirements below, please renominate with a detailed explanation of how it meets the following two criteria, and we will reconsider. To qualify, this bug must either: a) make the upgrade experience from TB2 very painful for a large number of users or b) be a new, reproducible, severe quality issue (eg dataloss, frequent crashes) Just because this bug doesn't block TB3.1 doesn't mean it can't or won't make the release. Once they're done with their blockers (if any), we encourage developers to keep working on non-blocking bugs, and to try to land them as early in the cycle as possible, as non-blocking bugs will become increasingly difficult to land in the later stages of the cycle.
According to the report in bug 556984, accounts may still be marked for secure authentication when migrating 2.0 -> 3.0.4, despite attachment 420904 [details] [diff] [review].
Turns out that bug 556984 was about "Use name and password", thus false alarm, nevertheless a case that yet has to be satisfactorily covered.
Have patches for this bug made it into released versions of Thunderbird? I had the opposite problem from this. I was able to send mail in TB2, but in TB3 I couldn't authenticate. What fixed the problem for me was checking the "Use secure authentication" box (there was no equivalent in TB2), which had been unchecked.
rsx11m what follow up needs feeling ? Marking fixed as this has been checked-in
The issue I've filed this bug for should be fixed with TB 3.0.2 and SM 2.0.3, the general "Use name and password" issue was either mitigated in the redesign of the security options in the server settings or should be handled in a separate bug if it's still the case (bug 524868 comment #13, bug 535033).