Last Comment Bug 244030 - Using 127.0.0.1 (nonrouteable IP) as HELO string
: Using 127.0.0.1 (nonrouteable IP) as HELO string
Status: RESOLVED FIXED
: fixed1.8.1
Product: MailNews Core
Classification: Components
Component: Networking: SMTP (show other bugs)
: Trunk
: All All
: -- normal with 4 votes (vote)
: mozilla1.9alpha2
Assigned To: Christian Eyrich
:
:
Mentors:
http://rfc.net/rfc2821.html#s4.1.1.1
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2004-05-19 04:46 PDT by Scott Clark
Modified: 2009-01-22 10:17 PST (History)
13 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
proposed patch (7.17 KB, patch)
2006-12-20 02:33 PST, Christian Eyrich
no flags Details | Diff | Splinter Review
patch v2 (7.43 KB, patch)
2006-12-21 01:28 PST, Christian Eyrich
mozilla: review+
Details | Diff | Splinter Review
patch v3 (7.32 KB, patch)
2006-12-21 11:24 PST, Christian Eyrich
mozilla: review+
Details | Diff | Splinter Review
patch v3 -u (9.73 KB, patch)
2006-12-21 11:25 PST, Christian Eyrich
no flags Details | Diff | Splinter Review

Description Scott Clark 2004-05-19 04:46:49 PDT
User-Agent:       Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)
Build Identifier: 

When sending email via smtp in version 0.6 of Thunderbird it presents the helo 
string as 127.0.0.1

Headers from my mail server:
Received: from flintstone.scottclark.me.uk (HELO ?127.0.0.1?) (82.68.38.138)
  by topcat.thunderweb.co.uk with SMTP; 19 May 2004 11:34:52 -0000
Message-ID: <40AB4659.6030703@scottclark.me.uk>
Date: Wed, 19 May 2004 12:34:49 +0100
From: Scott Clark <mail@scottclark.me.uk>
User-Agent: Mozilla Thunderbird 0.6 (Windows/20040502)

Only noticed this since upgrading to 0.6, was ok with 0.5

Reproducible: Always
Steps to Reproduce:
1. Send an email via my server.
2.
3.

Actual Results:  
As per details.

Expected Results:  
Actual hostname of the machine or domain of email address
Comment 1 OstGote! 2004-05-19 08:03:27 PDT
I guess bug 68877 is related. 
Comment 2 Andrew Boothman 2004-06-01 17:31:38 PDT
I've noticed the same behaviour under 0.6 - it appears to be happening with 
every single email.
Comment 3 Andrew Boothman 2004-06-01 17:34:08 PDT
Sorry - meant to add that I can supply detailed logs from my mail server to 
prove that Thunderbird is doing this.
Comment 4 Scott MacGregor 2004-06-02 22:45:30 PDT
this is working as designed per Bug #68877
Comment 5 Christian Eyrich 2004-06-03 05:03:41 PDT
mscott, an IP is ok, but 127.0.0.1? Normally it should be something else.
But 127.0.0.1 is ok if TB is configured to send messages over a proxy/AV on the
local machine. Scott, Andrew, what's the SMTP server you configured in TB?
Comment 6 Scott Clark 2004-06-03 05:12:48 PDT
My mail server is setup as topcat.thunderweb.co.uk

It doesn't go through any local based AV/Firewall.

S :~>
Comment 7 Andrew Boothman 2004-06-03 07:12:38 PDT
OK - in my case Norton Antivirus was scanning all outgoing email which was 
obviously leading to Thunderbird using the 127.0.0.1 interface to talk to the 
mail server. Disabling Norton's outgoing email scanning results in the correct 
IP being used in the HELO.

I'm a little confused why Thunderbird is aware that it is speaking to the 
localhost as the mail server is still set up as my local mail server - but the 
behaviour does appear to be readily explainable.
Comment 8 Christian Eyrich 2004-06-03 08:41:19 PDT
Andrew, yes, that's interesting.
I'd have a look at the traffic with a packet sniffer like ethereal. Of course
only if you're able to interpret the data.

Scott, I've no clue why your TB writes 127.0.0.1. If you know how you can use a
sniffer to investigate the traffic and which connection is really used.
Comment 9 Scott MacGregor 2004-06-03 09:31:07 PDT
> mscott, an IP is ok, but 127.0.0.1?

I didn't understand that comment. After fixing Bug #68877, if you have a
firewall between you and your internet connection, the IP address at your end of
the smtp connection is local host (127.0.0.1). That is the value of getSelfAddr();

Christian, why do you seem suprised to see that? I've always seen it in mozilla
and thunderbird since we fixed 68877. This is not a thunderbird specific behavior. 
Comment 10 Christian Eyrich 2004-06-03 10:04:23 PDT
I'm not surprised to see 127.0.0.1 on installations where a Mozilla/TB connects
to a firewal/proxy/AV on the same machine through loopback.
But if a SMTP server out there is entered in Server Name I expect to see the IP
that the local machine has from the next hop.

In case of our local net here Mozilla uses 192.168.0.3 (we're behind a
NAT/router). If my machine is connected to modem/DSL modem and dialed in
directly, I'd expect the IP assigned by my provider (e.g. 217.187.126.236) to be
used.
Comment 11 Marcus 2005-08-22 14:22:38 PDT
I am unable to use my outgoing SMTP server due to this EHLO/HELO mismatch. I 
don't have a workaround.

(In reply to comment #0)
> User-Agent:       Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET 
CLR 1.1.4322)
> Build Identifier: 
> When sending email via smtp in version 0.6 of Thunderbird it presents the 
helo 
> string as 127.0.0.1
> Headers from my mail server:
> Received: from flintstone.scottclark.me.uk (HELO ?127.0.0.1?) (82.68.38.138)
>   by topcat.thunderweb.co.uk with SMTP; 19 May 2004 11:34:52 -0000
> Message-ID: <40AB4659.6030703@scottclark.me.uk>
> Date: Wed, 19 May 2004 12:34:49 +0100
> From: Scott Clark <mail@scottclark.me.uk>
> User-Agent: Mozilla Thunderbird 0.6 (Windows/20040502)
> Only noticed this since upgrading to 0.6, was ok with 0.5
> Reproducible: Always
> Steps to Reproduce:
> 1. Send an email via my server.
> 2.
> 3.
> Actual Results:  
> As per details.
> Expected Results:  
> Actual hostname of the machine or domain of email address

Comment 12 Randolf Richardson 2006-11-06 17:14:46 PST
There's an easy solution to this problem:  In the SMTP server configuration dialuges, add an "Advanced" feature that allows customization of the HELO/EHLO string.

Pegasus Mail ( http://www.pmail.com/ ) allows this, and it has been a very handy feature a few times where ISPs don't bother providing PTR records for their IPs, or the DNS server providing the PTR records is down.

I would like to see this feature added, as without it the result is eMail will be blocked in some circumstances.  Thanks in advance.

P.S.:  I believe the severity for this bug should be changed to "blocker."
Comment 13 Mike Cowperthwaite 2006-11-07 12:19:15 PST
Tidying this up a bit.  This came up in the newsgroups today; the poster claims that the nonrouteable IP address is against spec.  I can't say whether or not that's so, but I'll put a link to the relevant RFC section here:
  <http://rfc.net/rfc2821.html#s4.1.1.1>

Apparently there are SMTP clients which are blocking such messages.
Comment 14 Hector Santos 2006-11-07 13:05:45 PST
Mike,

The RFC indicates a SHOULD, not a MUST.  TBIRD is hardcoded to use domain IP literals and as Randolf suggested, there needs to be an option to use local computer name at a minimum.   If it contains a DOT, in the increasingly more modern ANTI-SPAM SMTP world addressing spam with tigher SMTP transactions, this is being increasingly checked, such it is by default in our Wildcat! SMTP server product.   

If it has a IP LITERAL, no question, this is a 100% RFC MUST MATCH requirement.  So it must use the proper IP literal if its going to use one at all.  It can't use intranet sub-nets addresses such as 192.* or 10.*., in fact, that could be used as intelligence to switch to a local computer name.  This is what OE does I believe since I don't see an option for it as well.  But no doubt, it uses the local computer name when there is NO PTR record for the machine.

The problem exist. You just not seeing it much because its rare to find SMTP server doing these checks.  But once more and more of them do, TBIRD will suddenly have an increase in user support issues.   

My suggestion is to simply add an option to use the local computer name or have the code automatically check for a ptr record and if not found, use the local computer name.   I prefer the option.

Hector Santos




Comment 15 Randolf Richardson 2006-11-07 14:37:44 PST
The ability to set the hostname used in the HELO/EHLO string would really be best (and it can certainly default to the local computer name).

Regarding domain literals (sometimes referred to as IP literals), this is a very important point.  Unfortunately, simply using a domain literal (and IP address within squared brackets) for the outbound hostname will prove to be problematic since there are a lot of mail servers that don't comply with RFCs and will reject these.

On a side note, the reason a domain literals are needed is that an IP address by itself will normally be handled as a hostname, but there is no purely numeric top-level domain so such a resolution will fail.  Some Operating Systems, however, when resolving hostnames will detect when an IP address is used and just return it, but this isn't considered to be good practice because it eliminates the long term possibility of having purely numeric gTLDs ranging from 0 to 255.

Example of an IP address:  10.20.30.40
Example of a domain literal:  [10.20.30.40]

Indeed, I totally agree with Hector that down the road there will be many support issues if this isn't corrected soon.  But I must stress the importance of allowing the user to configure (through Advanced settings is preferred) the hostname just in case their ISP screws things up as well (which does occur more frequently than most people seem to realize).
Comment 16 Christian Eyrich 2006-11-09 13:04:49 PST
I fear that discussion will duplicate the one we had in bug 279525.

@ Hector
> TBIRD is hardcoded to use domain IP literals

Err no, since resolving mentioned bug it's only second choice; first PR_GetSystemInfo(PR_SI_HOSTNAME_UNTRUNCATED, ...) is tried. I don't know how that one works but it should return a FQDN of the system if one is available. If none is, an address literal is being sent.

The interesting question for me is, under what circumstances is 127.0.0.1 everything what Necko sees?

Having an option to customize the HELO/EHLO parameter sounds nice at first. But wouldn't that be quite impractical in reality? Wouldn't one have to change that parameter at least daily to match the current outside name/address?
Comment 17 Hector Santos 2006-11-09 13:56:40 PST
> @ Hector
> > TBIRD is hardcoded to use domain IP literals
> 
> Err no, since resolving mentioned bug it's only second choice; first
> PR_GetSystemInfo(PR_SI_HOSTNAME_UNTRUNCATED, ...) is tried. I don't know how
> that one works but it should return a FQDN of the system if one is available.
> If none is, an address literal is being sent.

I haven't looked at the entire code, but based on what I see in nsSmtpProtocol.cpp, GetUserDomainName() is used to get the client domain name to be used in the EHLO/HELO commands.  This is first used to begin the EHLO session in ExtensionLoginResponse() and then again in SendEhloResponse() when it does a fall back to HELO.

Looks to me GetUserDomainName() is hardcoded to use domain literals.

> Having an option to customize the HELO/EHLO parameter sounds nice 
> at first. But wouldn't that be quite impractical in reality? Wouldn't 
> one have to change that parameter at least daily to match the 
> current outside name/address?

The proper SMTP client like is:

 - If the peer address is 127.0.0.1, use "LocalHost" for the
   client domain name.

 - Otherwise, perform a PTR look for the peer address.  This may return 
   more than one host name, use the first one. If a SERVER is going to
   do IP/DOMAIN validation it will do the same thing. SERVERS supporting
   SPF will expect this as well.

 - If the PTR is null, then use the LOCAL COMPUTER NAME.

This will resolve it without requiring a UI change.

But you may want to add a UI option such as:

   [X] Perform PTR lookup for HELO/EHLO

In short, only use a domain literal IFF the smtp client 100% knows for the such the PEER address is matching.

---
HLS
Comment 18 Hector Santos 2006-11-09 14:09:07 PST
Grammar fix for READABILITY with additional info:

The proper SMTP client LOGIC is:

 - If the peer address is 127.0.0.1, use "LocalHost" for the
   client domain name.

 - Otherwise, perform a PTR look for the peer address.  This may return 
   more than one host name, use the first one. If a SERVER is going to
   do IP/DOMAIN validation it will do the same thing. SERVERS supporting
   SPF will expect this as well.

 - If the PTR is null, then use the LOCAL COMPUTER NAME.

This will resolve it without requiring a UI change.

But you may want to add a UI option such as:

   [X] Perform PTR lookup for HELO/EHLO
 
In short, only use a domain literal IFF (if and only if) the smtp client know 100% for sure the PEER address is going to match.  I don't see how it can do this reliably short of simply checking for a common non-routable sub-net, like 192.*, 10.*.  In such a case, probably best to just use the Local Computer Name.

The key point is that whatever client domain name is used for the HELO/EHLO, it must be correct, otherwise use something the server is going to SKIP checking, which is a name without a DOT in it (i.e, Local Computer Name).  Once it has a dot, the bar is raised for possible validation checks.

---
HLS
Comment 19 Randolf Richardson 2006-11-09 15:17:13 PST
(In reply to comment #16)
[sNip]
> Err no, since resolving mentioned bug it's only second choice; first
> PR_GetSystemInfo(PR_SI_HOSTNAME_UNTRUNCATED, ...) is tried. I don't know how
> that one works but it should return a FQDN of the system if one is available.
> If none is, an address literal is being sent.

This is a good idea for default behaviour, but the user should be permitted to override it.

> The interesting question for me is, under what circumstances is 127.0.0.1
> everything what Necko sees?

The circumstances can occur when the user's TCP/IP stack is screwed up, or their firewall software is messing things up, or if SpyWare is interfering, etc.

Unfortunately on Windows boxen, this is a much more common problem than most people realize, however most users just don't bother to complain and instead of complaining they will just give up and switch to some other software.  Don't doubt for a moment that users will switch back to their beloved OutLook or OutLook Express programs because I've seen it happen a few times already when the user was frustrated that something wasn't working.

The feature I'm recommending is just one solution to limit this problem further.

> Having an option to customize the HELO/EHLO parameter sounds nice at first.

No comment, because suddenly I'm getting very frustrated now (please read on)...

> But wouldn't that be quite impractical in reality? Wouldn't one have to
> change that parameter at least daily to match the current outside
> name/address?

No, it wouldn't need to be changed constantly because it can be used as a means to bypass dynamic IP filters for real clients.  The problem is that you are making an assumption about how all mail servers are configured, which I can assure you 100% is dead wrong.

I was really not expecting that I'd have to bring this up, but I don't see any other way because it seems like asking for a simple feature to be added is like pulling teeth!  So...

In other open source communities, suggestions are taken positively, rather than argued to the death with (in the case of simple features).  The fact that you have to come up with so many different arguments against it already demonstrates very clearly that it is a feature worth considering, otherwise a very simple argument would have put this issue to its end from the get-go.

Obviously there's a need for this feature, and it would be a very simple solution for those sufferring from the problem at the moment (which may not be fixable given the other reasons I indicated above for it occurring).  I've also already pointed out that Pegasus Mail has this option, which has been helpful a number of times so far with my own clients, but now I'm starting to get the impression (which I sincerely hope I'm dead wrong about) that you folks really don't want to add new features to Thunderbird -- that would be sad, because it would mean that I'd have to start looking for yet another alternative to OutLook for my clients, and I really don't want to do that because I like what Mozilla Thunderbird (with the Lightning Extension) does now in all respects, including replacing MS-OutLook.

At this point, I'm now left feeling very frustrated because of the way this problem has been handled.  I sincerely hope that I will have a reason very soon to not feel frustrated anymore.

So, once again, please add the following feature:  In the Advanced section of the SMTP server, an option to specify a custom HELO/EHLO message.  255 bytes should be reasonable size since section 4.5.3.1 of RFC 2821 specifies a maximum length of 255 characters for a domain name.

Thanks in advance.
Comment 20 Randolf Richardson 2006-11-09 15:53:41 PST
(In reply to comment #18)
[sNip]
> The key point is that whatever client domain name is used for the HELO/EHLO,
> it must be correct, otherwise use something the server is going to SKIP
> checking, which is a name without a DOT in it (i.e, Local Computer Name).
>  Once it has a dot, the bar is raised for possible validation checks.

That's yet another assumption about the way all mail servers are configured, but unfortunately this doesn't leave room for exceptions that an SMTP server administrator may choose to make for certain hostnames.  So far, this has been a tremendously useful work-around for my Pegasus Mail users (still combined with SMTP authentication, of course), and I'd love to be able to do the same with Thunderbird.

Thanks in advance.
Comment 21 Christian Eyrich 2006-11-09 16:04:33 PST
> I haven't looked at the entire code, but based on what I see
> in nsSmtpProtocol.cpp, GetUserDomainName()

Yes, GetUserDomainName() doesn't get the user domain name but only address literals. But in bug 279525 the new main handling code was added in some other place - unfortunately. See nsSmtpProtocol::ExtensionLoginResponse() especially lines from 525 on.

> The proper SMTP client like is:

Maybe I don't understand RFC 2821 correctly, but I read 3.6 and 4.1.1.1 that neither "localhost" nor "LocalHost" or any local computer name are allowed in the EHLO parameter. So a (local) address literal seems better to me than some fantasy name.
Comment 22 Christian Eyrich 2006-11-09 16:04:53 PST
Randolf, it might be you're frustrated or whatever. But you can't push me into doing something whose reason or correctness I don't see or even doubt.
And even if you could, I'm not a person that has something to say or that can make the decision if a patch goes in.

I can write and submit a patch here the same way you could. If you don't know the code and I don't understand why this or that could help, then either you learn to write the code yourself, you've to live with my questions, find someone else to write a patch (would be ok for me) or switch the client.



> The problem is that you are making an assumption about how all mail
> servers are configured

No, I don't but want to understand you they are and why something would help. And don't want to intentionally violate any RFC.

Let me try it that way:
From your point of view the problem is, that [127.0.0.1] is treated as spammer like behaviour by some servers, yes?

Why does it work in Outlook? What does Outlook send?
What would you enter in the custom domain name field? The literal for the IP that your mailserver thinks you are? Some fantasy name? Are you assigned a static or dynamic IP?
Comment 23 Christian Eyrich 2006-11-09 16:18:23 PST
Hey, that's interesting: bug 68877, comment #119 about having a fill-in domain name and #122 that maybe enlightens my syndrome on this topic. :)
Comment 24 Randolf Richardson 2006-11-09 21:41:21 PST
(In reply to comment #22)
> Randolf, it might be you're frustrated or whatever. But you can't push me into
> doing something whose reason or correctness I don't see or even doubt.
> And even if you could, I'm not a person that has something to say or that can
> make the decision if a patch goes in.
> 
> I can write and submit a patch here the same way you could. If you don't know
> the code and I don't understand why this or that could help, then either you
> learn to write the code yourself, you've to live with my questions, find
> someone else to write a patch (would be ok for me) or switch the client.

Sadly, Mozilla is looking more and more like "just another Open Source Software project run code-it-yourself-or-go-away snobs."  I'm not a C/C++ programmer as I've specialized in Assembler, Perl, and Java, so I find your "learn to write the code yourself" comment very discouraging -- does this reflect Mozilla's official policy on how to handle suggestions?  If so, it's Mozilla's loss because Open Source Software project that view suggestions from users as valid contributions have far more potential for success as they gain more end-user loyalty in the long term (which obviously can lead to more donations, word-of-mouth promotion, etc.).

Druid ( http://www.sourceforge.net/projects/druid/ ) is one example of an Open Source Software project that views end-user suggestions as valued contributions, and even though they don't implement everything that is suggested they do pay attention to their users and have a top-notch application that renders most of the commercial counterparts as either redundant or lagging behind.  I'm suprised at the arrogant "learn to code" attitude that I've received here (and you wonder why I'm frustrated?).

> > The problem is that you are making an assumption about how all mail
> > servers are configured
> 
> No, I don't but want to understand you they are and why something would help.
> And don't want to intentionally violate any RFC.

Having it as an option, you can even include a warning that there is a potential to violate RFC 2821 section XYZ if you like.  I think it's fantastic that the RFCs are being taken seriously, but inclusion of this feature doesn't necessarily mean the RFC will be violated (while, on the other hand, using the local computer name is more likely to result in an RFC violation).

> Let me try it that way:
> From your point of view the problem is, that [127.0.0.1] is treated as spammer
> like behaviour by some servers, yes?

By some servers, yes (and possibly blocked for other reasons as well, such as when dealing with a non-RFC-compliant server whose developer doesn't know anything about domain literals and assumed that "[" and "]" are not valid characters).

> Why does it work in Outlook? What does Outlook send?

As I recall, both MS-OutLook and MS-OutLook Express use the Windows Computer Name for their HELO/EHLO strings.  Unfortunately, many organizations are unwilling to change their local computer names when they're using a Microsoft Domain or Microsoft Active Domain scheme to manage their internal network, thus this becomes yet one more reason NOT to use Thunderbird.

> What would you enter in the custom domain name field? The literal for the IP
> that your mailserver thinks you are? Some fantasy name? Are you assigned a
> static or dynamic IP?

In my system, I use the client's internet domain name which is resolvable (most SMTP servers that check validity of claimed hostnames on the HELO/EHLO string are simply checking if it can be resolved, without bothering to compare the resulting IP address {probably because an FQDN can have multiple A and/or AAAA records associated with it, and to make the check even more complicated a CNAME record can result in additional processing overhead; in short, it's a mess, and easier to just check if the name resolves to something}).

Other systems that I know of will use a special code-word.

These SMTP servers used for sending eMail (because they're hosted by a third party such as myself) use this HELO/EHLO string to skip blacklist and filter checks (SMTP Authentication is still required on top of this anyway, but the point of this configuration is that it makes it possible to implement DNSBLs immediately after the HELO or EHLO command to reject at the SMTP envelope stage -- the earlier, the better, because spammers waste a lot of bandwidth).

Currently, with other eMail clients like Pegasus Mail, it's easy to just configure this string for multiple users.  For others, such as MS-OutLook and MS-OutLook Express, it's a nuisance because we have to know what the computer name is and program that into our servers.  On a system with thousands (or even tens of thousands) of users, this becomes a very tedious management nightmare in very little time.

Including this "custom hostname to use in HELO/EHLO" advanced feature in Thunderbird makes it easier to integrate into existing systems, and more flexible when replacing insecure legacy software such as MS-OutLook and MS-OutLook Express.

Given the number of screwed up anti-virus/firewall applications, broken TCP/IP stack implementations, and SpyWare interference, there's no question in my mind that this feature needs to be there.  The fact that Thunderbird is sending "[127.0.0.1]" makes it very clear the current scheme isn't working.  In short, this is the best and easiest solution because it also offers a great deal of flexibility to the highly qualified/experienced technical support people.
Comment 25 garth 2006-11-15 23:00:55 PST
I agree with the users' suggestions as ultimately it's the users who use the program.

The problem with the mozilla open source movement is there is no co-ordination of information and no review of comments.

This is where windows live is really taking off and they are not only making money by providing free support, the products on offer are getting better and more diverse as time goes on.
Comment 26 Christian Eyrich 2006-11-23 07:24:50 PST
Ok, for now I've a working patch that enables the user to assign a custom Hello argument for each SMTP server that's configured. I don't post it here for now since I guess you won't be happy with it - it has no UI.

Am I right that you guys want a UI? If yes, the question is where and how should it be named. The concerns and objection I know of from an earlier try are, that the normal user could be confused from another field, especially if it's optional. SMTP pane currently has no "Advanced" button to hide anything.
IIRC concerns were raised that even an "Advanced" button could confuse people.

Another question. Currently that pref must be set and filled out for each server and for each newly added one. Would a default make sense that's used if the pref for the specific not changed/cleared?
Comment 27 Randolf Richardson 2006-11-23 13:56:28 PST
(In reply to comment #26)
> Ok, for now I've a working patch that enables the user to assign a custom
> Hello argument for each SMTP server that's configured. I don't post it
> here for now since I guess you won't be happy with it - it has no UI.

Thanks!  I'm confident that this will be an excellent start for a quick-fix solution for those who need it right now.

When adding the UI bits, please make sure this is labelled as follows so that SMTP server administrators will immediately recognize it:

Use HELO/EHLO hostname: _____________________

When empty, a greyed out "(system default)" should appear inside, or on the right-hand side, of the text entry box.

Don't worry about data type checking, and please allow for a minimum of 64 characters since some hostnames can get quite lengthy.

> Am I right that you guys want a UI? If yes, the question is where and how
> should it be named. The concerns and objection I know of from an earlier try
> are, that the normal user could be confused from another field, especially if
> it's optional. SMTP pane currently has no "Advanced" button to hide anything.
> IIRC concerns were raised that even an "Advanced" button could confuse people.

Ah, but there is an "Advanced" button (see directions below...), and in my experience a typical end user stays away from such things (which is why I suggested placing it in there).

Placing it in the UI will greatly simplify over-the-phone support.  Where I firmly believe that it is most suitable is in the "Advanced" settings dialogue, which can be found here:

0. "Account Settings" menu item (in the "Tools" menu as I recall; please correct me if I'm mistaken)
1. "Outgoing Server (SMTP)" category
2. "Advanced" button

> Another question. Currently that pref must be set and filled out for each
> server and for each newly added one.  Would a default make sense that's
> used if the pref for the specific not changed/cleared?

That's a great idea, but only if using another SMTP Server entry with the same hostname as the default.

Rational:  Each SMTP server is configured with different policies, so copying this setting as a default for a different SMTP server could pose a potential security risk if the custom HELO/EHLO string is used as part of an authentication scheme.

Thanks again for getting a start on adding this feature.  I'm so happy about this.  By the way, I'm ~90% convinced someone else I know to switch their entire company from OutLook to Thunderbird -- the Lightning Extension (from the Sunbird project) makes it so easy to convince people to switch.
Comment 28 Christian Eyrich 2006-11-24 04:55:46 PST
> Use HELO/EHLO hostname: _________________

I for myself ignore options I don't understand. And I don't have the experience with other users. But I'd like to get some more input from different sides before I implement something.

> When empty, a greyed out "(system default)" should appear inside

"System default" won't help to know what will be sent. But I think that's a good idea for making clear that not nothing will be sent.

> Where I firmly believe that it is most suitable is in the "Advanced" settings
> dialogue, which can be found here:

Am I mad? I cannot find such a dialogue (neither in TB 1.5, 2, 3 and SM 1.1).
Not that it's impossible to add one, but I fear everything new makes it harder getting that feature in. And first we should try to get the same base for what we're talking about. :)
Comment 29 Hector Santos 2006-11-27 18:15:36 PST
In case anyone is interested, I've been in conversation with John Klensin regarding the Domain Literal Issue.  The good news is that the issue has been brought to his attention and he has asked me to write up an IETF I-D (individual draft) discussion the issue.

The thrust of the issue is that more and more servers are starting to enforce domains in SMTP sessions.  So it is important that TB use the proper public FDQN or address literal with the proper public address and not a private address.

My proposal is for servers to be "more relaxed" when the client is in AUTHENTICATION MODE.  In other words, if the client logs in, then there is less reason to do a domain client for the EHLO/HELO client domain name.

This is how we were able to make TB work against our own Wildcat! SMTP server by using the SUBMIT protocol and relaxing the EHLO/HELO domain checking.

---
HLS
Comment 30 Randolf Richardson 2006-11-29 16:40:06 PST
(In reply to comment #28)
> > Use HELO/EHLO hostname: _________________
> 
> I for myself ignore options I don't understand. And I don't have
> the experience with other users.  But I'd like to get some more
> input from different sides before I implement something.

Placing it under an "Advanced" button was just a suggestion.  I think you're right if you're implying that regular end-users typically won't venture into the SMTP Server settings on their own.

> > When empty, a greyed out "(system default)" should appear inside
> 
> "System default" won't help to know what will be sent. But I think
> that's a good idea for making clear that not nothing will be sent.

This is where the help screens come into play -- when someone presses F1 for help, the description for this field can read:

"Be default, Thunderbird will identify itself using your computer's local hostname (system default).  You can override the system default by specifying a hostname, which may be required by some ISPs."

> > Where I firmly believe that it is most suitable is in the
> > "Advanced" settings dialogue, which can be found here:
> 
> Am I mad? I cannot find such a dialogue (neither in TB 1.5, 2, 3 and SM 1.1).
> Not that it's impossible to add one, but I fear everything new makes it harder
> getting that feature in. And first we should try to get the same base for what
> we're talking about. :)

No, you're not mad, and that's my professional opinion as a computer networking consultant (although not as a psychiatrist or pscyhologist).  =D

I checked, and you're absolutely right -- there is no "Advanced" button.  It used to be there though, and I found a screen shot with Google.Com just minutes ago that shows what I was remembering:

http://www.cae.wisc.edu/img/wikiimg/nodes/smtpauthfiles/thunderbird-pop-5.gif
Comment 31 Randolf Richardson 2006-11-29 16:47:16 PST
(In reply to comment #29)
> In case anyone is interested, I've been in conversation with John Klensin
> regarding the Domain Literal Issue.  The good news is that the issue has been
> brought to his attention and he has asked me to write up an IETF I-D
[sNip]

Thanks!

> My proposal is for servers to be "more relaxed" when the client is in
> AUTHENTICATION MODE.  In other words, if the client logs in, then there is
> less reason to do a domain client for the EHLO/HELO client domain name.

Keep in mind that this will require mail server software vendors to update their software to perform hostname validity checking during or after the SMTP AUTHentication stage.  The end result will be that users will be told to use some other eMail software since the administrator is very unlikely to see this as justification to switch to a different SMTP server.

> This is how we were able to make TB work against our own Wildcat! SMTP server
> by using the SUBMIT protocol and relaxing the EHLO/HELO domain checking.

...and also have the effect of blocking less spam.
Comment 32 Hector Santos 2006-11-29 23:34:20 PST
(In reply to comment #31)

> Keep in mind that this will require mail server software vendors to update
> their software to perform hostname validity checking during or after the SMTP
> AUTHentication stage.  The end result will be that users will be told to use
> some other eMail software since the administrator is very unlikely to see this
> as justification to switch to a different SMTP server.

Not sure if you followed me here.

The I-D will propose that the SMTP client machine domain validation be relaxed under the SUBMIT PROTOCOL (port587) conditions only.  

In general, there is no "legal" or "technical" requirement to validate the domains when using PORT 25.  The SUBMIT protocol gives operators the technical and legal authority to do the validation and enforce it as well (reject) and this is because it enforces ESMTP AUTH.  With PORT 25, ESMTP AUTH is an option. Not a requirement.

The proposal is to "relax" the SUBMIT PROTOCOL requirement of rejecting the client domain name checking and this is based on the premise that AUTHENTICATION is a requirement, thus there is a lesser security impact under such conditions.

> ...and also have the effect of blocking less spam.

No, because the SMTP CLIENT must authenticate under the SUBMIT protocol.  Using Port 587 requires the SMTP client to login. No exception.  Hence in theory there is an lesser security issues involved under an authenticated environment.

Remember TB is not the only client with this issue.  Outlook has the same problem and all other MUA will have the same issue if they don't offer a manual  override to define the NAT machine domain  Hence, as a SMTP vendor I have to work in a general manner to help resolve all clients related issues, including TB.

We already implemented this proposal for our own server and it works really well. Now, I personally been able to use TB on port 587 and login on a private side workstation with no FQDN.   This is going to instantly help our customers.  The proposal will highlight the issue for other vendors to consider as well.  John mentioned that this could be a fast track RFC to update the official SUBMIT PROTOCL (RFC 4409).  But that won't happen without IETF SMTP community feedback first.

---
HLS
Comment 33 Randolf Richardson 2006-11-30 00:45:21 PST
(In reply to comment #32)
> (In reply to comment #31)
> 
> > Keep in mind that this will require mail server software vendors to update
> > their software to perform hostname validity checking during or after the SMTP
> > AUTHentication stage.  The end result will be that users will be told to use
> > some other eMail software since the administrator is very unlikely to see this
> > as justification to switch to a different SMTP server.
> 
> Not sure if you followed me here.
> 
> The I-D will propose that the SMTP client machine domain validation be relaxed
> under the SUBMIT PROTOCOL (port587) conditions only.  

Why only port 587?  This could exclude sites that have SMTP server software that handles SMTP AUTH as part of the main MTA but are not listening on port 587 for any number of reasons.  One of the most common reasons could be due to a lack of functionality in the SMTP server daemon in that it can only be configured to listen on one TCP port.

> In general, there is no "legal" or "technical" requirement to validate the
> domains when using PORT 25.  The SUBMIT protocol gives operators the technical
> and legal authority to do the validation and enforce it as well (reject) and
> this is because it enforces ESMTP AUTH.  With PORT 25, ESMTP AUTH is an option.
> Not a requirement.

I agree that it's not a requirement.  That doesn't change the fact that many SMTP servers are configured today to operate solely on TCP port 25.

> The proposal is to "relax" the SUBMIT PROTOCOL requirement of rejecting the
> client domain name checking and this is based on the premise that
> AUTHENTICATION is a requirement, thus there is a lesser security impact under
> such conditions.
> 
> > ...and also have the effect of blocking less spam.
> 
> No, because the SMTP CLIENT must authenticate under the SUBMIT protocol.  Using
> Port 587 requires the SMTP client to login. No exception.  Hence in theory
> there is an lesser security issues involved under an authenticated environment.

I don't agree with the point about security because many SMTP servers refuse to relay mail unless the client used SMTP AUTH successfully.

> Remember TB is not the only client with this issue.  Outlook has the same
> problem and all other MUA will have the same issue if they don't offer a manual
>  override to define the NAT machine domain  Hence, as a SMTP vendor I have to
> work in a general manner to help resolve all clients related issues, including
> TB.

Does that mean future Mozilla Thunderbird development is to be based entirely on Outlook in a monkey-see-monkey-do fashion?  If so, then what's the point of using Thunderbird instead of Outlook if the goal is to just copy Outlook?

> We already implemented this proposal for our own server and it works really
> well. Now, I personally been able to use TB on port 587 and login on a private
> side workstation with no FQDN.   This is going to instantly help our customers.

That's excellent.

>  The proposal will highlight the issue for other vendors to consider as well. 
> John mentioned that this could be a fast track RFC to update the official
> SUBMIT PROTOCL (RFC 4409).  But that won't happen without IETF SMTP community
> feedback first.

This is a really good idea, so long as it isn't a requirement.  SMTP server operators and vendors should not be forced to change their ways just to accomodate an eMail client application.

For some perspective, consider the following checklist which is regularly used to prove flaws in every so-called "ultimate and final" solution to the spam problem that has ever been proposed:

http://www.claws-and-paws.com/fussp.html

Although that list doesn't entirely apply to what you're proposing, some of the checklist items apply, and one in particular comes to mind:

"Huge existing software investment in SMTP."

Since the relay problem has been resolved in the vast majority of SMTP server software without moving relaying and SMTP AUTH functions exclusively or additionally to port 587, it simply isn't logical to assume that SMTP server vendors will be willing to make the change just because an eMail client requires it.

The most important thing remember is that more flexibility will make Thunderbird a better option for users, especially if that flexibility makes it possible to fit into every existing environment.
Comment 34 Hector Santos 2006-11-30 01:23:18 PST
(In reply to comment #33)
> Why only port 587?  This could exclude sites that have SMTP server software
> that handles SMTP AUTH as part of the main MTA but are not listening on port
> 587 for any number of reasons.  One of the most common reasons could be due to
> a lack of functionality in the SMTP server daemon in that it can only be
> configured to listen on one TCP port.

Two reasons; Politics and Chicken and Egg problem.

Port 587 implies authentication so this will be enough of a trigger to tell the MSA to avoid EHLO/HELO checking.  With PORT 25, it might be too late because authentication will not happen to AFTER the initial EHLO server response.  Doing it this way will require more server side code change since it will need to delay validation until it is known where a AUTH session will take place.

Politics because knowing my peers, they (including myself) will be resistant to relaxing SMTP compliance requirements especially FQDN or invalid client domain requirements.  Port 587 may be the compromise under these special cases and the realization that there is a growth market of MUA residing behind a NAT on SOHO workstations.

> I agree that it's not a requirement.  That doesn't change the fact that many
> SMTP servers are configured today to operate solely on TCP port 25.

Thats true, but as I indicated above, this will require a more complex software change requirement of delaying the validation UNTIL the authentication state is known.

In addition, again from a political standpoint, you may not get the "sympathy" of the SMTP community or endorsement and you should not be surprise to hear the protocol gods with strong voices in the IETF say "Tough ****! Fix your own bugs! or get a real MTA or proxy setup" to the MUA developers who have this problem.

The key is to convince enough to say that its not just a TB problem, but an overall MUA issue for a new growth market.

Please note I am not against the idea of providing recommendations for PORT 25 as well. But it will be a tougher sell.

> I don't agree with the point about security because many SMTP servers 
> refuse to relay mail unless the client used SMTP AUTH successfully.

(Note: Its ESMTP AUTH, not SMTP AUTH)

I don't see how that statement came into play or how it related to what I said.  Nonetheless, the BCP for all SMTP vendors to operate in this matter.  In general, authentication is all that required for routing mail. Local Submissions do not require authentication, and authentication can come a various forms, not just ESMTP AUTH.

> Does that mean future Mozilla Thunderbird development is to be based entirely
> on Outlook in a monkey-see-monkey-do fashion?  If so, then what's the point of
> using Thunderbird instead of Outlook if the goal is to just copy Outlook?

I have no idea what you are talking about and I am not interesting in continueing this discussion with you if you keep throwing in these red-herrings.

Soon enough I will be submitting the I-D and I'm quite sure Klensin will ask for discussion in the IETF-SMTP and IETF-SUBMIT mailing list.  I encourage you to participate when that happens to get in your input.

> This is a really good idea, so long as it isn't a requirement.  SMTP server
> operators and vendors should not be forced to change their ways just to
> accomodate an eMail client application.

Of course not. Local Policy always prevails. SMTP vendors are not in the business of stopping the flow of mail, yet, the new era requires stronger SMTP compliance to address the legacy market exploitations.  

> Since the relay problem has been resolved in the vast majority of SMTP server
> software without moving relaying and SMTP AUTH functions exclusively or
> additionally to port 587, it simply isn't logical to assume that SMTP server
> vendors will be willing to make the change just because an eMail client
> requires it.

No one is making a presumption anyone will change anything.  A proposal is just that - a proposal.  If a vendor is interested and it fits the bill for them, they will probably adapt. But there is no assumption everyone will adapt.  No, that is why TB still has to do its part.  The odds will be very high it will still run across a server were it isn't so relaxed or has adopted to any new SUBMIT protocol updates.

> The most important thing remember is that more flexibility will make
> Thunderbird a better option for users, especially if that flexibility makes it
> possible to fit into every existing environment.

True, and the only concern I have for the Manual field proposal is that the USER may screw up.  But he will see that quickly. Although it will be manually provided, it still needs to be a correct FQDN or address literal.    In this case, the "Help" should say "this field is to put the FQDN of the machine directly behind the NAT where the wire is connected to.  Don't put just any domain here."

---
HLS
Comment 35 Mike Cowperthwaite 2006-11-30 08:42:50 PST
Does the suggestion at bug 279525 comment 21 apply to this bug?  Is it essentially correct?  It has the advantage of being relatively simple.
Comment 36 Hector Santos 2006-11-30 10:52:57 PST
(In reply to comment #35)
> Does the suggestion at bug 279525 comment 21 apply to this bug?  Is it
> essentially correct?  It has the advantage of being relatively simple.

If I follow the history, I think this is the patch that was added to the trunk to include the FQDN as part of the EHLO/HELO iff it exist. Otherwise it falls back to the bracket address literal.

Yes, this is first basic requirement and part of the total fix.  Note, it doesn't exist in the current production release.  Chris presented the source to me that included this fix.

However, it doesn't solve the total problem.

The problem is with a "2nd" or Workstation that is on a private network not directly connected to the internal.  The problem is that we are seeing a growth market of SOHO (Small Office/Home Office) and/or end-users who sign up to the internet typically started with 1 PC where the NAT or DCHP assigned IP is located. Hence the MUA client has no problem with obtaining the FQDN or as with the current TB, use the proper public side address literal.

But then the user begin to add additional PC to the SOHO network and now begin to install TB on these machines.  Now the pubic FQDN or a public machine address is not available. With TB, it will use the private address literal.  The trunk with the FQDN logic, although correct, will return NULL and fall back to using the inproper private side address literal.

So this is where having the MANUAL field will help because now the user will be required to set the FQDN of the gateway or NAT machine.

The only other alternative is that users must use:

a) Smart Host or server with does not do EHLO/HELO checking,
b) NAT SMTP proxy,
c) SUBMIT Protocol.

With C, this is currently the push by ISPs to get their users to submit mail outside a blocked PORT 25.

However, the SUBMIT protocol requires all domains to be valid, including the EHLO/HELO field.

Hence, there is a contention between the wishes of the ISP and the need to perform stronger SMTP compliance.

My proposal is to allow the SUBMIT protocol to relax the initial EHLO/HELO checking because when the SUBMIT protocol is used, authentication (login) is a requirement.  Hence this allows the opportunity to be relaxed with this one aspect of the MUA behavior.

---
HLS
Comment 37 Randolf Richardson 2006-12-19 09:20:13 PST
(In reply to comment #36)
> (In reply to comment #35)
> > Does the suggestion at bug 279525 comment 21 apply to this bug?  Is it
> > essentially correct?  It has the advantage of being relatively simple.
> 
> If I follow the history, I think this is the patch that was added to the trunk
> to include the FQDN as part of the EHLO/HELO iff it exist. Otherwise it falls
> back to the bracket address literal.
> 
> Yes, this is first basic requirement and part of the total fix.  Note, it
> doesn't exist in the current production release.  Chris presented the source
> to me that included this fix.
> 
> However, it doesn't solve the total problem.
> 
> The problem is with a "2nd" or Workstation that is on a private network not
> directly connected to the internal.  The problem is that we are seeing a
> growth market of SOHO (Small Office/Home Office) and/or end-users who sign
> up to the internet typically started with 1 PC where the NAT or DCHP
> assigned IP is located. Hence the MUA client has no problem with obtaining
> the FQDN or as with the current TB, use the proper public side address
> literal.
> 
> But then the user begin to add additional PC to the SOHO network and now
> begin to install TB on these machines.  Now the pubic FQDN or a public
> machine address is not available. With TB, it will use the private address
> literal.  The trunk with the FQDN logic, although correct, will return NULL
> and fall back to using the inproper private side address literal.
> 
> So this is where having the MANUAL field will help because now the user will
> be required to set the FQDN of the gateway or NAT machine.

That's what myself and others are hoping for.  I view this as justification for implementing the suggested solution.

> The only other alternative is that users must use:
> 
> a) Smart Host or server with does not do EHLO/HELO checking,
> b) NAT SMTP proxy,
> c) SUBMIT Protocol.

An additional option "d" would be the simplist and most flexible solution to this problem that would ensure that Mozilla ThunderBird is a product that meets everyone's needs:

d) Advanced option to customize the HELO/EHLO string.

> With C, this is currently the push by ISPs to get their users to submit mail
> outside a blocked PORT 25.

Which ISPs?  I'm aware that some are promoting the use of this protocol, but there are many more who don't even know that it exists.

> However, the SUBMIT protocol requires all domains to be valid, including the
> EHLO/HELO field.

It also requires ISPs to update/reconfigure their SMTP servers, and then many hosting companies that provide SMTP service will have to do likewise.  The fact that very few such hosting companies (at least all those that I deal with on a regular basis don't, although I know that there are some that do) support the SUBMIT protocol means that they'll have to update/reconfigure their SMTP servers too.

> Hence, there is a contention between the wishes of the ISP and the need to
> perform stronger SMTP compliance.

I agree with that, but since SMTP client software that supports SMTP Authentication typically sends on TCP port 25 by default (sometimes with no option to change the port number), I expect there to be a lot more contention since these types of changes usually confuse and frustrate users -- do you really think ISPs want to hire additional help desk support people to facilitate this change of a currently working system, especially in the current climate of cut-throat pricing tactics that lead to less profitability?

Do you know that it's already hard enough to get ISPs to provide support for eMail software configuration when using ThunderBird?  They always say "Oh, we don't support ThunderBird, sorry but you'll have to use OutLook" because that's all they seem to know how to use.  If you folks don't make ThunderBird customizable enough (and sit here and continue to come up with reasons why NOT to include a useful feature), then this will just be one more reason for ISPs not to be motivated to support ThunderBird.

> My proposal is to allow the SUBMIT protocol to relax the initial EHLO/HELO
> checking because when the SUBMIT protocol is used, authentication (login) is a
> requirement.  Hence this allows the opportunity to be relaxed with this one
> aspect of the MUA behavior.

I think that's a good proposal, but it doesn't address the current need which is to simply provide a means for the user to customize the HELO/EHLO string used in the SMTP transaction when sending eMail.

I'm left with the impression here that you folks really don't want to implement new features into the product.  I've suggested this fix consistently from the beginning, yet a variety of justifications NOT to implement it has come up, and the bottom line is that, as a user (and someone who has been faithfully promoting ThunderBird with the Lightning extension in order to get users switched away from OutLook), this software remains broken.

Please fix it.
Comment 38 Brendan Eich [:brendan] 2006-12-19 10:36:42 PST
Cc'ing one of the Thunderbird owners, and the Necko owner.  Confirming.

Randolf, when you write "you folks", you are not addressing people who actually maintain the code.  I'm glad to have an informed debate here, and anyone can try to write a patch, but this bug needs to have the relevant module owners cc'd for it to get anywhere. 

/be
Comment 39 David :Bienvenu 2006-12-19 10:44:49 PST
I'm happy to take a patch that allows the user to set a pref specifying the HELO/EHLO string.

However, since we're in a string freeze for 2.0, it's very difficult to add UI for this in 2.0 - it could be set with the config editor, however.
Comment 40 Brendan Eich [:brendan] 2006-12-19 11:54:57 PST
UI-less pref will help those who know enough to want to set this, for sure.  Still time for Tb2, I take it.  David, can you take assignment of this bug?

/be
Comment 41 David :Bienvenu 2006-12-19 11:57:48 PST
I'll take this, but I'd love to see a patch from someone else :-) We should shoot for beta2, which will come out in very early Jan.
Comment 42 Christian Eyrich 2006-12-20 02:33:03 PST
Created attachment 249238 [details] [diff] [review]
proposed patch

So this is the patch I already mentioned. No UI, one pref per server.

Also did some cleanup and concentrated all hello argument stuff in one function.
Comment 43 David :Bienvenu 2006-12-20 09:18:15 PST
Thx very much for the patch, Christian!

If you change nsISmtpServer.idl, you need to change the uuid.

So, do we think that this pref should be per-server, or should it be global to smtp? One advantage of making it global is that it's much easier to tell people how to change a global pref. If it has to be per-server, then we can do that, but I just want to make sure that's the case.
Comment 44 Christian Eyrich 2006-12-20 09:39:58 PST
Uh sorry, I'll change the uuid.

I recently made a small poll in a newsgroup and people wanted a per server pref. And from what I read here that HELO argument is used for part of authentication, I guessed it should be per server. But lets hear what the others say.
Comment 45 Hector Santos 2006-12-20 09:50:42 PST
(In reply to comment #44)
> Uh sorry, I'll change the uuid.
> 
> I recently made a small poll in a newsgroup and people wanted a per server
> pref. And from what I read here that HELO argument is used for part of
> authentication, I guessed it should be per server. But lets hear what the
> others say.
> 

My input:

Make it per server, and I like the non-UI approach since under normal circumstance this a transparent protocol entity.  So we are dealing here only with the exception to the rule with is ok by me to make it part of the non-UI setup.  Buts my feeling about user ergonomics. What they don't need, don't present.

---
HLS
Comment 46 Randolf Richardson 2006-12-20 14:57:17 PST
(In reply to comment #45)
> (In reply to comment #44)
> > Uh sorry, I'll change the uuid.
> > 
> > I recently made a small poll in a newsgroup and people wanted a per server
> > pref. And from what I read here that HELO argument is used for part of
> > authentication, I guessed it should be per server. But lets hear what the
> > others say.
> 
> My input:
> 
> Make it per server, and I like the non-UI approach since under normal
> circumstance this a transparent protocol entity.  So we are dealing here only
> with the exception to the rule with is ok by me to make it part of the non-UI
> setup.  Buts my feeling about user ergonomics. What they don't need, don't
> present.

I agree with per-server.  To keep those concerned about user ergonomics pleased, one option might be to have new servers default to a previous server setting automatically upon the first time this particular setting is modified (but that's just a minor detail).

Thanks a lot folks, for putting this feature in.  It will be a great help.
Comment 47 Christian Eyrich 2006-12-21 01:28:24 PST
Created attachment 249341 [details] [diff] [review]
patch v2

Ok, so will see if this version with changed uuid gets review.
Comment 48 David :Bienvenu 2006-12-21 08:10:26 PST
Comment on attachment 249341 [details] [diff] [review]
patch v2

this looks good - a few nits:

nsSmtpProtocol::GetHelloArgument

looks like it should be called AppendHelloArgument.

It looks like a couple tabs might have snuck in.

Re nsSmtpServer::GetHelloArgument(char * *aHelloArgument)

Does it make sense to check the defBranch first? I.e., are there some users who are going to want to set up the default to use across smtp servers? If that's a reasonable thing, then it's also easier to tell people how to set that default pref (you don't need to know the number of your smtp server)
Comment 49 Christian Eyrich 2006-12-21 09:13:09 PST
(In reply to comment #48)
> nsSmtpProtocol::GetHelloArgument
>
> looks like it should be called AppendHelloArgument.

I also thought about that - ok changed that.

> It looks like a couple tabs might have snuck in.

No, they're right but the lines around also need indentation. In the -u patch it's all ok.

> Does it make sense to check the defBranch first? I.e., are there some users
> who are going to want to set up the default to use across smtp servers?

Don't know if having an default is interesting.
But if, do you really mean checking first, not after the specific one?

And what I just noticed. We don't need a SetHelloArgument, right? If necessary
for a later UI, we can it implement then, no?
Comment 50 David :Bienvenu 2006-12-21 09:31:11 PST
d'uh, of course, only get the default if the actual pref isn't set...

right, we don't need a setter for helo argument
Comment 51 Christian Eyrich 2006-12-21 11:24:18 PST
Created attachment 249367 [details] [diff] [review]
patch v3

Renamed function to AppendHelloArgument, added evaluation of default pref and made helloArgument readonly.
Comment 52 Christian Eyrich 2006-12-21 11:25:21 PST
Created attachment 249368 [details] [diff] [review]
patch v3 -u

-u version of patch v3.
Comment 53 David :Bienvenu 2006-12-21 11:27:02 PST
Comment on attachment 249367 [details] [diff] [review]
patch v3

looks good, Christian, thx a lot! I'll land this on the trunk (or rather, the -u diff)
Comment 54 Hector Santos 2006-12-21 11:48:57 PST
(In reply to comment #52)
> Created an attachment (id=249368) [edit]
> patch v3 -u
> 
> -u version of patch v3.
> 

Christian,

First, I, for one, appreciate your effort here.

Two things regarding the HELO/EHLO field and code change:

1) PER SERVER non-UI option 

On second thought about the Per Server change, as far as I see, I see no reason to have this on a per server basis.  Under normal situations where this is going to be necessary, there will be ONE "gate" to the outside world from the private network and it is this "gateway" FQDN that is required to be known and manually set at the TBIRD workstation.  I don't care either way, but I see no practical reason that this would be on a per outbound server basis.  

2) Full Source Code 

I hope it isn't asking alot, but since I do not have the complete source code setup facility here, would it be asking alot if you made the full source for the  specific modules being changed?  I would be able to give you a better review than looking at the diffs. 

I ask only because it seems you went alittle further with code changes here and I believe you already resolve the last issue regarding getting FQDN first before using the IP literal, including resolving the QUIT issue.   This bug fix would basically complete the 3 basic issues I found with TBIRD and it looks with this final change, I want to make sure all 3 are resolved.

So if its easy and possible to send me the source modules for review, I can help provide my input.

Thanks

---
HLS
Comment 55 Brendan Eich [:brendan] 2006-12-21 12:06:05 PST
HLS: The folks in that newsgroup who wanted per-server pref need to be heard from.

Also, you can see the patch in context by clicking on the Diff link, on the right in the Attachment table.  From there, you can rediff at larger context or click on File to get a full-file diff (this is at the top, starting at "Context:").  You can also click on the file path or line ranges to get views into the current rev of the source.

You should not need full source file attachments; patches are better.

/be
Comment 56 Hector Santos 2006-12-21 12:22:00 PST
(In reply to comment #55)
> HLS: The folks in that newsgroup who wanted per-server pref need to be heard
> from.

Which newsgroup and do you know the subject title?

---
HLS
Comment 57 Brendan Eich [:brendan] 2006-12-21 12:29:16 PST
(In reply to comment #56)
> (In reply to comment #55)
> > HLS: The folks in that newsgroup who wanted per-server pref need to be heard
> > from.
> 
> Which newsgroup and do you know the subject title?

See comment 44 -- Christian should jump in here.

/be
Comment 58 Christian Eyrich 2006-12-21 13:13:34 PST
> > HLS: The folks in that newsgroup who wanted per-server pref need to be heard
> > from.
> 
> Which newsgroup and do you know the subject title?

Don't know if a german language newsgroup helps you: de.comm.software.mozilla.mailnews, MID <ek4g5i$7qq$1@news1.nefonline.de>

But I don't think it matters. With the optional default value you can use it as global pref, and the ones who want a per server pref just set server specific ones. So everybody gets what he wants. The code overhead basically is one function call ...
Comment 59 David :Bienvenu 2006-12-21 13:17:43 PST
Yes, I agree with Christian that the server default effectively gives us a global pref, and experience says that someone is going to want to override the pref on a per-server basis.
Comment 60 Brendan Eich [:brendan] 2006-12-21 13:21:24 PST
Yeah, we shouldn't cavil at per-server prefs with fallback on default pref, with the patch in hand.  Opinions on pref plurality may vary; so may server configs and real-world topologies.  I'll be money that users will want both levels of pref.

/be
Comment 61 David :Bienvenu 2006-12-21 14:11:48 PST
fixed on trunk, thx again, Christian. I'll land this on the 2.0 branch sometime in the next few days.

We should document this in a FAQ somewhere...
Comment 62 Hector Santos 2006-12-21 15:07:54 PST
(In reply to comment #59)
> Yes, I agree with Christian that the server default effectively gives us a
> global pref, and experience says that someone is going to want to override the
> pref on a per-server basis.
> 

While I don't it matters at this point, the client domain name has nothing to dow to the SERVER that is being contacted but rather the MACHINE that is doing the sending - the client.  This Bug fix attempts to change the client domain name of the machine connected to the outside wire where the NAT translation is being done.

So unless the USER will be also have the option to define the *route* a client will take on a outbound session from within a private network to contact an external SERVER , this per server concept has no practical benefit.  In other words, there is only ONE route.

I'm sure there are possible some other "unknown" reason someone will state, but as far the BUG FIX is concern, the fix is to address the client IP vs client domain name association SMTP requirements which is correctly violated with TBIRD.

Thats it. Nothing more.

---
HLS
Comment 63 Karl Schmidt 2006-12-30 18:33:35 PST
AppendHelloArgument shoukd be spelt as AppendHeloArgument

helo is not hello.
Comment 64 Randolf Richardson 2006-12-31 00:40:22 PST
(In reply to comment #63)
> AppendHelloArgument shoukd be spelt as AppendHeloArgument
> 
> helo is not hello.

RFC 821, section 4.1.1, paragraph 3, uses "HELLO" and specifies "HELO" in parenthesis, therefore "helo" is "hello."
Comment 65 Magnus Melin 2008-01-12 04:16:35 PST
This was also landed on branch 2006-12-22 06:05, as per comment 61.
Comment 66 Karl Schmidt 2008-01-12 10:57:54 PST
Re #64 Hello syntax once again.

First, RFC821 is obsolete and replaced by RFC 2821.

Second the hello syntax is still helo. Thus "helo" is not "hello", but "hello" is "helo" 

See page 29 RFC 2821 where RFC talks of using  the "helo  command"  not the "hello command". The helo command is in effect saying "hello", but it is still the helo command. 


The point is that the initial use of the spelling "helo" was a terrible idea as it has produced confusion and has wasted countless man days looking for this syntax spelling bug. 

I would make an alias so that either spelling works, or produce an error that points out the spelling problem. 
Comment 67 Randolf Richardson 2008-01-12 12:02:25 PST
(In reply to comment #66)
> Re #64 Hello syntax once again.
> 
> First, RFC821 is obsolete and replaced by RFC 2821.
> 
> Second the hello syntax is still helo. Thus "helo" is not "hello", but "hello"
> is "helo" 
> 
> See page 29 RFC 2821 where RFC talks of using  the "helo  command"  not the
> "hello command". The helo command is in effect saying "hello", but it is still
> the helo command. 
> 
> 
> The point is that the initial use of the spelling "helo" was a terrible idea as
> it has produced confusion and has wasted countless man days looking for this
> syntax spelling bug. 

I suspect the reason for starting out with short commands that are an exact length of 4 characters was possibly to simplify server programming.  I've noticed that some SMTP servers don't require a space after the command, which is obviously a violation of the specifications in addition to making it clear that some progrmamer made an assumption that all SMTP commands are only 4 characters in length.

> I would make an alias so that either spelling works, or produce an error that
> points out the spelling problem. 

This is a really bad idea because it's non-standard.  There is no documentation anywhere in RFC 821 nor RFC 2821 that indicates that "HELLO" is a valid command.  Of course, if you do decide to go against the standard anyway, at least have the decency to add support for these commands also:

RECIPIENT
SEND OR MAIL
SEND AND MAIL
RESET
VERIFY
EXPAND

[Sarcasm]
Whoops!  Some of those commands include 2 spaces!  Whatever shall we do?!?  Doing something like this will probably create jobs for many people, so it wouldn't be a total loss.
[/Sarcasm]

In fact, RFC 821 clearly includes in section 4.1.2 the following information that supports my rebuttle to your suggestion to add an aleas to "correct the spelling" when clearly there is no spelling mistake here -- the commands are clearly documented with their intended spellings:

"The commands consist of a command code followed by an argument field.  Command codes are four alphabetic characters.  ..."

So, if someone actually writes an eMail client or server that uses the "HELLO" command, then in addition to not complying with the well-established standards, they obviously also haven't tested their software properly and should probably be fired for incompetency.  By deviating from well-established standards that are extremely common in their use, we are clearly doing an injustice to the internet as a whole by threating its long term stability.  In short, don't do this!
Comment 68 Randolf Richardson 2008-01-12 12:19:02 PST
As far as configuration file parameters are concerned, using "Hello" is fine because it covers both the "HELO" and "EHLO" commands.  If one used the word "Helo" instead, then some administrators may wonder if "Ehlo" must also be defined.
Comment 69 Karl Schmidt 2008-01-12 18:33:36 PST
Sadly, I don't seem to have people understanding what I'm saying - I'm not talking about changing any RFC - 

I'm talking about AppendHelloArgument vs AppendHeloArgument 

This variable is an argument to the HELO command. There is no HELLO command.

Comment 70 Magnus Melin 2008-01-13 09:54:17 PST
Karl, file a new bug about that.

Note You need to log in before you can comment on or make changes to this bug.