417942 - Thunderbird sends local network (LAN) IP address

Status: RESOLVED INVALID

(MailNews Core :: Networking: SMTP, defect)

Description

[Biju]

not a big issue...

In an outgoing mail from Thunderbird we can find the senders PC's local/home network ip address.

Test
1. send mail to your self
2. receive the mail you send to you
3. select the mail in inbox
4. select from TB menu "View > Message Source"
5. look at the first "Received: from" header (usually the bottom most)
6. you will see your local LAN ip address 

If you dont see configure your outgoing mail server as gmail and test 
smtp server:   smtp.gmail.com
port:     587    
user name: your full gmail-id 
secure connection: STARTTLS

The IP address is substituted at hostname.
I would suggest makeup hostname by expression 

hostname = from_address.replace(/@/g,'.')

Comment 1

[Matthias Versen [:Matti]]

Always, always include the used Version of something you are reporting a bug.
The Ehlo command should use the local domain, see bug 279525

Comment 2

[Biju]

version 3.0a1pre (2008021603)

Comment 3

[Philip Prindeville]

Actually, the sender's IP address is present if the machine doesn't have a hostname w/ domain set (see http://lxr.mozilla.org/seamonkey/ident?i=AppendHelloArgument for details).

If you're going through a NATting gateway, the gateway should either do deep-inspection/rewriting (yeah, not bloody likely) or else run an ALG on outgoing connections that rewrites your local private address with its public address.

But you've not yet described exactly *why* you think having your local PC's address in the email is somehow a bug.  Is it *not* the address that the message is indeed originating from?

Comment 4

[Philip Prindeville]

There's nothing really new in this bug report.

All it does is document known behavior.

It doesn't describe:

(1) why this behavior is incorrect or undesirable;
(2) what "better" behavior would be;

If the submitter doesn't do that within 30 days of opening the bug, I suggest that it be closed as INCOMPLETE or INVALID (preferably).

Comment 5

[Biju]

(In reply to comment #3)
> But you've not yet described exactly *why* you think having your local PC's
> address in the email is somehow a bug.  Is it *not* the address that the
> message is indeed originating from?

again, not a big issue to me...
but personally i would like to avoid sending any extra information which not needed.
 
* at least some cooperates i know dont like disclose the internal ip.
* i may be somebody in a country with an oppressive regime who want any extra way to archive privacy.

(2) what "better" behavior would be;
* outlook dont give ip address, also dont give correct hostname
* it looks like outlook coin a hostname.
* we may bluff somewhat random ip address
* or give hostname using formula.
   hostname = from_address.replace(/@/g,'.')

Comment 6

[Biju]

(In reply to comment #3)
> Is it *not* the address that the
> message is indeed originating from?

People commonly expect the one provided by http://whatismyip.com/

(In reply to comment #4)
> If the submitter doesn't do that within 30 days of opening the bug,
> I suggest that it be closed as INCOMPLETE or INVALID (preferably).

may be a WONTFIX

Comment 7

[Philip Prindeville]

(In reply to comment #5)
> again, not a big issue to me...
> but personally i would like to avoid sending any extra information which not
> needed.
> * at least some cooperates i know dont like disclose the internal ip.
> * i may be somebody in a country with an oppressive regime who want any extra
> way to archive privacy.

It's not optional.  Read RFC-2821:  "An Internet mail program MUST NOT
change a Received: line that was previously added to the message header.
SMTP servers MUST prepend Received lines to messages; they MUST NOT change
the order of existing lines or insert Received lines in any other location."

It also specifies what goes into the Received: line.  Read section 4.4 of RFC-2821, and then tell me why we should violate this standard.

BTW:  Fixing oppressive regimes is outside of the scope of Thunderbird bugfixes.

> (2) what "better" behavior would be;
> * outlook dont give ip address, also dont give correct hostname
> * it looks like outlook coin a hostname.
> * we may bluff somewhat random ip address
> * or give hostname using formula.
>    hostname = from_address.replace(/@/g,'.')

The whole point of *precise* information identifying the host is so that the exact path taken by the message, and when, maybe clearly determined for tracing and diagnostic purposes.

I'm sorry if this offends your sense of privacy.  Use another protocol instead.

Thunderbird complies to the standard, and this is correct behavior.  I'm closing this bug.

Comment 9

[OrPLUvXMIe]

> personally i would like to avoid sending any extra information

I think the same. And I like internet standards, too, so I investigated a bit.
Guessing based on received mails is cumbersome, so instead I checked what my Thunderbird sent. I'll try and upload the log as an attachment. You'll see that the only part relevant to LAN IPs is

> » EHLO [192.168.███.███]\r

So let's discuss whether Thunderbird is required by any web standard to expose this, or whether it could offer users an option to omit it or just lie.

> Read RFC-2821:

For issue reader's convenience, here's the link: https://tools.ietf.org/html/rfc2821#section-4.4

> "An Internet mail program MUST NOT change a Received: line
> that was previously added to the message header.

Ok so if Thunderbird adds a Received: header, my mail server should leave it alone. The recorded session did not include any Received: lines, so we can defer any discussion about cases where Thunderbird does send them.

Let's instead have a look at section
> 4.1.1.1  Extended HELLO (EHLO) or HELLO (HELO)
https://tools.ietf.org/html/rfc2821#section-4.1.1.1 :

> The argument field contains the fully-qualified domain name of
> the SMTP client if one is available.

Fortunately we don't yet need to discuss where the RFC authors might have taken that "contains" assumption from. That might become an interesting follow-up bug, since I can't readily find any statement that requires the client program to provide its FQDN. Back to…

> In situations in which the SMTP client system does not have a
> meaningful domain name […], the client SHOULD send an address
> literal (see section 4.1.3), optionally followed by information
> that will help to identify the client system.

The recorded session didn't have the latter, I just quoted that "optionally" part in case Thunderbird ever decides to put more info there.

So in case we decide to do as we SHOULD, what is required?

> (see section 4.1.3)
https://tools.ietf.org/html/rfc2821#section-4.1.3

> Sometimes a host is not known to the domain name system and
> communication […] is blocked.

In my test LAN, both were true, the latter because of NAT.

> To bypass this barrier a special literal form of the address
> is allowed as an alternative to a domain name.

The remainder of section 4.1.3 describes representations of an IP address, without additional instruction on which IP address we should send, so to me it looks like we must base our decision on the declared intent of this feature:

> Sometimes […] communication (and, in particular, communication
> to report and repair the error) is blocked. To bypass this
> barrier[,] a special literal form of the address is allowed

As I understand it, this means that in my LAN case, Thunderbird shall send an address literal that, in case of communication problems, helps the LAN's mail admin (happens to be me) debug the problem that blocked communication. In my case, this is the NAT. The IP address chosen by Thunderbird was not very helpful for me, and I could have provided a much better, more informative address literal. (In addition, that one would even have been on public DNS!)

Therefor,

* I request that Thunderbird, by default, doesn't send internal LAN IPs that it just guesses might sometimes be helpful. As far as I understand 4.1.3, we could send "0.0.0.0" to indicate that Thunderbird didn't know any appropriate address literal. After all, how could Thunderbird know what might help the NAT admin debug the connection? Oh, right!

* I request that Thunderbird offers an easy to configure option for that address literal, and a feild for:

> information that will help to identify the client system.

Thunderbird shall make it very easy for end users to enter both these fields when on the phone with their local NAT admin; this means to offer selection of the supported address scheme (at least IPv4) and syntax verification of the address literal they entered, even with some form of positive feedback like "your input is valid", so the admin can specifically ask whether that message or tick mark or whatever is there, is green etc. For the optional additional information, Thunderbird shall try its best to format it in a way that it most likely survives the intermediate mail transfer servers.

Thanks for reading, and thanks in advance for fixing!

Comment 10

[OrPLUvXMIe]

Attachment: Recorded SMTP session with Thunderbird

Comment 11

[OrPLUvXMIe]

An example for a use case of the custom address literal:

> « 220 █████.█████.███ ESMTP Postfix (Ubuntu)\r
> » EHLO [192.168.███.███]\r
> « 250-█████.█████.███\r
>   250-PIPELINING\r
> *** Connection lost: Unable to send: Socket is closed

Now imagine four similar cases occurring at about the same time. Yeah, I could enable DNS lease assignment logging so I can try and guess stuff about the sending machine based on its IP at some given time, hoping nobody accidentially tunneled his SMTP through another machine on my LAN. Or I could tell them to go to Preferences, Identities, click (whatever) and where it says "diagnostic information address literal", enter the stuff I'll tell them.

Comment 12

[OrPLUvXMIe]

Oh and for those cases where admins do want the LAN IP, please offer something like "guess best LAN IP" as one of the options for the diagnostic information address literal.

Comment 13

[OrPLUvXMIe]

Sorry for yet another notification mail, I can't find any edit button. Just wanted to clarify that it was intentional that I described the config click path as including "Identities": The address should be configured per SMTP identity in order to allow it to very early convey information about what data could be expected later in the connection if it had not been interrupted.

I could imagine a case where a connection vanishes after "AUTH LOGIN" / "334 VXNlcm5hbWU6", where the user accidentially selected a profile not intended for that network, and they have a personal firewall software that claims it "protects your logins against exfiltration". The mail admin could then see that the EHLO address was not affected by the config change, and could ask whether there are more identities configured.

Comment 14

[OrPLUvXMIe]

Also that "DNS lease assignment logging" was meant as DHCP, of course. :-)

Comment 15

[Philip Prindeville]

(In reply to OrPLUvXMIe from comment #9)
> > personally i would like to avoid sending any extra information

There are so many things wrong with this comment, I don't know where to start.

> I think the same. And I like internet standards, too, so I investigated a
> bit.
> Guessing based on received mails is cumbersome, so instead I checked what my
> Thunderbird sent. I'll try and upload the log as an attachment. You'll see
> that the only part relevant to LAN IPs is
> 
> > » EHLO [192.168.███.███]\r
> 
> So let's discuss whether Thunderbird is required by any web standard to
> expose this, or whether it could offer users an option to omit it or just
> lie.

It's there for a reason.  So that logs are meaningful, so that there's a valid security audit in the case of an exploit, and to defeat misconfigured hosts which are looping mail and trying to talk to themselves.

Just out of curiosity, what is so terrible about exposing 192.168.x.x as an address?  It's an RFC-1918 address, which means it's NOT GLOBALLY ROUTABLE and therefore only locally significant.  This address is probably in use on literally MILLIONS of NATted subnets all over the world at any given moment.

> > Read RFC-2821:
> 
> For issue reader's convenience, here's the link:
> https://tools.ietf.org/html/rfc2821#section-4.4
> 
> > "An Internet mail program MUST NOT change a Received: line
> > that was previously added to the message header.
> 
> Ok so if Thunderbird adds a Received: header, my mail server should leave it
> alone. The recorded session did not include any Received: lines, so we can
> defer any discussion about cases where Thunderbird does send them.

Thunderbird is an MUA (mail user agent), and therefore doesn't add Received: lines.  It's the job of MTA's (mail transfer agents) or MS's (message stores, optionally) to add Received: lines.

Cases where TB doesn't send them is "always and everywhere".

> Let's instead have a look at section
> > 4.1.1.1  Extended HELLO (EHLO) or HELLO (HELO)
> https://tools.ietf.org/html/rfc2821#section-4.1.1.1 :
> 
> > The argument field contains the fully-qualified domain name of
> > the SMTP client if one is available.
> 
> Fortunately we don't yet need to discuss where the RFC authors might have
> taken that "contains" assumption from. That might become an interesting
> follow-up bug, since I can't readily find any statement that requires the
> client program to provide its FQDN. Back to…

There's no ambiguity about what "contains" means.

And that IS the statement that requires the client to provide its FQDN.  It's pretty clear.

> > In situations in which the SMTP client system does not have a
> > meaningful domain name […], the client SHOULD send an address
> > literal (see section 4.1.3), optionally followed by information
> > that will help to identify the client system.
> 
> The recorded session didn't have the latter, I just quoted that "optionally"
> part in case Thunderbird ever decides to put more info there.
> 
> So in case we decide to do as we SHOULD, what is required?

It's clear.  Send an FQDN if you have a VALID one, and a bracketed address literally if you don't.

This was all litigated already in bug #279525.

> > (see section 4.1.3)
> https://tools.ietf.org/html/rfc2821#section-4.1.3
> 
> > Sometimes a host is not known to the domain name system and
> > communication […] is blocked.
> 
> In my test LAN, both were true, the latter because of NAT.

Okay, so in that case, send your locally address as a bracketed dotted-quad.

> > To bypass this barrier a special literal form of the address
> > is allowed as an alternative to a domain name.
> 
> The remainder of section 4.1.3 describes representations of an IP address,
> without additional instruction on which IP address we should send, so to me
> it looks like we must base our decision on the declared intent of this
> feature:

You're overthinking this.  Your IP address is your IP address (not the loopback interface, obviously).

If you don't know how to correctly and conclusively figure out what your IP address is, then you need to educate yourself about that before you'll have a convincing argument.

> > Sometimes […] communication (and, in particular, communication
> > to report and repair the error) is blocked. To bypass this
> > barrier[,] a special literal form of the address is allowed
> 
> As I understand it, this means that in my LAN case, Thunderbird shall send
> an address literal that, in case of communication problems, helps the LAN's
> mail admin (happens to be me) debug the problem that blocked communication.
> In my case, this is the NAT. The IP address chosen by Thunderbird was not
> very helpful for me, and I could have provided a much better, more
> informative address literal. (In addition, that one would even have been on
> public DNS!)

This makes no sense, and you're providing no tangible examples.  What was your IP address on the LAN, what other address are you referring to, and what does that have to do with "public DNS"?

> Therefor,
> 
> * I request that Thunderbird, by default, doesn't send internal LAN IPs that
> it just guesses might sometimes be helpful. As far as I understand 4.1.3, we
> could send "0.0.0.0" to indicate that Thunderbird didn't know any
> appropriate address literal. After all, how could Thunderbird know what
> might help the NAT admin debug the connection? Oh, right!

It's not "guessing" what's sometimes helpful.  It's sending the correct information.  How many other hosts on your LAN share your IP address (here's a hint: "none").

Therefore, sending that address uniquely identifies your machine in the context of your LAN, which is exactly what the HELO/EHLO command is supposed to do.

On the other hand, if 50 machines on your LAN (again, all with unique addresses) all use "0.0.0.0" as their alleged address, how is that in any way useful?

> * I request that Thunderbird offers an easy to configure option for that
> address literal, and a feild for:
> 
> > information that will help to identify the client system.

They have that.  It's your IP address.

> Thunderbird shall make it very easy for end users to enter both these fields
> when on the phone with their local NAT admin; this means to offer selection
> of the supported address scheme (at least IPv4) and syntax verification of
> the address literal they entered, even with some form of positive feedback
> like "your input is valid", so the admin can specifically ask whether that
> message or tick mark or whatever is there, is green etc. For the optional
> additional information, Thunderbird shall try its best to format it in a way
> that it most likely survives the intermediate mail transfer servers.

So you want to make something that's already automated and foolproof be done manually by an (apparently) misguided end-user?  And that's somehow an improvement?

> Thanks for reading, and thanks in advance for fixing!

Read it.  Didn't really get any useful information out of it.  Nothing to fix.

Comment 16

[Philip Prindeville]

(In reply to OrPLUvXMIe from comment #11)
> An example for a use case of the custom address literal:
> 
> > « 220 █████.█████.███ ESMTP Postfix (Ubuntu)\r
> > » EHLO [192.168.███.███]\r
> > « 250-█████.█████.███\r
> >   250-PIPELINING\r
> > *** Connection lost: Unable to send: Socket is closed
> 
> Now imagine four similar cases occurring at about the same time. Yeah, I
> could enable DNS lease assignment logging so I can try and guess stuff about
> the sending machine based on its IP at some given time, hoping nobody
> accidentially tunneled his SMTP through another machine on my LAN. Or I
> could tell them to go to Preferences, Identities, click (whatever) and where
> it says "diagnostic information address literal", enter the stuff I'll tell
> them.

If you're NOT logging DHCP addresses, then you're leaving yourself without the means to conduct a correct and meaningful audit in the case of an intrusion.

Comment 17

[OrPLUvXMIe]

Thanks for your feedback! I'll answer the easy parts first:

> so that there's a valid security audit in the case of an exploit,

Yes, there are different kinds of paranoia when it comes to surveillance, security, and privacy. They all have their place. I'm trying to make TB conform to the local policy priorities for the computers and network it shall run in, while preserving as much RFC compliance as I can.

> Just out of curiosity, what is so terrible about exposing
> 192.168.x.x as an address?

It might be just a policy default to expose the least amount of technical details of our internal network infrastructure. It's in a country that most citizens wouldn't deem an oppressive regime, still there's a tendency towards secrecy since news tell us that other countries have agencies, some with impressive funds, whose job it is to collect every last bit of tech details they can find about us. Judging by the measures used for espionage, there has to be some value in our data, even if we can't discover that value ourselves yet.

> and what does that have to do with "public DNS"?

That was about a VPN interface that makes TB's LAN IP socket end up on another machine, and at the time I posted I didn't yet know it was the tunnel's job to translate the EHLO IP.

> It's clear. Send […] a bracketed address literally if you don't.

Yup, in those cases where we agree it SHOULD send one, it's all about the choice of *which* bracketed address literal to send.

> It's sending the correct information.

Seems to me we just have different beliefs about what the RFC intends; and even different beliefs about how extraordinary your circumstances must be to warrant to omit something that the RFC says "SHOULD" be done.

> How many other hosts on your LAN share your IP address (here's a
> hint: "none").

Same problem: I could make up some scenarios involving multi-user systems, VMs, or whatever; they wouldn't be any more convincing than the personal firewall debugging scenario from comment 13, as long as we can't agree on whom the RFC intends the info to be useful for.
At least I now see that you agreed (in comment #7) that it's about diagnostics of the message path, but we'll probably still disagree on who should determine which parts of that message path are most interesting in which scenario.

> Thunderbird is an MUA (mail user agent), and therefore doesn't add
> Received: lines.

Yes, I'd be very surprised if Thunderbird included any. Seeing comment #7 with your invocation of the RFC part about Received: lines, I felt a need to clarify that the cited part cannot apply to the bug reporter's problem.
Maybe at that time it wasn't yet clear that the IP problem in the headers traces back to the EHLO. Since that happens before we even know whether any mail body will be transfered, rules for Received: lines can neither explain nor mandate the EHLO behavior. Without a clarification, this might have been less obvious to other (even future) readers, it might therefore have come accross more authoritative or even intimidating than it should.

> This was all litigated already in bug #279525.

Thanks for pointing me there! I had missed it in comment #1 because I assumed (falsely) the number there was the bug listed as duplicate. We should get a "Related bugs:" field below the "Duplicates:" field so we can put 279525 there, to give it the visibility (and thus hopefully, attention) that it deserves.

From that thread I learned the solution to my immediate problem (which I'll post extra so it gets a comment number to link to), as well as other interesting things about mail and the Thunderbird project.

The historical information in comment 279525#c109 is another interesting consideration about what kinds of "truth" to expect from the EHLO argument. The split VPN case there has some similarity with my case of multiple IP addresses on the same host or even interface, even without FQDNs.

At that stage of reading I was still clinging to my own ideals about software, which center around utility for the user. I was so biased by my UX training that I took for granted a software should try its best to do what users wish it did, that I got puzzled how people in that thread could argue at length about special cases of "admin knows better than TB" without requesting a config option to just tell TB which FQDN to use.
It appeared to be such a simple fix for those edge cases, very similar to how I requested to make the IP configurable. We could even solve both problems by allowing to configure the arguments for HELO and EHLO, the latter with a checkbox "same as HELO" to make it even easier. Then everybody who cares deeply enough can optimize their spam score for the actual SpamAssassin they individually are dealing with! They could even change it to fit hotmail before they write an email to a hotmail user, then revert the config change. (A rules engine for selecting the best EHLO based on recipient list, can be left to addon authors.) At last in 279525#c143, after 142 comments, someone invents the idea of a switch, although less mighty than custom HELO / EHLO.

Later I realized that there are other stakeholders than users, and they might have other priorities. Could it be a pride belief of a perfect software? I can't really judge the next quote because of limited mastery of English, so I'll assume that "you" is the general one meaning anyone, or some theoretical admin, not me.

> If you don't know how to correctly and conclusively figure out what
> your IP address is, then you need to educate yourself about that

If that theoretical admin arrived at a situation like the one I described, with multiple IP addresses, they'd have to select a single "my" IP address. I'd pick the static one because it's easier to identify. People from the other thread used as criteria whether the behavior to be decided would lead to hotmail accepting or destroying their email.
Software like TB (without AI) always has very limited scope of knowledge, following narrow paths that take very little context into account. Usually that alone should indicate that there may be cases where the admin knows better and it might be Thunderbird who needs to be educated about how to pick that IP.

> need to educate yourself about [how to determine "my" IP]
> before you'll have a convincing argument

Of that last part I can't make sense except for one that would be ad hominem, as we'd be the only ones who might need an argument in this context, and you obviously know how. If that was indeed the intended meaning, you'll need to write it in easier words for me, as it would imply the selection problem with multiple IPs were just an illusion on my side and I were too dumb to grasp it.

At least for the case of where some port on 127.0.0.1 is just a gateway to some other network that might not even be IP based, the other thread clarified: It's the gateway's job to translate the EHLO line. If your gateway is too plain (like a standard SSH tunnel), don't expect any assistance from TB, instead get a better gateway.

As mentioned I'm still under the same impression as some people in the other thread, that there might be situations where automatic selection could fail. Thus I wonder, how can it be more important to defend that a software with the limits described above is still infallible and thus doesn't need any channel to accept advice from a human? There's a ton of sci-fi demonstrating why important mechanisms usually hopefully do have that "manual override" for extraordinary situations.
Imagine a thriller movie where the secret agent tells the tech guy to not worry about the scheduled military strike, he has mailed his agency that the mission was solved under cover, everything is fine now. Tech guy: "Does your agency still use spicymail for their covert channels?" Agent: "Yeah, and Thunderbird on our anonymous off-the-shelf notebooks." Tech guy, starting to run: "##%*&# For the holiness of RFCs, everyone GTFO while you can!"

So is it holy war about exegesis of the book? I'd say no, apostles wouldn't accidentually cite an unrelated verse from the RFC.

There could be a more convincing argument: Support cost. Any additional config option adds to maintainance cost. Even more severe, if the mechanism to change EHLO was inside of Thunderbird, people that misconfigured it might have a higher tendency to try and discuss their resulting tech problems at some Mozilla forum or even this bugtracker, which could degrade those communication channels for the developers.

Is that a major reason to prefer fully automatic choice of the EHLO argument? I could totally accept that explanation and there would be no need I repeat my experiment to document more precisely yet another example of user disagreeing with TB's choice, as there are already quite many in the other thread.

Comment 18

[OrPLUvXMIe]

Now for addressing Biju's originally reported problem from a user's utility perspective.


===============================================
How to make your mail's EHLO IP more anonymous.
===============================================

Basically, make Thunderbird connect to your mail server through the single most commonly present network interface: 127.0.0.1, and it will use that as the EHLO IP. (Using "localhost" could leak information about whether your host supports IPv6.) (On WinXP and maybe other systems, you might need to install a special driver software to ensure you really have a 127.0.0.1 interface. If in doubt, ask an network support forum for your OS.)

Since I can't find easy options for the "through", replace it with "to" - make a new Thunderbird user profile in which you tell TB that your SMTP server's address actually *is* 127.0.0.1, and use another program to tunnel some port on that IP to your actual mail server. The tunnel program must be running at the time you want to send mail, e.g. via an autostart command. (If this part is uncommon to you, ask a search engine how it works for your operating system.)

A lightweight tool for easy TCP tunneling might be "socat". Suppose your real SMTP server is "Host: smtp.example.net, Port: 465" and you'd want to open a tunnel that makes "Host: 127.0.0.1, Port: 1465" work as the SMTP server you set in Thunderbird. (The last port number I just made up placing any digit in front. If you want another number and are unsure, or that one didn't work, ask in a network forum for your OS.)
The socat command would be:

socat TCP-LISTEN:1465,bind=127.0.0.1,reuseaddr,fork TCP:smtp.example.net:465

However, the remote mail server will still see your internet connection's public IP address and might put that one in the mail. To disguise that, you'll need to connect to your SMTP server through another machine.

----------
SSH tunnel
----------

If you know about SSH tunnels, that's an option, and will most likely result in the Received: header having your SSH target machine's IP or hostname in it. If you have SSH access to your mail server as a regular user (so you cannot simply re-configure that SMTP to anonymize you), you can set 127.0.0.1 as the remote tunnel side's target and headers may be totally different since the server might think the mail originates from an automated program hosted there.

----------
TOR tunnel
----------

The "another machine" to disguise your public IP address can also be the TOR network. If you don't have access to it yet, see https://www.torproject.org/ for how to install the software. Anonymity is just one feature of TOR, but one that earned it a bad reputation with some admins, so if your other tunnel worked and TOR doesn't, chances are your SMTP server's company is so scared of TOR that they just don't allow it. In that case, you can try and make another email account at another email company.

Be aware that the recipients' server may fear TOR as well, in which case it might discard your mail or mark it as spam.

Be aware that when using TOR, some random computer anywhere on the internet will see what your Thunderbird talks with your mail server. That computer might be run by anyone, including evil people or people whose computer is infected with evil software. Never send any secrets (e.g. mail message or mail account login) over the TOR network in plain. If you talk to your SMTP via TOR, use the strongest encryption that Thunderbird can provide, and educate yourself whether that is secure enough for your project.

To make a socat tunnel like the example above, just with TOR, change your autostart command to:

socat TCP-LISTEN:1465,bind=127.0.0.1,reuseaddr,fork SOCKS4A:localhost:smtp.example.net:465,socksport=9050,socksuser=nobody

… and make sure there are no old tunnels left over that might interfere. (If you have ANY doubts about how to manage your autostart processes, please refrain from using TOR for SMTP. You'll do yourself more harm than good at that level of expertise.)

----------
Conclusion
----------

In any case, check and verify the resulting mails before you use those tunnels for sending mails that really need to be anonymous for whatever reason. Usually there are way more identity clues lurking in headers, than just the IP address.

Comment 19

[Philip Prindeville]

(In reply to OrPLUvXMIe from comment #17)
> Thanks for your feedback! I'll answer the easy parts first:
> 
> > so that there's a valid security audit in the case of an exploit,
> 
> Yes, there are different kinds of paranoia when it comes to surveillance,
> security, and privacy. They all have their place. I'm trying to make TB
> conform to the local policy priorities for the computers and network it
> shall run in, while preserving as much RFC compliance as I can.

You're trying to make it conform to what you've cherry-picked as being important while dismissing generally accepted Best Practices for security as well as requirements for RFC-compliance.

You want to do that?  Run your own locally-hacked version of TB.  There's nothing stopping you.

> > Just out of curiosity, what is so terrible about exposing
> > 192.168.x.x as an address?
> 
> It might be just a policy default to expose the least amount of technical
> details of our internal network infrastructure. It's in a country that most
> citizens wouldn't deem an oppressive regime, still there's a tendency
> towards secrecy since news tell us that other countries have agencies, some
> with impressive funds, whose job it is to collect every last bit of tech
> details they can find about us. Judging by the measures used for espionage,
> there has to be some value in our data, even if we can't discover that value
> ourselves yet.

You're missing the point.  Please read RFC-1918 to understand why this is a non-issue.

I'm talking specifically about non-routable, locally significant ONLY addresses and you're waving your arms about "oppressive regimes" and "measures used for espionage".

Worse, you're talking about "some value in our data" that you can't prove, but you're adamantly convinced still exists.

> > and what does that have to do with "public DNS"?
> 
> That was about a VPN interface that makes TB's LAN IP socket end up on
> another machine, and at the time I posted I didn't yet know it was the
> tunnel's job to translate the EHLO IP.

IT'S NOT.  When you connect outbound via that interface, the socket will be bound locally to the interface that's used to reach the first hop to that destination (in this case, the remote endpoint of your tunnel).  It will CORRECTLY say "ehlo [n.n.n.n]" where n.n.n.n is the local address of your VPN tunnel.  No rewriting required.

Before you speculate, walk through the kernel of your OS to understand what connect() ACTUALLY does.

> > It's clear. Send […] a bracketed address literally if you don't.
> 
> Yup, in those cases where we agree it SHOULD send one, it's all about the
> choice of *which* bracketed address literal to send.
> 
> > It's sending the correct information.
> 
> Seems to me we just have different beliefs about what the RFC intends; and
> even different beliefs about how extraordinary your circumstances must be to
> warrant to omit something that the RFC says "SHOULD" be done.

You're choosing to ignore what the RFC clearly states.

If you READ the discussion about bug #279525, you'll see that I considered a LOT of scenarios including ones even more improbable than yours.  Don't impugn my competence or judgment without educating yourself first.  And certainly not if you want to win me over to your way of thinking.

> > How many other hosts on your LAN share your IP address (here's a
> > hint: "none").
> 
> Same problem: I could make up some scenarios involving multi-user systems,
> VMs, or whatever; they wouldn't be any more convincing than the personal
> firewall debugging scenario from comment 13, as long as we can't agree on
> whom the RFC intends the info to be useful for.
> At least I now see that you agreed (in comment #7) that it's about
> diagnostics of the message path, but we'll probably still disagree on who
> should determine which parts of that message path are most interesting in
> which scenario.

How is having ALL clients generate the SAME string going to be MORE helpful in tracing a problem?

You can't have it both ways:  you're either trying to INCREASE the utility of diagnostic information, or you're trying to LIMIT information exposed to an adversary which can be exploited.  They are completely opposed objectives.

> > Thunderbird is an MUA (mail user agent), and therefore doesn't add
> > Received: lines.
> 
> Yes, I'd be very surprised if Thunderbird included any. Seeing comment #7
> with your invocation of the RFC part about Received: lines, I felt a need to
> clarify that the cited part cannot apply to the bug reporter's problem.
> Maybe at that time it wasn't yet clear that the IP problem in the headers
> traces back to the EHLO. Since that happens before we even know whether any
> mail body will be transfered, rules for Received: lines can neither explain
> nor mandate the EHLO behavior. Without a clarification, this might have been
> less obvious to other (even future) readers, it might therefore have come
> accross more authoritative or even intimidating than it should.

"Since that happens before we even know whether any mail body will be transfered [sic], rules for Received: lines can neither explain nor mandate the EHLO behavior."

What?  That doesn't even make sense.

RFC-2821, Section 4.4:

4.4 Trace Information

   When an SMTP server receives a message for delivery or further
   processing, it MUST insert trace ("time stamp" or "Received")
   information at the beginning of the message content, as discussed in
   section 4.1.1.4.

...


   This line MUST be structured as follows:

   -  The FROM field, which MUST be supplied in an SMTP environment,
      SHOULD contain both (1) the name of the source host as presented
      in the EHLO command and (2) an address literal containing the IP
      address of the source, determined from the TCP connection.

There's NO AMBIGUITY AT ALL about what is being required here, or why.  There's no room for second guessing "intention".  There's just you choosing to ignore what's before you.

> > This was all litigated already in bug #279525.
> 
> Thanks for pointing me there! I had missed it in comment #1 because I
> assumed (falsely) the number there was the bug listed as duplicate. We
> should get a "Related bugs:" field below the "Duplicates:" field so we can
> put 279525 there, to give it the visibility (and thus hopefully, attention)
> that it deserves.

Again, proof of your not making good use of the information in front of you.

Stop assuming, and start absorbing.

> From that thread I learned the solution to my immediate problem (which I'll
> post extra so it gets a comment number to link to), as well as other
> interesting things about mail and the Thunderbird project.
> 
> The historical information in comment 279525#c109 is another interesting
> consideration about what kinds of "truth" to expect from the EHLO argument.
> The split VPN case there has some similarity with my case of multiple IP
> addresses on the same host or even interface, even without FQDNs.

Again, this solution was considered in depth.  There was AMPLE discussion about it.  If you have anything new to add to this discussion, you have yet to do so.

Consider the case where you're talking to a public SMTP server (like gmail) but tunneling out across a VPN tunnel that's terminated not on your desktop but on your local router.  In this case, it's completely transparent to you and neither you nor your SMTP server have any way to know it's happening.

In this case, TB CAN'T NOR SHOULD IT be aware of this situation nor change its behavior in any way.

Your connection will APPEAR to the SMTP server to be coming from the PUBLIC address of your remote tunnel's endpoint (i.e. the security gateway or remote SGW), but your EHLO will have your local, non-routable RFC-1918 address.  This is normal, and acceptable behavior.

There is NO REASON TO CHANGE THIS.

> At that stage of reading I was still clinging to my own ideals about
> software, which center around utility for the user.

The user doesn't troubleshoot connectivity or mail issues: the site-administrator does.  Your user doesn't know what an EHLO is nor does he care.  Nor should he care.

> I was so biased by my UX
> training that I took for granted a software should try its best to do what
> users wish it did, that I got puzzled how people in that thread could argue
> at length about special cases of "admin knows better than TB" without
> requesting a config option to just tell TB which FQDN to use.

Okay, go back and reread this conversation.  You have little understanding of what address gets locally bound to a connected socket in the case of split-tunneling, for instance.  This is why you're not well situated to decide what is or what isn't useful tracing information.

> It appeared to be such a simple fix for those edge cases, very similar to
> how I requested to make the IP configurable. We could even solve both
> problems by allowing to configure the arguments for HELO and EHLO, the
> latter with a checkbox "same as HELO" to make it even easier.

Okay, I'll bite (mostly because I'm apparently a self-loathing masochist).

What happens if my address is "192.168.1.50", and your address is "192.168.1.51", and I decide to configure the tracing information as "192.168.1.51" and start sending millions of Spam?  Who do you think people are going to come looking for?

> Then everybody
> who cares deeply enough can optimize their spam score for the actual
> SpamAssassin they individually are dealing with!

Again, you're trying to fix the problem, not the cause.  Your SpamAssassin (and I contribute to that as well as TB) is scoring things incorrectly.  So you're trying to lie to it so that it won't ding you.

Why not fix SpamAssassin so it's not penalizing correct behavior?

My SA, for example, doesn't even score email received over an SMTP+AUTH connection.

Ding ding ding!!!! Problem solved without changing TB.

> They could even change it
> to fit hotmail before they write an email to a hotmail user, then revert the
> config change. (A rules engine for selecting the best EHLO based on
> recipient list, can be left to addon authors.)

You have no idea how ridiculous this sounds, do you?  "Which lie will be the most convincing to SpamAssassin, let's go with that."

FIX SPAMASSASSIN SO YOU DON'T NEED TO LIE TO IT.

TB isn't the culprit here.

Listen, I'm a big fan of both pieces of software.  Been using both for more than a decade.  But don't be wanting to change TB because you've given up on getting SA to be well-behaved.

That's your failure: not TB's.

> At last in 279525#c143, after
> 142 comments, someone invents the idea of a switch, although less mighty
> than custom HELO / EHLO.

No, that person invented a misconception which he refused to abandon:

"Thunderbird message headers are causing problems when sending to hotmail users because of how the headers are formulated."

3 words into the sentence and he's already showing that he has no understanding of the problem space.  The headers AREN'T BEING GENERATED BY TB BUT BY THE RECEIVING MTA.

Just as you're choosing to not abandon your glaring misconceptions.  

"It turns out that when I send email from Mozilla Thunderbird to a Hotmail address in which my address is not already in their contact list or in the "Safe List", it simply gets dropped by Hotmail."

The user here has UNKNOWINGLY acknowledged that TB isn't the culprit: had he/she sent the email from ANY OTHER MUA, HE/SHE STILL WOULD NOT BE IN THE RECEIVER'S CONTACT LIST.  PERIOD.

More misconceptions:

"Well it turns out that both Thunderbird and Hotmail are at fault (in my opinion) on this one. Thunderbird sends an IP address in the SMTP HELO command. This is basically asking to get flagged as spam by any decent spam catcher, like SpamAssassin."

This person isn't sending an email from TB to Hotmail.  He's sending an email to HIS MTA WHICH THEN FORWARDS IT TO HOTMAIL.  Hotmail should be paying attention to the topmost Received: line and no other, because none of the others are verifiable.

If you want to cling to the explanations of poorly informed commentators, then there's nothing I can do for you.

> Later I realized that there are other stakeholders than users, and they
> might have other priorities. Could it be a pride belief of a perfect
> software?

Could it ALSO be that some of us understand what we're talking about better than others?

> I can't really judge the next quote because of limited mastery of
> English, so I'll assume that "you" is the general one meaning anyone, or
> some theoretical admin, not me.

> > If you don't know how to correctly and conclusively figure out what
> > your IP address is, then you need to educate yourself about that

No, I meant YOU individually.  The one and only.

> If that theoretical admin arrived at a situation like the one I described,
> with multiple IP addresses, they'd have to select a single "my" IP address.

Right.  Which they'd know how to do because they understand how to read the routing table of a machine with a VPN split-tunnel configured, or how to interpret the XFRM of an IPsec configuration.

It's not magic: it just requires some self-education.

> I'd pick the static one because it's easier to identify.

And that's why you'd be wrong.

Why YOU would pick it has nothing to do with why your KERNEL would bind the socket to any one of the other addresses available.

> People from the
> other thread used as criteria whether the behavior to be decided would lead
> to hotmail accepting or destroying their email.

"People"?  People who possibly don't know what they're talking about?  Those people?

> Software like TB (without AI) always has very limited scope of knowledge,
> following narrow paths that take very little context into account. Usually
> that alone should indicate that there may be cases where the admin knows
> better and it might be Thunderbird who needs to be educated about how to
> pick that IP.

The context, ALL the context, that is relevant in this case is what the RFC's require.  You're starting off by ignoring those or claiming that they say something that they don't, or claiming that they don't say things which they very clearly do.

THE MACHINE PICKS THE IP.  THE MACHINE ALWAYS PICKS THE IP.

You want a situation where the end-user, the site-administration, and OS, and the AUTHOR OF EACH AND EVERY NETWORK APP get a vote in what IP gets used?  That's insanity.

Determinism is a good thing.

> > need to educate yourself about [how to determine "my" IP]
> > before you'll have a convincing argument
> 
> Of that last part I can't make sense except for one that would be ad
> hominem,

Not ad hominem at all.  I'm saying an argument grounded in concrete, demonstrable observations (such as what address an OS chooses to bind a connected socket to) are evidence-based and convincing.

Arguments based on "well there may or may not be people out to get me which I couldn't prove even if there were" is prima face ridiculous.

And wholly beyond the scope of TB bugs.

That's an attack on your argument.  Not you.

If you're more knowledgeable than you appear but are choosing to hide it in your argumentation... well, then you're picking a bad strategy.

> as we'd be the only ones who might need an argument in this
> context, and you obviously know how. If that was indeed the intended
> meaning, you'll need to write it in easier words for me, as it would imply
> the selection problem with multiple IPs were just an illusion on my side and
> I were too dumb to grasp it.

THERE IS NO SELECTION PROBLEM.  YOUR KERNEL KNOWS HOW TO ROUTE, IT DOES SO RELIABLY AND EFFECTIVELY FOR TB, AND THROUGH MODULARITY AND GOOD SOFTWARE DESIGN, THIS PROCESS IS INSULATED FROM TB WHICH HAS NO NEED TO UNDERSTAND ROUTING.

I can't speak to whether you're dumb or not, but you're choosing to be distracted by irrelevant information while ignoring that which is most topical (imaginary villains versus competent site administrators, and who is best served and how).

> At least for the case of where some port on 127.0.0.1 is just a gateway to
> some other network that might not even be IP based, the other thread
> clarified: It's the gateway's job to translate the EHLO line. If your
> gateway is too plain (like a standard SSH tunnel), don't expect any
> assistance from TB, instead get a better gateway.

Again, you're showing how much you clearly don't understand.  127.0.0.1 will never take you off of your machine, much less off your local network.  It's an address which is SIGNIFICANT ONLY ON THE SAME MACHINE.

Read RFC-1918, and while you're at it, RFC-5735.  You might find that much of what you're worrying about is inconsequential.

> As mentioned I'm still under the same impression as some people in the other
> thread, that there might be situations where automatic selection could fail.

If it FAILS, you wouldn't have any connection AT ALL, no message would be delivered, and there'd be nothing to mark as SPAM.

If a message is getting through, then by definition that right address has been selected.  This is how routing works.

> Thus I wonder, how can it be more important to defend that a software with
> the limits described above is still infallible and thus doesn't need any
> channel to accept advice from a human? There's a ton of sci-fi demonstrating
> why important mechanisms usually hopefully do have that "manual override"
> for extraordinary situations.

You're going to quote Sci-Fi to bolster your argument?

Really?  Does that usually work out well for you?

And what in this constitutes "extraordinary situations"?  That someone with a poorly configured SpamAssassin dropped your message?

> Imagine a thriller movie where the secret agent tells the tech guy to not
> worry about the scheduled military strike, he has mailed his agency that the
> mission was solved under cover, everything is fine now. Tech guy: "Does your
> agency still use spicymail for their covert channels?" Agent: "Yeah, and
> Thunderbird on our anonymous off-the-shelf notebooks." Tech guy, starting to
> run: "##%*&# For the holiness of RFCs, everyone GTFO while you can!"

Okay, and this is where I realize just how much of a masochist I am.

> So is it holy war about exegesis of the book? I'd say no, apostles wouldn't
> accidentually cite an unrelated verse from the RFC.
> 
> There could be a more convincing argument: Support cost. Any additional
> config option adds to maintainance cost. Even more severe, if the mechanism
> to change EHLO was inside of Thunderbird, people that misconfigured it might
> have a higher tendency to try and discuss their resulting tech problems at
> some Mozilla forum or even this bugtracker, which could degrade those
> communication channels for the developers.

"Any additional config option adds to maintainance [sic] cost."  Except that no one paid me to fix bug #279525.  And the only cost is to me to have to waste my Sunday afternoon explaining to people with concrete facts which they're going to ignore in favor of fanciful doomsday scenarios.

Can you even cite which SpamAssassin rule was being triggered or why?

What was the rule name and how was it scored?

What text in your message was it matching?

> Is that a major reason to prefer fully automatic choice of the EHLO
> argument? I could totally accept that explanation and there would be no need
> I repeat my experiment to document more precisely yet another example of
> user disagreeing with TB's choice, as there are already quite many in the
> other thread.

Lots of people with not much to say, but saying it loudly and enthusiastically all the same...

Comment 20

[OrPLUvXMIe]

Thank you for bearing with my ignorance about actual routing at the kernel level. I think I see how for that kind of routing problems, the IP of the socket is the single most important information.

All debug scenarios I had in mind are for higher layers, and you're right in that just using the same IP won't do them any good. Using the EHLO for early hints about what could have been planned to transfer over a connection that breaks too early, would require that it is customized rather than just anonymized.

I'm probably too cynical about the likeliness of fixing the recipient's SpamAssassin, compared to the sender's mailer. You're totally right that my approach tends to

> you're trying to fix the problem, not the cause.

nowadays, as it just too often turned out that the proper fix at the cause was out of reach for me. In cases of a big freemail provider misbehaving, it seems just impossible if you don't know someone there.

> Lots of people with not much to say, but saying it loudly and
> enthusiastically all the same...

I know that situation from other areas where I have more expertise, and it's really an uphill battle as new people join the internet every day, most of them even less educated about network stuff and RFCs than I seem to be.
Now that this thread has a hint on how to change the easily visible behavior, there's a chance the less educated may at least complain less often about that EHLO IP. For most of them it's probably unimportant whether what TB does is a bug, they just fear some kind of abuse that could happen from disclosing a piece of information of which they don't understand why they should disclose it. The fact that other popular mail software does it other ways, sure won't make it easier for them to see that Thunderbird is right. Now they can make it look (to them) as if it "worked", and if someone manages to educate them about why trying to hide their IP is wrong, maybe they'll even start to understand why not to do it.

I'll try to learn more about the routing aspects you mentioned, and try to understand the other RFCs you told me. I can't know yet whether I'll ever grasp entirely what kinds of suffering you had to go through, but I hope it helps you to know that at least your brave ("masochist" as you call it) attemt at explaining shall not be wasted.

Comment 21

[Philip Prindeville]

(In reply to OrPLUvXMIe from comment #17)

> Again, you're trying to fix the problem, not the cause. 

Or rather, the symptom and not the cause.

Comment 22

[Philip Prindeville]

(In reply to OrPLUvXMIe from comment #20)

> nowadays, as it just too often turned out that the proper fix at the cause
> was out of reach for me.

No, vote with your feet.  Yahoo's abuse department has been ignoring complaints for years.  Now if someone tries to email us from Yahoo, they get a bounce saying "send us an email from a responsible ISP".

> In cases of a big freemail provider misbehaving, it
> seems just impossible if you don't know someone there.

No one says you HAVE to deal with them.  Don't.

> > Lots of people with not much to say, but saying it loudly and
> > enthusiastically all the same...
> 
> I know that situation from other areas where I have more expertise, and it's
> really an uphill battle as new people join the internet every day, most of
> them even less educated about network stuff and RFCs than I seem to be.
> Now that this thread has a hint on how to change the easily visible
> behavior, there's a chance the less educated may at least complain less
> often about that EHLO IP. For most of them it's probably unimportant whether
> what TB does is a bug,

And it's not a bug.  It's close compliance with the RFC's.

If the RFC's are wrong, try to update them.  There are lots of RFC's out there which
correct errata in earlier RFC's or update and supersede them altogether.

> they just fear some kind of abuse that could happen
> from disclosing a piece of information of which they don't understand why
> they should disclose it. The fact that other popular mail software does it
> other ways, sure won't make it easier for them to see that Thunderbird is
> right.

Claiming that is irresponsible.  I just sent an email to a mailing list
(users-spamassassin) as it turns out, and Mail.app did exactly the same thing.

You can't go around making blatantly false assertions if you want to be taken
seriously.

> Now they can make it look (to them) as if it "worked", and if someone
> manages to educate them about why trying to hide their IP is wrong, maybe
> they'll even start to understand why not to do it.

If they don't want their desktop IP to be exposed, then don't use SMTP.  You can
get Outlook and talk to an exchange server over MAPI.

Problem solved.

> I'll try to learn more about the routing aspects you mentioned, and try to
> understand the other RFCs you told me. I can't know yet whether I'll ever
> grasp entirely what kinds of suffering you had to go through, but I hope it
> helps you to know that at least your brave ("masochist" as you call it)
> attemt at explaining shall not be wasted.

I hope that's true and even if it's not I appreciate your saying it.