Closed Bug 1710224 Opened 4 years ago Closed 3 years ago

Cannot send email via easyname smtp account

Categories

(MailNews Core :: Networking: SMTP, defect)

Product:
MailNews Core
Component:
Networking: SMTP
Version:
Thunderbird 89
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 1726106

People

(Reporter: hlein, Unassigned)

References

Details

(Keywords: regression)

Attachments

(19 files, 3 obsolete files)

failed.png
9.14 KB, image/png
Details
settings.png
15.80 KB, image/png
Details
tbird_log.txt - log when starting TB, mail to be sent in outgoing
709.07 KB, text/plain
Details
smtp.log of failing send
4.66 KB, application/octet-stream
Details
2021-05-10 10_18_10-Window.png after mailnews.smtp.jsmodule set to false
6.76 KB, image/png
Details
new Send_Easyname.log
6.24 KB, application/octet-stream
Details
Test send smtp.eml as requested in comment #16
2.46 KB, message/rfc822
Details
Test no pgp.eml PGP disabled - same problem
831 bytes, text/plain
Details
Test without PGP # 2 from outbox.eml
853 bytes, text/plain
Details
Output_81_82.zip containing files for Comment # 26
1.14 KB, application/x-zip-compressed
Details
error-message.png
3.02 KB, image/png
Details
tbird_log.txt.moz_log
125.31 KB, text/plain
Details
15-09-2020.txt.moz_log
7.92 KB, text/plain
Details
16-09-2020.txt.moz_log
7.92 KB, text/plain
Details
TB89-beta4.txt
3.46 KB, text/plain
Details
workaround-fix-for-easyname-smtputf8-issue.diff
3.44 KB, patch
Details | Diff | Splinter Review
why-is-it-null.png
2.67 KB, image/png
Details
Easyname's up to now last response.
3.20 KB, text/plain
Details
Attachement for comment #102
11.11 KB, text/plain
Details
Attached image failed.png —

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:88.0) Gecko/20100101 Firefox/88.0

Steps to reproduce:

Try to send email via specific account

No settings have been changed and it works with TB 88.x
After update to TB 89 it fails

Actual results:

Error message sending failed for unknown reason

Expected results:

mail should be sent.

Attached image settings.png —
Component: Untriaged → Networking: SMTP
Product: Thunderbird → MailNews Core

Are you using TB 89 beta 3? A few "cannot send" bugs have been fixed lately. Does your password have a non-ASCII character? Bug 1707062 fixes that, but it will only be included in TB 89 beta 4 or TB 90 beta 1.

If you can't get it to work, please provide a log from the Error Console (Tools > Developer Tools > Error Console) in a file after setting the pref mailnews.smtp.loglevel to All.

Yes, I am using TB 89 beta 3. I am using the same password for another smtp server without problems (it contains letters, digits and #)

In the attached smtp.log line 60 I see "mailnews.smtp: S: 550 Sorry, this message is spam!" (account at easyname)

Note I can send exactly the same mail from another account (gmx) and from an Android mobile phone, also easyname account.

Can you send the same mail with mailnews.smtp.jsmodule set to false? If not, set mailnews.send.jsmodule to false as well

(In reply to Ping Chen (:rnons) from comment #6)

Can you send the same mail with mailnews.smtp.jsmodule set to false? If not, set mailnews.send.jsmodule to false as well

Needs to restart TB after the change. If it still doesn't work, then it's a server problem.

Keywords: regression

mailnews.smtp.jsmodule set to false (and also mailnews.send.jsmodule to false) leads to a message "Sorry, this message is spam"

I'm afraid you need to contact your mail service provider, they may have added you to a block list by accident. When the two .jsmodule prefs are set to false, the old C++ sending backend is used, I don't think there is any code change in that part.

Further to Ping's reply: Some background. Starting at TB 88 beta and more in TB 89 beta, the "sending system" was replaced leading to a few problems which all have been fixed now. You cannot send neither with the old system (both prefs set to false) nor with the new system (both prefs set to true). Clearly the server considers the mail you're about to send to be SPAM. There's nothing we can do about that. Talk to that provider. Or try sending a simple message to see whether that's rejected as well. Perhaps the provider rejects you based on your IP address. In that case you could use a VPN/tunneling service, or connect your laptop(?) via tethering via a mobile service to get a different IP (or search Google for a free open proxy).

Bug 1710066 is probably relevant: the server message doesn't propagate back to the UI properly.

See Also: → 1710066

Additional info:
The same mail(s) can be sent by the same account and settings from a mobile phone (K9-Mail) and also on the same PC by Windows Mail and eM client. It seems that it is only Thunderbird causing the problem (maybe due to problematic building of the mail body before sending?).

Attached file new Send_Easyname.log —

Can you try saving the mail as a draft, then save the draft as an eml file, then upload the eml file here?

You can also try importing the eml file into other mail clients you mentioned.

Do you remember the last TB version that worked for you? Old versions can be downloaded from https://ftp.mozilla.org/pub/thunderbird/releases/, do you mind downloading a few older versions to find out? Thanks

Attachment #9223529 - Attachment is patch: true
Attachment #9223529 - Attachment mime type: message/rfc822 → text/plain
Attachment #9223529 - Attachment is patch: false
Attachment #9223529 - Attachment mime type: text/plain → message/rfc822

So this is a OpenPGP encrypted mail, when you tested with other mail clients, do they support OpenPGP encryption as well?

Looks like you're using OpenPGP and the draft was encrypted, so we can't see anything. Instead of saving the draft, you should sent the message "later": "File > Send Later" (Ctrl+Shift+Enter). Then you can retrieve it from the Outbox and attach it here. Please attach it as text/plain instead of message/rfc822.

Instead of installing old versions, please check whether sending still works with the old C++ modules. To do so, switch the preferences mailnews.smtp.jsmodule and mailnews.send.jsmodule to false (see comment #6).

BTW, the server answer was "550 Sorry, this message is spam!"

K9-Mail from mobile phone supports and is configured to use PGP, I did not configure eM client and Windows Mail to use PGP.
However, the problem also occurs in TB if I deconfigure PGP for this acccount

saved from outbox as requested in comment #19

@ comment #19
I did also try with the the C++ modules - it also fails (see comment #8)

I will try to find an older version that works.

Well, looks like that Thunderbird is rejected in general, there is nothing special in the messages you attached. Another test would be to override the user agent string. So set preference general.useragent.override. Usually the value is something like
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.8.0
You could see what K9 or eM use or just try setting it to "freenetMail" (without the quotes), or "Horde Application Framework 5", they are both web-clients.

It seems it still works with 81.0.b4 and is broken with 82.0.b3. I did not test the versions between.

Can you use send later on these two versions, and compare the output, do you find anything different?

The attachment contains files from the "Send Later". I do not see a significant difference (besides the Message ID).
The were created from the same source mail ("Als neu bearbeiten").

Seems to me your smtp server is inconsistent if it accepts one but rejects the other.

Well, this could be one explanation.
But, on the other hand, how would you explain that the physically same mail (i.e. copied from Unsent Messages folder TB 81 to TB 89, or vice versa) can be sent using TB up to 81 and failing from TB 82 onwards? There must be some difference in what is sent which irritates th easynme smtp server.

Maybe you can do another test. Send from another account on 81 and 82 to your easyname account, and compare the actually received mails.

The problem is in sending mails from easyname (by Thunderbird) to any other address. Mails sent from any account to my easyname account can be retrieved without any problem by Thunderbird of any version (imap).

There must be some difference in what is sent which irritates th easynme smtp server.

Comment 30 was a test for this, since you were suggesting there are differences in what is sent. Mails content in outbox are not exactly the same as mails sent to smtp server.

In comment 4, you said

Note I can send exactly the same mail from another account (gmx) and from an Android mobile phone, also easyname account.

Does this account work on both 81 and 82?

Yes. I know it is strange. The other accounts do not have problems with either version of TB. I.e. I can use TB (any version) to send messages from my gmx and gmail accounts. Easyname fails. So, it seems the mail composition seems to be generally ok.

On the other side, I could say, easyname smtp accepts mails to be sent from any client, besides TB 82 onwards. So, there must be a difference in the mail compostion.

Unfortunately, I do not know any way to trace the dialog between TB and the smtp server (as it is SSL, wireshark does not help me).

Between TB 81 and TB 82 not much changed in the SMTP handling, see here:
https://hg.mozilla.org/comm-central/log/tip/mailnews/compose/src/nsSmtpProtocol.cpp
The only bug that sticks out is bug 1563891 which introduced SMTPUTF8 support in TBN 82. Looking at the changes there, it should affect you. If you're willing to try two Daily versions, one just before the change and one containing the change, we can rule that out as the cause (or confirm it). Other than that, you should contact Easyname's support and ask whether there issues with sending with Thunderbird.

But before you do all that, run TB on a fresh new profile. You can start TB with -p to create one. Run TB 81 first to confirm it works, then TB 82 to confirm it fails, and then TB 89.

Could you give me a hint which daily versions to try?

The change I mentioned landed on 16-09-2020 in bug 1563891 comment #52. So therefore the last version without the change is this:
http://ftp.mozilla.org/pub/thunderbird/nightly/2020/09/2020-09-15-10-39-35-comm-central/
The first version with the change is this:
http://ftp.mozilla.org/pub/thunderbird/nightly/2020/09/2020-09-16-10-52-32-comm-central/
If you're on Windows, I suggest to download the thunderbird-82.0a1.en-US.win64.zip file, you can unpack that without the need to install and then run it from a cmd prompt, like thunderbird -p, that starts the profile manager and you can create a new profile.
These TB 82 versions use the C++ modules for sending and here you can read how to obtain SMTP logging:
https://wiki.mozilla.org/MailNews:Logging#Main_module_options_within_MailNews
Anyway, I'd be surprised if there were a difference between these two versions.

A note of caution: Once you run the old Daily, it will auto-update, if that happens, you need to unpack the ZIP file again.

I tried with the version from http://ftp.mozilla.org/pub/thunderbird/nightly/2020/09/2020-09-15-10-39-35-comm-central/

creating a new profile and setting

MOZ_LOG=SMTP:5,timestamp
MOZ_LOG_FILE=C:\mnt\d\tmp\Thunderbird_15.log

The send failed as before, the logs are empty (size 0)

I have a .bat file with this content:
set MOZ_LOG=SMTP:5,timestamp
set MOZ_LOG_FILE=D:\Desktop\smtp-log.txt
"C:\Program Files\Mozilla Thunderbird 78\thunderbird.exe" -p
Of course you need to adjust the paths accordingly.

Since you're already using Dailies, you could use mozregression to pinpoint when things started to fail, apparently somewhere between TB 81 and TB 82: https://mozilla.github.io/mozregression/quickstart.html. Usually we have contributors who search for issues for us, but no one but yourself has access to the Easyname provider. The 82 started started on 2020-08-24 and ended on 2020-09-21, so this is the range to try. If you don't want to use mozregression, you can download the Dailies starting here http://ftp.mozilla.org/pub/thunderbird/nightly/2020/ and bisecting the time interval. It's only four weeks, so you should be done in 5 steps.

I have created a mailbox and user for you and your contributors to test.

Account settings:
IMAP: imap.easyname.com
SMTP: smtp.easyname.com
user: 113289mail2
password: only4test
Email address: Thunderbird@familie-leininger.at

Hope this helps for testing

Alice, the reporter claims that somewhere during TB 81 beta and TB 82 beta sending via the server specified in comment #39 started failing.

We can see the failure in TB 89 beta, the server replies with "Sorry, this message is spam!". Could you test this for us? When it fails, it would be good to get the SMTP log as specified in comment #38. But that's not essential, if you can determine the last good and first bad version we can then get the log ourselves. BTW, according to comment #37, this version here 2020-09-15-10-39-35-comm-central already failed. So if the reporter is correct and TB 81 beta is still working, then the regression range is 2020-08-24 to 2020-09-15.

Flags: needinfo?(alice0775)

Hmm, all this looks like a false alarm to me. Check the Sent box, I've sent this message with TB 89 beta

Message-ID: <0a9e09a8-4c82-f392-882f-530cc183336c@familie-leininger.at>
Date: Thu, 27 May 2021 11:01:54 -0500
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101
 Thunderbird/89.0
Content-Language: en-US
To: h.leininger@gmx.at
From: 113289mail2 <Thunderbird@familie-leininger.at>
Subject: Test TB 89
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

test here we go

Alice, sorry about the noise. I can't see an issue here.

Flags: needinfo?(alice0775)

I have seen the dialogs. I really is very, very strange. It keeps failing from my PC's.

I have informed easyname about several times during the last days, also referring to this bug description.

I am perplexed and disillusioned

Attached image error-message.png —

I tried to send using info from comment 39 and using 89.0b4.

Error Console threw this:

11:28:04.002 NS_ERROR_NOT_AVAILABLE: 2 ActivityManager.jsm:129

11:28:04.976 Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1084

11:28:07.864 mailnews.smtp: Command failed: 550 Sorry, this message is spam!; currentAction=_actionStream SmtpClient.jsm:483:19

11:28:07.865 mailnews.smtp: Message sending failed. SmtpClient.jsm:1242:21

11:28:07.865 mailnews.send: Sending failed;
The message could not be sent using Outgoing server (SMTP) smtp.easyname.com for an unknown reason. Please verify that your Outgoing server (SMTP) settings are correct and try again., exitCode=2153066725, originalMsgURI= MessageSend.jsm:321:27

11:28:07.865 Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1084

I used thunderbird@familie-leininger.at as the username as using 113289mail2 failed to complete the new account creation steps. I used only4test as the password.

@ Arthur K.
your experience matches mine. But it seems Jose (comment 41) dird not have problems. ???

Just tested using Portable TB 78.10.2 and it went through just fine. Nothing in Error Console.

Well, I can confirm bug 1563891 is the culprit. I tested http://ftp.mozilla.org/pub/thunderbird/nightly/2020/09/2020-09-15-10-39-35-comm-central/
and it sent through just fine. Sending with http://ftp.mozilla.org/pub/thunderbird/nightly/2020/09/2020-09-16-10-52-32-comm-central/ failed with the same error from comment 43.

So 20200915103935 works.

Build Configuration
Thunderbird Daily 82.0a1 - 20200915103935
Thunderbird Daily source
https://hg.mozilla.org/comm-central/rev/768dcaaaefe9ec9abafd0dd087b67fc666b4ccf0
Platform source
https://hg.mozilla.org/mozilla-central/rev/24b917577203190f0e1c0b21f6ffaf4d07f1e8f6

We're chasing a ghost here. The version of 15-09-2020 doesn't work according to comment #37 but works according to comment #46. I've just sent an e-mail with the version of 16-09-2020 which supposedly doesn't work. I have the impression that the provider Easyname rejects messages at random. You should get an SMTP log following comment #38.

Attached file tbird_log.txt.moz_log (obsolete) —

Your logging method from comment 38 didn't work for me. Had to use the logging methods from here: https://www.lifewire.com/pop-imap-smtp-traffic-thunderbird-1173156 and it seemed to capture something.

Please see attached file. Hope there's something useful there.

Tested using 90.0a.

Thunderbird Daily 90.0a1 - 20210527101654
Thunderbird Daily source https://hg.mozilla.org/comm-central/rev/49766c5f681291ac14e06712535da14bcc49a319
Platform source https://hg.mozilla.org/mozilla-central/rev/b1d195d012a4af03c9cd001af57bc3d4d95396cd

Attached file tbird_log.txt.moz_log —

Sorry. Just realized it's got both IMAP and SMTP. The first log file I sent was only capturing IMAP not SMTP. Here's a newer log using that 82.0a build that fails for me.

That's again a pure IMAP log. Please note that SMTP has moved from the "old" C++ modules to the "new" JS modules in TB 87. So logging is different depending on which version you're using. We already have logging for more recent versions, see comment #4 and following comments. To present new evidence, we'd need to look at an SMTP log from a 82 that doesn't work.

(In reply to José M. Muñoz from comment #50)

That's again a pure IMAP log. Please note that SMTP has moved from the "old" C++ modules to the "new" JS modules in TB 87. So logging is different depending on which version you're using. We already have logging for more recent versions, see comment #4 and following comments. To present new evidence, we'd need to look at an SMTP log from a 82 that doesn't work.

The attached log from comment 49 is from the 82.0a build that fails for me.

Attachment #9223853 - Attachment is obsolete: true

(In reply to José M. Muñoz from comment #50)

That's again a pure IMAP log. Please note that SMTP has moved from the "old" C++ modules to the "new" JS modules in TB 87. So logging is different depending on which version you're using. We already have logging for more recent versions, see comment #4 and following comments. To present new evidence, we'd need to look at an SMTP log from a 82 that doesn't work.

You want me to also try getting a proper SMTP log using 90.0a?

(In reply to Arthur K. [He/Him] from comment #51)

The attached log from comment 49 is from the 82.0a build that fails for me.

OK, but as I said, that's a pure IMAP log. No SMTP logging in there, none at all. SMTP logging has this string "I/SMTP SMTP" a bazillion times, for example:
2021-05-27 20:19:22.309000 UTC - [(null) 10576: Main Thread]: I/SMTP SMTP Send: EHLO [192.168.1.18]

BTW, this works:
set MOZ_LOG=SMTP:5,timestamp
set MOZ_LOG_FILE=D:\Desktop\smtp-log.txt
"C:\Program Files\Mozilla Thunderbird 78\thunderbird.exe" -p

A real SMTP log of the TB 82 version that works (allegedly 15-09-2020) and the one that fails (apparently 16-09-2020) would be good. I'd do it myself, but it doesn't fail for me. In the end, I suspect the log will say "Sorry, this message is spam!" which will just prove that the provider rejects messages at random, perhaps based on the senders IP address.

I have my batch file set up as such:

set MOZ_LOG=SMTP:5,timestamp
set MOZ_LOG_FILE=C:\Users\Art\Desktop\smtp.txt
"C:\Users\Art\Desktop\thunderbird\thunderbird.exe" -p

When I do the test and TB fails, I then close TB 82.0a but the logs are all 0KB (empty). Got any other ideas?

Attached file 15-09-2020.txt.moz_log (obsolete) —

Log file with binary from 15-09-2020. Successful send.

Attached file 16-09-2020.txt.moz_log (obsolete) —

Log file with binary from 16-09-2020. Unsuccessful send. I got the "this is SPAM message", but nothing points to it in the log.

Note: I managed to reproduce the issue this time by using a different network (IP address).

Comparing the two log files, the only difference is this line:
I/SMTP SMTP Send: MAIL FROM:<Thunderbird@familie-leininger.at> BODY=8BITMIME SMTPUTF8 SIZE=462

And that log certainly stops short of the last line.

Looks like it's really regressed by bug 1563891. But as I said before, the behaviour appears to be somewhat random, maybe depended on the IP address of the client.

I guess when the reported said that the 15-09 version didn't work for him, he had started it a second time and it already updated to the current Daily. I warned you! I had to unpack various times since during the second run it's already updated.

Ping and Gene, can you try using the details from comment #39.

Flags: needinfo?(remotenonsense)
Flags: needinfo?(gds)
Attached file 15-09-2020.txt.moz_log —

Hmm, as Arthur observed, the log files don't seem to get flushed, here a more complete one.

Attachment #9223888 - Attachment is obsolete: true
Attachment #9223892 - Attachment is obsolete: true
Attached file 16-09-2020.txt.moz_log —

And here the failing one.

15-09 has:
2021-05-27 23:10:00.336000 UTC - [(null) 12480: Main Thread]: I/SMTP SMTP entering state: 0
2021-05-27 23:10:00.336000 UTC - [(null) 12480: Main Thread]: I/SMTP SMTP Response: 250 OK id=1lmP8Z-0002dH-Sf
2021-05-27 23:10:00.336000 UTC - [(null) 12480: Main Thread]: I/SMTP SMTP entering state: 9
2021-05-27 23:10:00.336000 UTC - [(null) 12480: Main Thread]: I/SMTP SMTP Send: QUIT

16-09 has:
2021-05-27 23:04:47.057000 UTC - [(null) 9936: Main Thread]: I/SMTP SMTP entering state: 0
2021-05-27 23:04:47.057000 UTC - [(null) 9936: Main Thread]: I/SMTP SMTP Response: 550 Sorry, this message is spam!
2021-05-27 23:04:47.057000 UTC - [(null) 9936: Main Thread]: I/SMTP SMTP entering state: 9
2021-05-27 23:04:49.113000 UTC - [(null) 9936: Main Thread]: I/SMTP SMTP Send: QUIT

The preceding difference is the SMTPUTF8 bit.

Not sure if it's related to SMTPUTF8, I tried to comment out https://searchfox.org/comm-central/rev/7dc9ab8a012750252902a311da4e0244285c39b8/mailnews/compose/src/SmtpClient.jsm#246 on trunk, but in both cases I got mailnews.smtp: S: 451 Sorry, you are spamming!

Flags: needinfo?(remotenonsense)

I see the "you are spamming" even with TB 78 which doesn't have SMTPUTF8 support. Also, see it with "Claws Mail 3.17.5" which also doesn't support SMTPUTF8 it appears. (I just installed Claws for testing this since the reporter(s) claim there is no problem sending with other random clients.)

Here's the claws SMTP log that looks very much like the TB SMTP log:

* Account '113289mail2@imap.easyname.com': Connecting to SMTP server: smtp.easyname.com:465...
[2021-05-28 02:22:59] SMTP< 220 mx.easyname.com ESMTP welcomes you.
[2021-05-28 02:22:59] ESMTP> EHLO xps17lt
[2021-05-28 02:22:59] ESMTP< 250-mx.easyname.com Hello 047-040-228-000.res.spectrum.com [47.40.228.0]
[2021-05-28 02:22:59] ESMTP< 250-SIZE 104857600
[2021-05-28 02:22:59] ESMTP< 250-8BITMIME
[2021-05-28 02:22:59] ESMTP< 250-AUTH PLAIN LOGIN
[2021-05-28 02:22:59] ESMTP< 250-CHUNKING
[2021-05-28 02:22:59] ESMTP< 250-SMTPUTF8
[2021-05-28 02:22:59] ESMTP< 250 HELP
[2021-05-28 02:22:59] ESMTP> AUTH LOGIN
[2021-05-28 02:22:59] ESMTP< 334 VXNlcm5hbWU6
[2021-05-28 02:22:59] ESMTP> [USERID]
[2021-05-28 02:22:59] ESMTP< 334 UGFzc3dvcmQ6
[2021-05-28 02:22:59] ESMTP> [PASSWORD]
[2021-05-28 02:23:00] ESMTP< 235 Authentication succeeded
[2021-05-28 02:23:00] ESMTP> MAIL FROM:<Thunderbird@familie-leininger.at> SIZE=339
[2021-05-28 02:23:00] SMTP< 250 OK
[2021-05-28 02:23:00] SMTP> RCPT TO:<gds@chartertn.net>
[2021-05-28 02:23:23] SMTP< 451 Sorry, you are spamming!
** error occurred on SMTP session
*** Error occurred while sending the message:
451 Sorry, you are spamming!

What is the "correct" parameter for "EHLO"? Above it is my hostname. Tb sends the IP address of the hostname. Could that be what the problem is? Also, note that there is a 23 second delay between sending RCPT TO and the "Sorry, you are spamming" response. I see this with TB too. Others TB logs attached above seem to successfully send the message but the server responds with little delay with "Sorry, this is spam".

Not sure is this might be something that an "easyname" user might need to configure to get email to send:
https://www.easyname.com/en/support/email/184-what-is-spf-spam-protection

Flags: needinfo?(gds)

FYI:
these are the current DNS settings (automatically generated by easyname, I did not change anything)

familie-leininger.at SOA ns1.easyname.eu hostmaster@easyname.eu 2021052702
familie-leininger.at NS ns1.easyname.eu
familie-leininger.at NS ns2.easyname.eu
familie-leininger.at TXT v=spf1 include:spf.easyname.com -all
familie-leininger.at MX mx01.easyname.eu
familie-leininger.at MX mx02.easyname.eu
*.familie-leininger.at A 185.51.8.50
autoconfig.familie-leininger.at CNAME mailconfig.easyname.com
autodiscover.familie-leininger.at CNAME mailconfig.easyname.com
easyname._domainkey.familie-leininger.at CNAME dkim.easyname.com 3600 familie-leininger.at A 185.51.8.50
mycloud.familie-leininger.at A 185.51.8.50
mycloud.familie-leininger.at AAAA 2a01:aee0:0:9::11
webtrees-test.familie-leininger.at A 185.51.8.50
webtrees.familie-leininger.at A 185.51.8.50
www.familie-leininger.at A 185.51.8.50

Today's answer from easyname support (support@easyname.com, Ticketnumber: 2094035), translated to English:

// begin citation
The sending of mails is not possible due to a too high spamcore:

Warning: SPAMCHECK: (spamassassin) score: 14.1 / report:
EN_RELAYCOUNTRY_GOOD=-0.5,
HTML_MESSAGE=0.3,
KHOP_HELO_FCRDNS=0.399,
MIME_HTML_ONLY=1.5,
RCVD_IN_PBL=0.001,
RCVD_IN_ZEN_LASTEXTERNAL=8,
RDNS_DYNAMIC=0.363,
SPF_FAIL=4,
TVD_RCVD_IP=0.001

Where the deciding scores are these:
RCVD_IN_ZEN_LASTEXTERNAL=8,
SPF_FAIL=4,

Apparently the IP of the outgoing client was or is listed on one or more blacklists.

// end citation

I do not understand, because the IP of the outgoing client is the same for every client on my system.

This provider seems a little fishy. Five people from different countries tried (me: Mexico and others via VPN, Ping(?), Gene: US, reporter: Austria(?), Arthur(?)) and it failed in most cases, and three people (me + Arthur + reporter) did observe different behaviour on the 15-09 vs. 16-09 versions without/with SMTPUTF8. The only reliable IP not rejected was in fact the one from Mexico ;-) - Go figure!

As Gene stated, there is a delay in the sending process which I also noticed visually. So it seems that the provider goes off checking the blacklists and maybe in some cases gets a timeout, in which case they make an arbitrary decision. That's of course a hypothesis.

The provider asked me about an hour ago to try with another client so that he could check the logs (Would it please be possible to send a test mail via a working client so I can check this in the logs?)

I have done so, sending by eM client and informed him about.

Attached file TB89-beta4.txt —

Here is a successful send log with TB 89 beta 4 from IP 169.57.90.215 (Mexico).

Attachment #9223857 - Attachment mime type: application/octet-stream → text/plain

(In reply to José M. Muñoz from comment #63)

This provider seems a little fishy. Five people from different countries tried (me: Mexico and others via VPN, Ping(?), Gene: US, reporter: Austria(?), Arthur(?)) and it failed in most cases, and three people (me + Arthur + reporter) did observe different behaviour on the 15-09 vs. 16-09 versions without/with SMTPUTF8. The only reliable IP not rejected was in fact the one from Mexico ;-) - Go figure!

As Gene stated, there is a delay in the sending process which I also noticed visually. So it seems that the provider goes off checking the blacklists and maybe in some cases gets a timeout, in which case they make an arbitrary decision. That's of course a hypothesis.

I agree there certainly is something more to this than bug 1563891. If your theory is right about blacklist checking, it could explain some of this behavior. Seeing as how it can be repro'ed using 78.x, as Gene said, it's got to be some weird syzygy of things such that, when just right, it fails.

It would be great to get these easyname.com folks involved here and have them watch in real-time what's happening on their end. I cannot explain why 20200915 worked for me and 20200916 didn't. If bug 1563891 is the only thing that was different between the two but I was using the same IP, then some other very subtle part of this issue is at play. The "This is spam!" certainly puts the ball in easyname's court as it could be a blacklist or SPF if it's being farmed out to some 3rd party service provider.

Has anyone else noticed that when going through the account creations wizard using the 202009xx 82.0a and plugging in the info from comment 39 that it takes a REALLY long time for it to respond? 89.0b4 seems a lot snappier with the process but it has the newer SMTP client. Just mentioning as I observed it and it might be related.

My original test in comment 60 with claws and tb clients was using my ISP (Spectrum). The easyname server must have Spectrum blacklisted since it responds to "rcpt to:" with "Sorry, you are spamming" and the actual message is never even sent.

I then tried sending using just openssl and see exactly the same thing, i.e., fails on "rcpt to:"

I then setup my wifi so it is tethered to my cell phone so my ISP's network is out of the picture. The cell phone network is AT&T. When I manually send the email using the tether, the "rcpt to:" works OK and I can sent the message successfully via openssl command line. However, this is only successful if I leave off the "SMTPUTF8" from the "mail from:" string.

This works:

220 mx.easyname.com ESMTP welcomes you.
ehlo dude
250-mx.easyname.com Hello dude [107.112.218.190]
250-SIZE 104857600
250-8BITMIME
250-AUTH PLAIN LOGIN
250-CHUNKING
250-SMTPUTF8
250 HELP
AUTH PLAIN AHRodW5kZXJiaXJkQGZhbWlsaWUtbGVpbmluZ2VyLmF0AG9ubHk0dGVzdA==
235 Authentication succeeded
MAIL FROM:<Thunderbird@familie-leininger.at> BODY=8BITMIME SIZE=443
250 OK
rcpt to: gds@chartertn.net
250 Accepted
data
354 Enter message, ending with "." on a line by itself
Subject: test

test  
.
250 OK id=1lmkgY-0002dd-Oh

but this fails:

220 mx.easyname.com ESMTP welcomes you.
ehlo dude
250-mx.easyname.com Hello dude [107.112.218.190]
250-SIZE 104857600
250-8BITMIME
250-AUTH PLAIN LOGIN
250-CHUNKING
250-SMTPUTF8
250 HELP
AUTH PLAIN AHRodW5kZXJiaXJkQGZhbWlsaWUtbGVpbmluZ2VyLmF0AG9ubHk0dGVzdA==
235 Authentication succeeded
MAIL FROM:<Thunderbird@familie-leininger.at> BODY=8BITMIME SMTPUTF8 SIZE=443
250 OK
rcpt to: gds@chartertn.net
250 Accepted
data
354 Enter message, ending with "." on a line by itself
Subject: test with smtputf8

test test test
.
550 Sorry, this message is spam!

So definitely the SMTPUTF8 string is causing a problem for the easyname server. See bug 1563891 comment 60

So I was looking over RFC6531 and saw this section. Possibly related?

3.7.2. Mail eXchangers

If multiple DNS MX records are used to specify multiple servers for a
domain (as described in Section 5 of RFC 5321 [RFC5321]), it is
strongly advised that all or none of them SHOULD support the SMTPUTF8
extension. Otherwise, unexpected rejections can happen during
temporary or permanent failures, which users might perceive as
serious reliability issues.

https://datatracker.ietf.org/doc/html/rfc6531#section-3.7.2

(In reply to Helmut Leininger from comment #39)

I have created a mailbox and user for you and your contributors to test.

Hope this helps for testing

I sent an email to the support folks asking if they'd be willing to get involved in this bug directly and also brought up the possibility of what I mention in comment 68 (if it's not a blacklist / IP address issue). Hope they respond.

(In reply to Arthur K. [He/Him] from comment #69)

(In reply to Helmut Leininger from comment #39)

I have created a mailbox and user for you and your contributors to test.

Hope this helps for testing

I sent an email to the support folks asking if they'd be willing to get involved in this bug directly and also brought up the possibility of what I mention in comment 68 (if it's not a blacklist / IP address issue). Hope they respond.

Thank you for your efforts. I believe that this puzzle can be solved only through cooperation of both sides.

Looking at https://datatracker.ietf.org/doc/html/rfc6531#section-3.4 it says:

If the SMTPUTF8-aware SMTP client is aware that neither the envelope nor the message being sent
requires any of the SMTPUTF8 extension capabilities, it SHOULD NOT supply the SMTPUTF8
parameter with the MAIL command.

TB always includes the SMTPUTF8 parameter on the MAIL command if the EHLO response indicates it is supported even when the message and headers contains only US-ASCII. TB is not in total violation of the spec since this is a "SHOULD NOT" and it works OK with pure US-ASCII addresses for gmail and courier that both support SMTPUTF8. (The definition of "SHOULD NOT" is here: https://datatracker.ietf.org/doc/html/rfc2119.)

It is possible to change the TB code so that SMTPUTF8 is only added to the MAIL command if there are actually non-US-ASCII characters in the recipient addresses. I'm not sure if this is correct but it does work-around the "Easyname" issue of this bug. I made the patch on the deprecated c++ file instead of the new JavaScript port since I am more familiar with the c++ version.

P/S: Today it seems my ISP is no longer blacklisted and I can send emails in TB via the Easyname account as long as the SMTPUTF8 parameter is excluded from the "MAIL TO:" command.

(In reply to Ping Chen (:rnons) from comment #59)

Not sure if it's related to SMTPUTF8, I tried to comment out https://searchfox.org/comm-central/rev/7dc9ab8a012750252902a311da4e0244285c39b8/mailnews/compose/src/SmtpClient.jsm#246 on trunk, but in both cases I got mailnews.smtp: S: 451 Sorry, you are spamming!

Yes, that occurs in response to "RCPT TO:" with about 23 second delay and seems to be due to blacklisting and not SMTPUTF8 parameter.

The other error that occurs after the message "DATA" is sent, "Sorry, this message is spam!" only occurs when SMTPUTF8 is present in the "MAIL TO" command. But you have to be off the hypothetical "blacklist" to even get to the point where the message DATA is sent.

TB always includes the SMTPUTF8 parameter on the MAIL command if the EHLO response indicates it is supported even when the message and headers contains only US-ASCII.

Yes, I was wondering whether it should be sent unconditionally even when not necessary. There's still time to modify the behaviour before it hits the next ESR. That said, sending from Mexico always works, SMTPUTF8 or not, so they must have a very strange SPAM algorithm: Sometimes everything is blocked, sometimes it depends on SMTPUTF8 and sometimes nothing is blocked.

Right now we only check the addresses in the envelope for non-ASCII and convert to punycode if SMTPUTF8 is not present. I don't understand the RFC well enough to know if that's enough or if we need to check the messages body content too for any non-US-ASCII. If so, that would make it a lot more complex.

Alessandro, Do you have any insights into the issues around this bug regarding SMTPUTF8? Specifically regarding comment 71 and down. I'm asking since you originally requested STMPUTF8 be supported in TB in bug 1563891.

Flags: needinfo?(vesely)
Attached image why-is-it-null.png —

So here's a new twist. I saw gene's mention that it worked for him today from his ISP in comment 71. I once again tried to see what would happen using 82.0a from 5-16-2020 but it failed. Then it auto updated itself to 90.0a from 29-5-2021 and failed again.

Then I wondered: What happens if I try to send from Thunderbird@familie-leininger.at to my gmail.com address. I got a bit of a different server response in 90.0a. It's saying I was spamming but that the recipient was "(null)". Obviously not but...huh?

Below is what 90.a spit out in error console.

Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1084
mailnews.smtp: Socket timed out. SmtpClient.jsm:512:17
mailnews.smtp: Command failed: 451 Sorry, you are spamming!; currentAction=_actionRCPT SmtpClient.jsm:502:19
mailnews.smtp: RCPT TO failed for: <removed_to_prevent_spam>@gmail.com SmtpClient.jsm:1191:19
mailnews.smtp: Can't send mail - all recipients were rejected SmtpClient.jsm:1204:21
mailnews.send: Sending failed; An error occurred while sending mail. The mail server responded:
Sorry, you are spamming!.
Please check the message recipient "(null)" and try again., exitCode=2153066783, originalMsgURI= MessageSend.jsm:321:27
Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1084

So at this point, I went back and set up 82.0a 5-16-2020 again, made sure it couldn't auto-update to 90.0a and, again, tried to send to my gmail.com account. Error Console spit out nothing at all but I got the error message "An error occurred while sending mail. The mail server responded: Sorry, you are spamming!. Please check the message recipient "<removed_to_prevent_spam>@gmail.com" and try again." So, unlike what 90.0a did, it wasn't a "(null)" recipient like 90.0a.

I'm not trying to go off the reservation with this new info but it's weird for sure and just wanted to share this result.

Alessandro, Do you have any insights into the issues around this bug regarding SMTPUTF8? Specifically regarding comment 71 and down.

I read your description of the behavior in comment 67. It looks like a bug in their implementation. Does it work at all if there is something non-ASCII in the header? I wrote to their support team, but had no reply.

Flags: needinfo?(vesely)

(In reply to Alessandro Vesely from comment #77)

I wrote to their support team, but had no reply.

Same here, no reply and not likely to be one. Helmut, do you think, since you're a customer, you could try reaching out again and pointing their support folks to this bug?

Hi,
Thanks for your support. I have written to the support team again, expressing my disappointment of how the treated customer problems and requesting them definitely to cooperate/take part in the solution oof this bug.

I have just done an additonal test. I am on vacation now, which means in another network and maybe get assinged another smtp server.

The nslookup from here for smtp.easyname.com delicers 77.244.243.66
Sending a mail with TB 89.0b4 was ok.

I am not sure, but I think i Recall I was getting another addres for smtp.easyname.com at home.

(In reply to Helmut Leininger from comment #79)

Hi,
Thanks for your support. I have written to the support team again, expressing my disappointment of how the treated customer problems and requesting them definitely to cooperate/take part in the solution oof this bug.

Thanks Helmut. I would think that their non-response, to those of us non-customers who've reached out to them, would be perceived as an attempt at social engineering / phishing or something. It is more and more looking like misconfiguration of some kind on their side though.

(In reply to Helmut Leininger from comment #80)

I have just done an additonal test. I am on vacation now, which means in another network and maybe get assinged another smtp server.

The nslookup from here for smtp.easyname.com delicers 77.244.243.66
Sending a mail with TB 89.0b4 was ok.

I am not sure, but I think i Recall I was getting another addres for smtp.easyname.com at home.

Makes sense. They must have multi SMTP servers serving IPs depending on geolocation but maybe, in the end, synching up with a primary server that has anger management issues.

The up to now last response by easyname to me, translated to English

(In reply to Helmut Leininger from comment #83)

The up to now last response by easyname to me, translated to English

Well, it's something. Seems by their own admission they're restrictive on their spam blocking. If it helps these tech folks to check their logs, I am coming from 69.243.140.80 / 2601:243:1d00:184e:6916:286f:86f:c135 but I am primarily on IPv6.

I got a reply back from the easyname tech support team:

Hi Arthur,

Thank you for assisting and contacting us. Unfortunately for us, this is indeed a bug with our mailsystem, related to smtputf8.

We are relying on spamassassin and rspamd, prefering (score-wise) rspamd. But as it happens, rspamd doesn't support International Domain Names and instantly flags the message as spam. Therefore we skip scanning SMTPUTF8 flagged messages with rspamd.

While this approach works well on our setup, in this particular case, SA marks messages from this customer as spam (including SPF_FAIL, spamscore 14.5). So we deny the message and thunderbird displays our generic "you are spamming" error for such a situation.

We will take a look at this and fix it in the next few days.

So they are now aware of it and working toward resolution. Woot!

Hi all,
many thanks for supporting me even if it turned out to be not a Thunderbird failure. Without your great expertise and your eagerness my problem would not have come near a resolution.

Thanks again

(In reply to Helmut Leininger from comment #86)

Hi all,
many thanks for supporting me even if it turned out to be not a Thunderbird failure. Without your great expertise and your eagerness my problem would not have come near a resolution.

Thanks again

Welcome. This would likely have hit users from other ISPs that were using similar tools or setups. The easyname support person working with me said he'd email me once they have the fix running live in order for me/us to test.

I think this is the issue with rspamd they are referring to: https://github.com/rspamd/rspamd/issues/1991
It's not clear to me if the problem in rspamd is with the SMTPUTF8 parameter itself being sent even when it is strictly not needed or if the rspamd bug occurs when an International Domain Name is actually present in a header. I know my testing and AFAIK others testing has just been with plain old 7-bit US-ASCII email addresses with no special mult-byte UTF-8 chars in them.
Also, why would the SMTPUTF8 parameter be OK when sent from Mexico?

(In reply to gene smith from comment #88)

I think this is the issue with rspamd they are referring to: https://github.com/rspamd/rspamd/issues?q=smtputf8+
It's not clear to me if the problem in rspamd is with the SMTPUTF8 parameter itself being sent even when it is strictly not needed or if the rspamd bug occurs when an International Domain Name is actually present in a header. I know my testing and AFAIK others testing has just been with plain old 7-bit US-ASCII email addresses with no special mult-byte UTF-8 chars in them.
Also, why would the SMTPUTF8 parameter be OK when sent from Mexico?

You want me to pass along that link to the support folks? And yes, why DID Mexico work? That would for sure have fallen into the International Domain Name realm.

I'm not kidding you, attachment 9223975 [details] shows IP 169.57.90.215, check it here: https://www.ip2location.com/demo/169.57.90.215.

(In reply to José M. Muñoz from comment #90)

I'm not kidding you, attachment 9223975 [details] shows IP 169.57.90.215, check it here: https://www.ip2location.com/demo/169.57.90.215.

I believe it but it doesn't make it less weird. Based on https://github.com/rspamd/rspamd/issues/1991, it seems both known and wontfix. But since that issue got closed 2+ years ago, maybe someone will take it up. A user posted to that bug that TB is having issues. I won't hold my breath on them fixing it though. Doesn't seem they want to fix or support SMTPUTF8.

During the last days (on vacation) I tried several times and all mails were transmitted ok.

I have an idea that you might eventually verify. There are two MX records (see comment 61):
mx01.easyname.com -> resolves to 146.255.61.214
mx02.easyname.com -> resolves to 77.244.243.43

All mails that I could send without problems were treated by an mx server 77.244.243.xx (example: ... from mx104.easyname.com ([77.244.243.85]) by mx-ha.gmx.net (mxgmx116 [212.227.17.5] ... )

Could you check if your ok-mails were treated by the servers 77.244.243.x range ? Or also by others from nslookup mx.easyname.com ?

I'm not kidding you...

Maybe the route from Mexico ends up at a different server that doesn't have rspamd running. Just a guess.

A user posted to that bug that TB is having issues.

That user was me. :)

You want me to pass along that link to the support folks?

Couldn't hurt.
You might also suggest to them that they just remove support for SMTPUTF8 from their smtp server(s). I don't know what their SMTP server is but a very common server, Postfix, has fairly simple instruction for enabling/disabling SMTPUTF8:
http://ftp.uma.es/mirror/postfix/doc/SMTPUTF8_README.html.
Since their spam filter add-on rspamd pretty much breaks when it sees SMTPUTF8 parameter or "international" addresses in UTF-8 this couldn't hurt anything while the rspamd bug is present. And it will resolve this TB bug. Then if/when the rspamd bug is fixed, they can turn SMTPUTF8 back on at their SMTP server.

(In reply to gene smith from comment #93)

You might also suggest to them that they just remove support for SMTPUTF8 from their smtp server(s).

I don't think that's the best course of action. Easyname wrote they can use SpamAssassin to scan SMTPUTF8 messages. I saw patches to SA-3.4.1 to support IDN, and 3.4.2 (Debian stock version) supports it quite all-right. (Albeit they may have some problems here and there, for example DKIM signatures with d=foà.it...).

Rspamd sooner or later will have to support SMTPUTF8, or it will just be forgotten. SMTPUTF8 is having a very slow adoption rate, for example compared with 8BITMIME, but it'd be really overhasty to consider it dead now and decide that we can live without it.

(In reply to gene smith from comment #93)

You want me to pass along that link to the support folks?

Couldn't hurt.

For now I've sent them the github link but no response since yesterday.

(In reply to Alessandro Vesely from comment #94)

(In reply to gene smith from comment #93)

You might also suggest to them that they just remove support for SMTPUTF8 from their smtp server(s).

I don't think that's the best course of action. Easyname wrote they can use SpamAssassin to scan SMTPUTF8 messages. I saw patches to SA-3.4.1 to support IDN, and 3.4.2 (Debian stock version) supports it quite all-right. (Albeit they may have some problems here and there, for example DKIM signatures with d=foà.it...).

From comment 85:

We are relying on spamassassin and rspamd, prefering (score-wise) rspamd.

Sound like they are sending each email through both filters. My suggested action (disable SMTPUTF8) would only be temporary until rspamd is fixed or if rspamd is entirely removed leaving only SA. Then SMTPUTF8 could be re-enabled.

Rspamd sooner or later will have to support SMTPUTF8, or it will just be forgotten. SMTPUTF8 is having a very slow adoption rate, for example compared with 8BITMIME, but it'd be really overhasty to consider it dead now and decide that we can live without it.

Again, not saying they should live without it permanently, just temporarily until this issue is resolved since emails sent with SMTPUTF8 parameter set or with multi-byte UTF-8 addresses are rejected by them (except for emails from Mexico for unknown reasons).

Here's an update from easyname in response to pointing them to the github issue:

Yes, this is excatly the bug why we skip scanning with rspamd in case it is a SMTPUTF8 message.
We do not check whether the from/rcpt is really a IDN, we just skip over rspamd in that case all over.

Our spamassassin setup seems fine, we get like 0-3 mails per day that are denied with the same spam-flags.
It looks like as if SA doesn't recognize the mail is sent by an authenticated user and tags it with SPF_FAIL and
RCVD_IN_ZEN_LASTEXTERNAL.

We will take a closer look at this by next week.

I forgot to post this reply from the other week.

Our workaround works quiet well.
We are Austria-/Germany-based, so IDN's and Umlaute (ä, ö, ü, ß) are not so uncommon around our customers.

IDN's indeed do work without SMTPUTF8, but we need it for the local-part of an address.
So disabling SMTPUTF8 would kind of be a last resort option.

We will try to recreate the issue with thunderbird next week and try to get new clues.

Still haven't heard back since the 4th so I'll follow up and see what they say. Gene, have you seen anything improve on your end?

Gene, have you seen anything improve on your end?

Nothing new here. Haven't really looked at this since my comment 96 post. Will wait for what they see when they test with TB "next week".

I can confirm that there is some problem in easyname's configuration.

The last week I have been on holiday 400 km from home -> works with TB 90.b1 (I assume in headings "Received: from.... by mx.easynamer.com with utf8esmtpsa ...." means it was sent with SMTPUTF8 flag)

At home again -> same notebook, sending fails (unknown reason, SMTP-server smtp.easyname.com).

This is the latest response:

Hi!

Not quiet yet. I have a feeling that it is somehow related to SA's handling of network checks.
Our customer told us, that he has been in a different location in the same country and that the problem didnt occure from there.
Client/workstation/settings were the same, just the IP differed.

We also got trouble to recreate this bug triggering.
But this might explain, why we barely see this in our logs.

I am waiting for his feedback regarding IPs, hope this will get us closer to a solution.

So maybe we need to fill up their logs with some errors? I'll ask if that's something they want to try and perhaps on a specific date?

Hi,
this afternoon I got a mail from the providers, saying that they think to have found and solved the problem. I sent a test mail from my account, being at home, and it worked.
I will attach the test mail for the headers to be checked.

(In reply to Helmut Leininger from comment #102)

Hi,
this afternoon I got a mail from the providers, saying that they think to have found and solved the problem. I sent a test mail from my account, being at home, and it worked.
I will attach the test mail for the headers to be checked.

Thanks for reporting back Helmut.

Status: UNCONFIRMED → RESOLVED
Closed: 3 years ago
Resolution: --- → INVALID

See Bug 1726106 comment 28. Looks like the change similar to what is suggested in comment 71 above has been implemented in the new SMTP javascript module. So I think this bug should be changed to be a duplicate of Bug 1726106.

Resolution: INVALID → DUPLICATE
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: