1701804 - Outgoing mail to Earthlink SMTPAUTH immediately failed.

Status: RESOLVED FIXED

(MailNews Core :: Networking: SMTP, defect)

Description

[steve.fritts]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0

Steps to reproduce:

Reinstalled 88.0.1. No change.
Uninstalled 88. Installed 87.0.3 - outgoing mail worked again.
Updated to 88. Failed SMTPAUTH.
Deleted & re-added ACCOUNT SETTINGS under all configs. No change.
Removed McAfee. No change.

Actual results:

Spent 4 hours doing a matrix of versions & conditions. Full reboots, upgrades & Clean re-installs.

Expected results:

should work

Comment 1

[Ping Chen [:rnons] (away)]

Hi, can you please try the following

1. mailnews.smtp.jsmodule to true and mailnews.smtp.loglevel to All
2. send a mail
3. paste logs containing mailnews.smtp here

Thanks.

Comment 2

[steve.fritts]

I am not a T-Bird programmer. I am a user.
Where are these parameters located?
How do I change them?

Comment 3

[steve.fritts]

Also, the GET MESSAGES operation is inconsistent and very intermittent.
Sometimes it works, but not often. It will collect messages on its own, but not when I manually select the button.

Comment 4

[Ping Chen [:rnons] (away)]

(In reply to steve.fritts from comment #2)

I am not a T-Bird programmer. I am a user.
Where are these parameters located?
How do I change them?

1. On the Options page, search for "config editor", click the "config editor" button to open it. Then search for mailnews.smtp.loglevel, double click it and set the value to All.
2. Ctrl+Shift+I to open the devtools
3. send a mail
4. in the Console tab of the devtools, search for mailnews.smtp
5. paste logs containing mailnews.smtp here

Your report and feedback will help TB get better, thanks.

Comment 5

[steve.fritts]

I just tried SENDING but it's been sitting there on SENDING MESSAGE screen for several minutes. (NOTE: I added my GMAIL account and it works just fine sending & receiving)

Some code in handling SMTPAUTH.EARTHLINK.NET has changed in 88.0.b1 that was OK in 87.0.b3.

And even my manual GET MESSAGES from POP3.MINDSPRING.COM is not working properly.

So there is where the coders need to compare 88 to 87 and see what changed?

Comment 6

[steve.fritts]

Is this what you are looking for?

Lightning: Couldn't find (UTC-05:00) Eastern Time (US & Canada) 6 CalTimezoneService.jsm:206
NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIAutoCompleteInput.popup] LoginManagerChild.jsm:194
Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1033
mailnews.smtp: EHLO not successful, trying HELO localhost SmtpClient.jsm:792:19
mailnews.smtp: Socket timed out. SmtpClient.jsm:478:17

Comment 7

[steve.fritts]

01:46:14.132
mailnews.smtp: Socket timed out. SmtpClient.jsm:478:17
01:46:25.437 Flattening unsafe node (descendants are preserved). Element: tel:(515)%20604-9000. calViewUtils.jsm:424:35
01:46:25.437 Flattening unsafe node (descendants are preserved). Element: http:. calViewUtils.jsm:424:35
01:46:34.388
Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1033
01:46:44.952 ‘src’ attribute of <script> element is not a valid URI: “/80e800b4c89ca8c27472fbfecefb36a0ac800a341e7dff618d5026b0a4bfcd9c/ns.js” 5 blank:1:1
01:46:50.616 Flattening unsafe node (descendants are preserved). Element: [e-mail]. calViewUtils.jsm:424:35
01:46:50.617 Flattening unsafe node (descendants are preserved). Element: https:. calViewUtils.jsm:424:35
01:46:50.617 Flattening unsafe node (descendants are preserved). Element: sip:[e-mail]. calViewUtils.jsm:424:35
01:46:50.617 Flattening unsafe node (descendants are preserved). Element: tel:%2b12106069466,,01594252686%23%2301. calViewUtils.jsm:424:35
01:46:50.617 Flattening unsafe node (descendants are preserved). Element: tel:%2b18662827366,,01594252686%23%2301. calViewUtils.jsm:424:35
01:46:50.617 Flattening unsafe node (descendants are preserved). Element: https:. 5 calViewUtils.jsm:424:35
01:47:26.420 Lightning: Couldn't find (UTC-05:00) Eastern Time (US & Canada) 6 CalTimezoneService.jsm:206
01:54:45.740
NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIAutoCompleteInput.popup] LoginManagerChild.jsm:194
01:55:01.704
Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1033
01:55:02.026 mailnews.smtp: EHLO not successful, trying HELO localhost SmtpClient.jsm:792:19
01:55:12.037
mailnews.smtp: Socket timed out. SmtpClient.jsm:478:17
02:09:05.427
<Provider> does not support changing store on the fly. It is most likely that you see this error because you updated to Redux 2.x and React Redux 2.x which no longer hot reload reducers automatically. See https://github.com/reactjs/react-redux/releases/tag/v2.0.0 for the migration instructions. react-redux.js:881:13
02:10:09.643 window.controllers/Controllers is deprecated. Do not use it for UA detection. blank
02:10:18.629
NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIAutoCompleteInput.popup] LoginManagerChild.jsm:194
02:10:24.600
Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1033
02:10:24.869 mailnews.smtp: EHLO not successful, trying HELO localhost SmtpClient.jsm:792:19
02:10:34.879
mailnews.smtp: Socket timed out. SmtpClient.jsm:478:17

​

Comment 8

[Ping Chen [:rnons] (away)]

Thanks, I only need those containing mailnews.smtp, you can filter the logs in the Console tab next time. Currently the only useful log I see is Socket timed out, since it works for GMAIL but fails for EARTHLINK, I don't have any immediate idea, will investigate the code.

Comment 9

[Ping Chen [:rnons] (away)]

Hi Steve, can you please try again? Clear all logs in the Console tab first, then send a mail, the first few lines of useful log should look like

mailnews.smtp: S: 220...
mailnews.smtp: C: EHLO...

Comment 10

[steve.fritts]

03:37:38.249 mailnews.smtp: S: 220-elasmtp-mealy.atl.sa.earthlink.net ESMTP Exim 4 #1 Wed, 31 Mar 2021 03:37:38 -0400
220-NO UCE. EarthLink does not authorize the use of its computers or network
220 equipment to accept, transmit, or distribute unsolicited e-mail.

SmtpClient.jsm:415:17
03:37:38.251 mailnews.smtp: C: EHLO localhost SmtpClient.jsm:548:19
03:37:38.279 mailnews.smtp: S: 550 error in helo

SmtpClient.jsm:415:17
03:37:38.280 mailnews.smtp: EHLO not successful, trying HELO localhost SmtpClient.jsm:792:19
03:37:38.280 mailnews.smtp: C: HELO localhost SmtpClient.jsm:548:19

​

Comment 11

[steve.fritts]

one more time

03:41:27.252 mailnews.smtp: S: 220-elasmtp-masked.atl.sa.earthlink.net ESMTP Exim 4 #1 Wed, 31 Mar 2021 03:41:27 -0400
220-NO UCE. EarthLink does not authorize the use of its computers or network
220 equipment to accept, transmit, or distribute unsolicited e-mail.

SmtpClient.jsm:415:17
03:41:27.255 mailnews.smtp: C: EHLO localhost SmtpClient.jsm:548:19
03:41:27.293 mailnews.smtp: S: 550 error in helo

SmtpClient.jsm:415:17
03:41:27.294 mailnews.smtp: EHLO not successful, trying HELO localhost SmtpClient.jsm:792:19
03:41:27.295 mailnews.smtp: C: HELO localhost SmtpClient.jsm:548:19

Comment 12

[steve.fritts]

And .. after I initially OPEN the TBird app and it does an initial download of email, it fails on almost every subsequent attempt to Manually GET MESSAGES.

03:45:50.152 An iframe which has both allow-top-navigation and allow-top-navigation-by-user-activation for its sandbox attribute will permit top navigations. Drafts
03:45:55.433 An iframe which has both allow-top-navigation and allow-top-navigation-by-user-activation for its sandbox attribute will permit top navigations. blank
03:45:59.854
Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1033
03:46:10.885
Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1033
03:46:15.316
Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1033

So the code in TBird has changed on BOTH receiving the Incoming Mail retrieving from POP3.MINDSPRING.COM and Sending to SMTPAUTH.EARTHLINK.NET.

If I reload 87.0.b3, that seems to still work just fine. But my saved-mail in v88.0 Profile will not work back down on 87.0.3.

Comment 13

[steve.fritts]

Here's the user message.

Sending of password for user sfritts did not succeed. Mail server pop.mindspring.com responded: bad password or unknown username

So something in the code is sending a bad PW to Mindspring & Earthlink?

This has worked flawlessly for years & years; what changed?

Comment 14

[Ping Chen [:rnons] (away)]

Thanks a lot, we introduced a new implementation in bug 1661694, sorry it broke your use cases. I will make a possible fix later today or tomorrow.

Comment 15

[Ping Chen [:rnons] (away)]

Attachment: custom-hello-argument.png

I know I have taken much of your time, but can you try another thing? From your logs, the problem is here

03:41:27.255 mailnews.smtp: C: EHLO localhost
03:41:27.293 mailnews.smtp: S: 550 error in helo

It seems Earthlink's SMTP server doesn't like the localhost part. Can you please open the Config Editor again, search for mail.smtpserver.smtp, there may be many of them, look for the key ends with .hostname and value contains earthlink.net. Suppose it's mail.smtpserver.smtp1.hostname. Now add a new String entry named mail.smtpserver.smtp1.hello_argument, set the value to your ip address, or any valid ip address, like 1.1.1.1. Now try sending again.

Comment 16

[steve.fritts]

I'm not happy that it's broken, but very happy to have you help troubleshoot and try to fix. Thanks.
So I just entered YOUR FIX and IT WORKED! I'm Now able to SEND msgs.
:-)

Comment 17

[Ping Chen [:rnons] (away)]

Attachment: Bug 1701804 - Send ip address in EHLO command in SmtpClient.jsm. r=mkmelin

Comment 18

[steve.fritts]

UPDATE -

• Manually SENDING mail has been working since the last change.

• However, GET MESSAGES still fails intermittently. I get this message pop-up: "Sending of password for user sfritts did not succeed. Mail server pop.mindspring.com responded: bad password or unknown username"

• CLOSING T-Bird and Reopening did not solve this.

• Here's the error-log of attempt to manually retrieve mail.
  23:31:10.849
  Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1033
  23:31:10.860
  Exception { name: "NS_ERROR_NOT_AVAILABLE", message: "Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIDocShell.domWindow]", result: 2147746065, filename: "resource://devtools/server/actors/targets/browsing-context.js", lineNumber: 402, columnNumber: 0, data: null, stack: "get window@resource://devtools/server/actors/targets/browsing-context.js:402:5\n_windowReady@resource://devtools/server/actors/targets/browsing-context.js:1463:24\nwatch@resource://devtools/server/actors/targets/browsing-context.js:1714:25\n_onDocShellCreated/<@resource://devtools/server/actors/targets/browsing-context.js:851:32\nexports.makeInfallible/<@resource://devtools/shared/ThreadSafeDevToolsUtils.js:103:22\nopenWindowPrompt@resource:///actors/PromptParent.jsm:161:19\nreceiveMessage@resource:///actors/PromptParent.jsm:108:21\nopenPrompt@resource://gre/modules/Prompter.jsm:1174:36\nopenPromptSync@resource://gre/modules/Prompter.jsm:1054:10\nalert@resource://gre/modules/Prompter.jsm:1313:17\n", location: XPCWrappedNative_NoHelper }
  ThreadSafeDevToolsUtils.js:82:13
  23:31:10.915
  Exception { name: "NS_ERROR_NOT_AVAILABLE", message: "Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIDocShell.domWindow]", result: 2147746065, filename: "resource://devtools/server/actors/targets/browsing-context.js", lineNumber: 402, columnNumber: 0, data: null, stack: "get window@resource://devtools/server/actors/targets/browsing-context.js:402:5\n_windowReady@resource://devtools/server/actors/targets/browsing-context.js:1463:24\nDebuggerProgressListener.prototype.onWindowCreated<@resource://devtools/server/actors/targets/browsing-context.js:1809:23\nexports.makeInfallible/<@resource://devtools/shared/ThreadSafeDevToolsUtils.js:103:22\nopenWindowPrompt@resource:///actors/PromptParent.jsm:161:19\nreceiveMessage@resource:///actors/PromptParent.jsm:108:21\nopenPrompt@resource://gre/modules/Prompter.jsm:1174:36\nopenPromptSync@resource://gre/modules/Prompter.jsm:1054:10\nalert@resource://gre/modules/Prompter.jsm:1313:17\n", location: XPCWrappedNative_NoHelper }
  ThreadSafeDevToolsUtils.js:82:13
  23:31:12.653
  NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIDocShell.domWindow] browsing-context.js:402
  23:31:12.710
  Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1033
  23:31:12.734
  Exception { name: "NS_ERROR_NOT_AVAILABLE", message: "Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIDocShell.domWindow]", result: 2147746065, filename: "resource://devtools/server/actors/targets/browsing-context.js", lineNumber: 402, columnNumber: 0, data: null, stack: "get window@resource://devtools/server/actors/targets/browsing-context.js:402:5\n_windowReady@resource://devtools/server/actors/targets/browsing-context.js:1463:24\nwatch@resource://devtools/server/actors/targets/browsing-context.js:1714:25\n_onDocShellCreated/<@resource://devtools/server/actors/targets/browsing-context.js:851:32\nexports.makeInfallible/<@resource://devtools/shared/ThreadSafeDevToolsUtils.js:103:22\nopenWindowPrompt@resource:///actors/PromptParent.jsm:161:19\nreceiveMessage@resource:///actors/PromptParent.jsm:108:21\nopenPrompt@resource://gre/modules/Prompter.jsm:1174:36\nopenPromptSync@resource://gre/modules/Prompter.jsm:1054:10\nconfirmEx@resource://gre/modules/Prompter.jsm:1449:10\n_promiseAuthPrompt/<@resource:///modules/MsgAsyncPrompter.jsm:24:18\n_promiseAuthPrompt@resource:///modules/MsgAsyncPrompter.jsm:22:12\nrun@resource:///modules/MsgAsyncPrompter.jsm:47:23\n", location: XPCWrappedNative_NoHelper }
  ThreadSafeDevToolsUtils.js:82:13
  23:31:12.932
  Exception { name: "NS_ERROR_NOT_AVAILABLE", message: "Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIDocShell.domWindow]", result: 2147746065, filename: "resource://devtools/server/actors/targets/browsing-context.js", lineNumber: 402, columnNumber: 0, data: null, stack: "get window@resource://devtools/server/actors/targets/browsing-context.js:402:5\n_windowReady@resource://devtools/server/actors/targets/browsing-context.js:1463:24\nDebuggerProgressListener.prototype.onWindowCreated<@resource://devtools/server/actors/targets/browsing-context.js:1809:23\nexports.makeInfallible/<@resource://devtools/shared/ThreadSafeDevToolsUtils.js:103:22\nopenWindowPrompt@resource:///actors/PromptParent.jsm:161:19\nreceiveMessage@resource:///actors/PromptParent.jsm:108:21\nopenPrompt@resource://gre/modules/Prompter.jsm:1174:36\nopenPromptSync@resource://gre/modules/Prompter.jsm:1054:10\nconfirmEx@resource://gre/modules/Prompter.jsm:1449:10\n_promiseAuthPrompt/<@resource:///modules/MsgAsyncPrompter.jsm:24:18\n_promiseAuthPrompt@resource:///modules/MsgAsyncPrompter.jsm:22:12\nrun@resource:///modules/MsgAsyncPrompter.jsm:47:23\n", location: XPCWrappedNative_NoHelper }
  ThreadSafeDevToolsUtils.js:82:13
  23:31:15.409
  NS_ERROR_NOT_AVAILABLE: Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIDocShell.domWindow]

Comment 19

[Ping Chen [:rnons] (away)]

Hi, POP is a different protocol from SMTP, I don't know much about it. Do you minding filing a separate bug for the problem? So that other people can take a look.

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/512138052532
Send ip address in EHLO command in SmtpClient.jsm. r=mkmelin

Comment 21

[alta88]

This _getHelloArgument() function is rather a fail.

1. Why would the internal network's ip assignment be leaked in this way, totally unnecessarily?
2. The component string needs to end with ";1" to be successful, and even then why should a device's name be leaked like this?

Of course, the string can be overridden via the mail.smtpserver.default.hello_argument pref, but that is not something a user should have to do to prevent leaks like this.

Please set the default to be the pref first, then 127.0.0.1.

Comment 22

[Magnus Melin [:mkmelin]]

I don't think there's anything new with setting the hostname there: that is per the spec, and if you don't to do that there's going to be servers that block you.

Comment 23

[alta88]

Clearly, due to the Cc string typo, localhost is always returned, which this particular server rejects. Clearly, per comment 15 and 16, any valid ip format string succeeds, as likely would any other string, so it is unnecessary to leak 'fingerprint' information about actual internal names/ips, whether new or not. What does Outlook send for HELO hostname - anything a user chooses to call their machine.

At the end of the day, despite Bug 68877 and Bug 279525, it isn't going to matter. And the spec here says it must not: https://tools.ietf.org/html/rfc5321#section-4.1.4

Comment 24

[Matt]

Originating the email using the local IP address is rejected by other ISP's who use the spamhaus blocklists to block their own users from submitting mail.

Comment 25

[José M. Muñoz]

Looks like at least some of bug 1701864, bug 1702840, bug 1701268, bug 1701996 and bug 1702838 are caused by the EHLO issue.

Comment 26

[Magnus Melin [:mkmelin]]

(In reply to alta88 from comment #23)

At the end of the day, despite Bug 68877 and Bug 279525, it isn't going to matter. And the spec here says it must not: https://tools.ietf.org/html/rfc5321#section-4.1.4

Section 2.3.5:

The domain name given in the EHLO command MUST be either a primary
host name (a domain name that resolves to an address RR) or, if
the host has no name, an address literal, as described in
Section 4.1.3 and discussed further in the EHLO discussion of
Section 4.1.4.

So using the address literal sound right to me.

Comment 27

[Ping Chen [:rnons] (away)]

Attachment: 1701804-dns.patch

Thanks alt88 for noticing the contract id bug.

About the hostname in EHLO, I think we are following the spec. There is bug 1680197 if you want to continue the discussion. Thanks.

Comment 28

[alta88]

No, you're not following the spec. Almost always, now that the Cc string is fixed, the returned value will be some machine name, which is certainly not a FQDN. And the spec is non followable for all network configurations, which is why 4.1.4 states a server MUST not reject for that reason. A well configured server will accept and respond with a message stating the address literal is not a FQDN. And it knows the external ip anyway, so whatever the client sends is useless, despite the spec. Too clever by half servers will fail on 'localhost' or 127.0.0.1 for spam purposes, as if that would be sufficient.

Comment 29

[Ping Chen [:rnons] (away)]

Attachment: 1701804-dns.patch

Changed to never send hostname. Because rfc5321#section-2.3.5 says

A domain name that is not in FQDN form is no more than a local alias.
Local aliases MUST NOT appear in any SMTP transaction.

Comment 30

[José M. Muñoz]

So that also addresses bug 1680197?

Comment 31

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/c876b1a065f4
Do not send hostname in EHLO command in SmtpClient.jsm. r=mkmelin

Comment 33

[Ping Chen [:rnons] (away)]

Comment on attachment 9212966 [details]
Bug 1701804 - Send ip address in EHLO command in SmtpClient.jsm. r=mkmelin

[Approval Request Comment]
Regression caused by (bug #): bug 1661694
User impact if declined: For a few SMTP servers, sending always fails.
Testing completed (on c-c, etc.): c-c
Risk to taking this patch (and alternatives if risky): low

Comment 34

[Ping Chen [:rnons] (away)]

Comment on attachment 9213515 [details] [diff] [review]
1701804-dns.patch

[Approval Request Comment]
Regression caused by (bug #): bug 1661694
User impact if declined: No impact, the removed code didn't work correctly
Testing completed (on c-c, etc.): c-c
Risk to taking this patch (and alternatives if risky): low

Comment 35

[Wayne Mery (:wsmwk)]

Comment on attachment 9213515 [details] [diff] [review]
1701804-dns.patch

[Triage Comment]
Approved for beta

Comment 36

[Wayne Mery (:wsmwk)]

Comment on attachment 9212966 [details]
Bug 1701804 - Send ip address in EHLO command in SmtpClient.jsm. r=mkmelin

[Triage Comment]
Approved for beta

Comment 37

[Rob Lemley [:rjl]]

Thunderbird 88.0b3:
https://hg.mozilla.org/releases/comm-beta/rev/b72c08af2715
https://hg.mozilla.org/releases/comm-beta/rev/3ee6fe2e30da