1789314 - POP3 stopped working (getting new messages) if server does not support UIDL

Status: VERIFIED FIXED

(MailNews Core :: Networking: POP, defect)

Description

[Michael Schierl]

Attachment: Captured TCP stream

Steps to reproduce:

I am using a local POP proxy (written myself a decade ago) that collects information from forums and social media and provides it as a fake POP3 account. It only supports the bare minimum of the POP3 protocol (USER, PASS, STAT, LIST, RETR, DELE, QUIT) and worked fine with email clients so far. After updating from Firefox 9x to 102, it stopped working. Note that "Leave messages on the server" is NOT checked.

Actual results:

POP3 MOZ_LOGs appear empty, but Wireshark shows that the communication stops after the client calls UIDL command and receives an error code. The connection stays open and email is not retrieved.

Expected results:

The mail client should detect that the server does not support UIDL and instead just retrieve the emails and delete them.

Comment 1

[Magnus Melin [:mkmelin]]

Must be super rare...

You can for the moment still set mailnews.pop3.jsmodule false to get the old implementation.

Comment 2

[Michael Schierl]

Thank you, setting the option (as expected) worked around the issue.

It may be rare, but still I think that the implementation should not suddenly hang (and leave the socket open) without showing an error message if the server acts weirdly in any way. (And in this case, it is compliant behaviour as UIDL is an optional POP3 command).

OTOH, on Reddit there have been some threads about some POP3 proxys stop working with the Thunderbird 102. As nobody attached/posted a trace, it is not 100% certain it is the same issue, but it may be.

But you are obviously right, most people will concentrate on a very small amount of POP3 server implementations (Gmail, Office365, Hotmail, Yahoo, ...) and even more people will not use POP3 but IMAP, so the issue probably affects only a tiny subset of the users.

Comment 3

[Ping Chen [:rnons] (away)]

I will try to fix this. I have a question about the old behavior. If "Leave messages on the server" is checked, and you delete a message from the message list, will this message be deleted on server?

I ask because for servers with UIDL, delete a message from the message list doesn't send DELE immediately, but only change a UIDL flag. DELE is sent in the next connection. I'm not sure how it works for servers without UIDL.

Comment 4

[Ping Chen [:rnons] (away)]

I found the answer, an error message is shown if "Leave messages on the server" or some other features are turned on but the server doesn't support UIDL https://searchfox.org/comm-esr102/rev/d9800535d0b488462c8ae64f7fe6647cc42535e1/mailnews/local/src/nsPop3Protocol.cpp#2516

Comment 5

[Magnus Melin [:mkmelin]]

Michael, is the server you wrote available somewhere to test this live? (Or installable?)

Comment 6

[Ping Chen [:rnons] (away)]

Attachment: Bug 1789314 - Support server without UIDL capability in Pop3Client. r=mkmelin

Comment 7

[Michael Schierl]

Attachment: sample server

The server is not on the internet (it provides no authentication and allows every password), and I cannot really share it as is as it has hard-coded API credentials for social media APIs.

However, I have attached a stripped down version. The POP3 code is unchanged, just the message generation part was replaced by a simple loop that creates one message for every minute between last mail fetch and now.

It is written in Java, tested with Java 8, 11 LTS and 17 LTS. You can compile with Maven or use the included binary jar if you trust it.

Run via the included script and point your Thunderbird to POP3 server localhost:20110 (any username and password).

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/ee629f3bad31
Support server without UIDL capability in Pop3Client. r=mkmelin

Comment 9

[Ping Chen [:rnons] (away)]

Comment on attachment 9293428 [details]
Bug 1789314 - Support server without UIDL capability in Pop3Client. r=mkmelin

[Approval Request Comment]
Regression caused by (bug #): bug pop3-js
User impact if declined: POP3 server without UIDL capability doesn't work
Testing completed (on c-c, etc.): c-c
Risk to taking this patch (and alternatives if risky): low

Comment 10

[Wayne Mery (:wsmwk)]

Comment on attachment 9293428 [details]
Bug 1789314 - Support server without UIDL capability in Pop3Client. r=mkmelin

[Triage Comment]
Approved for beta

Comment 11

[Daniel Darnell [:dandarnell]]

Thunderbird 105.0b4:
https://hg.mozilla.org/releases/comm-beta/rev/214a506fa835

Comment 12

[Wayne Mery (:wsmwk)]

Comment on attachment 9293428 [details]
Bug 1789314 - Support server without UIDL capability in Pop3Client. r=mkmelin

[Triage Comment]
Approved for esr102, because This may be less rare than we think.

If you disagree with taking this in 102.3.0 to ship next week, please reset the flag.

Comment 13

[Ping Chen [:rnons] (away)]

Attachment: 1789314-esr102.patch

Rebased to esr102, thanks.

Comment 14

[Wayne Mery (:wsmwk)]

Comment on attachment 9294579 [details] [diff] [review]
1789314-esr102.patch

[Triage Comment]
approved for esr102

Comment 15

[Rob Lemley [:rjl]]

Thunderbird 102.3.0:
https://hg.mozilla.org/releases/comm-esr102/rev/5a36d47f84d1

Comment 16

[Rob Lemley [:rjl]]

Attachment: 1789314-esr102-followup.patch

[Triage Comment]
Fixes call to pop3Daemon(). Verified working on local build.

Comment 17

[Rob Lemley [:rjl]]

Thunderbird 102.3.0:
https://hg.mozilla.org/releases/comm-esr102/rev/bbafef6ed407

Comment 18

[Michael Schierl]

Can confirm it works again without workaround in Thunderbird 102.3.0.

Comment 19

[Michael Schierl]

I just noticed a small glitch. When there are many new messages, the "receiving message x of y" status bar messages alway show y as 0 (i.e. "receiving message 147 of 0"). But I can live with that :)