1902930 - Thunderbird skips new Proton emails when running filters if opened on another platform

Status: RESOLVED WONTFIX

(Thunderbird :: Filters, defect)

Description

[David McDivitt]

Using version 128.0b1 (64-bit)

I recently transitioned to Proton email for the security it affords. Proton provides IMAP only at this time, planning to have POP3 soon. They provide something called Proton Bridge which runs on the Windows PC, has two-factor authentication, and provides an IMAP server at 127.0.0.0 on the local machine.

For the most part this works well except rules to run when mail is received. They have an android app to view mail and mail can be viewed on the web. New email will list on the android app and web with no problem but any emails opened with the android app or web will be skipped in Thunderbird on my PC when running rules. If I open a message then mark unread, Thunderbird still skips the email.

Thunderbird should recognize all emails newly appearing in the inbox folder regardless of status set by other applications.

Gmail uses IMAP and does not exhibit this problem. I sent emails to a Gmail account, opened them on the web, then opened Thunderbird and read the Gmail mailbox. All emails were recognized whether opened elsewhere and rules ran on those messages.

Comment 1

[David McDivitt]

The Proton Email people did testing to reproduce this issue and provided feedback resolving this issue.

For bug 1895549 I was told to put my mail processing rule in two steps with the first step copying to the destination folder and the second step marking read in the source folder. If both are done in the same step the email is marked read before being copied regardless of the order given in the step and email shows read in the destination folder, too.

Unfortunately, using two steps prevents Thunderbird from processing the rule if the message has been opened elsewhere on a different platform, before being pulled and processed in Thunderbird.

Putting the copy plus mark read back into the same step, to try and resolve this issue, of not processing email if already opened, is something I never would have thought of doing. I made one rule with one step and bug 1902930 (this one) went away. I also backed up my Thunderbird folders, did some test emails, and restored. Thunderbird pulled and processed all the emails a second time, which is good.

Probably bug 1902930 (this one) should remain a bug and bug 1895549 not worried about at this time.

Comment 2

[Wayne Mery (:wsmwk)]

Is there any reason to believe this only started in version 128?

Comment 3

[gene smith]

Just want to make sure, but when you refer to "rules" you mean the filter rules you setup in TB and not "server side" rules/filter setup on the proton bridge server, right?

Does the bridge support imap IDLE (*)? If so, and you have server setting "Allow immediate server notification when new messages arrive" set, this should cause any message arriving in the Bridge inbox to be immediately filtered (copied) by TB to the destination folder before any other client or platform can affect the new message. Of course, this assumes that TB is always running.

If proton bridge server does NOT support IDLE then you have to rely on polling based on the server setting "Check for new messages every X minutes". What is your setting for "X"? Making it smaller (maybe 1 minute) will make it less likely that you the other clients will affect the message before the TB filter runs. Again, his assumes TB is continuously running.

Anyhow, my guess (since I don't have a proton bridge server) is that maybe the server is resetting or never setting a messages flag (maybe \RECENT) when the message is accessed by another client before TB polls for new messages (not using IDLE) and this is causing TB to not see the message as new so it doesn't run the filter on it.

If allowing TB to use IDLE (set the "Allow immediate ..." server setting) or setting the poll time for new messages lower ("X" value mentioned above) doesn't help, we might be able to tell more if you attach an IMAP:4 log using the info shown here: https://wiki.mozilla.org/MailNews:Logging#Windows. Maybe do this:

1. Shutdown TB
2. Using the proton app or webmail, wait for messages to come in that TB should filter and open it.
3. Start TB using the batch files described in the above wiki link.
4. Verify in TB that the message has arrived but never got filtered.
5. Shutdown TB
6. Attach the resulting log file using the "Attach new file" button above.

*) If you don't know if IDLE supported by bridge, we can tell by looking at the resulting IMAP:4 log under the imap CAPABILITIES response. Or you can ask Proton support.

Comment 4

[David McDivitt]

Gene, this has nothing to do with the Proton server. It's good to assess all sides of an issue and for an IT environment where info comes from. It's just weird functioning of Thunderbird rules.

If in my rules I say to copy to another folder one would think it would copy, then any changes made to the original would not affect the copy because there would then be two messages, But attributes are shared throughout the duration of the rule. Copying, then changing the original, affects the copy, also. I remember this much from when I was playing with this trying to get my new set up to work. For some reason the which I don't remember I was trying to put everything in one rule: mark unread, copy, mark read, move to trash. The read attribute would affect the copied email so this wouldn't work.

The Proton people advised me to make a separate rule for each action. I did that and had sudden happiness.

I removed the rule to mark unread initially. That allows me to read emails through their application on my phone, and allow the read status to flow through to final destination in Thunderbird.

The Proton Bridge is an excellent product and the only thing like it I've seen. It does two-factor authentication when setting up but only for set up. After that Thunderbird works with it just as any other server. Emails are encrypted, too, so that not even Proton employees can read them. They cannot restore access to an account, either. I had been pulling email from a web hosting company for many years via POP3 but continued worrying over how unsecure that is finally prompted me to try Proton.

Unless someone wants to address how rules are processed this bug should be closed.

Comment 5

[gene smith]

Thanks for the info. But since you said this:

Gmail uses IMAP and does not exhibit this problem. I sent emails to a Gmail account, opened them on the web, then opened Thunderbird and read the Gmail mailbox. All emails were recognized whether opened elsewhere and rules ran on those messages.

I thought you meant that bridge was behaving differently than gmail. I.e., the same TB filter(s) worked with gmail and didn't work with bridge after accessing the message with another client.
Anyhow, if the filtering is working OK now and you're happy, closing as WONTFIX is fine with me!

Comment 6

[David McDivitt]

Yes, Gene, now that I think about it and remember there was a difference between Gmail and Proton. I don't know what functionality the Proton Bridge provides and whether it's just a pass through to the server. I know it has caching.