Closed Bug 1789958 Opened 2 years ago Closed 2 years ago

Duplicate emails coming in (Zoho, POP3)

Categories

(MailNews Core :: Networking: POP, defect)

Product:
MailNews Core
Component:
Networking: POP
Version:
Thunderbird 102
Platform:
Desktop
Windows 8.1
Type:
defect

Tracking

(thunderbird_esr102 fixed, thunderbird105 affected)

Status:
RESOLVED FIXED
Milestone:
106 Branch
Tracking Flags:
Tracking Status
thunderbird_esr102 --- fixed
thunderbird105 --- affected

People

(Reporter: mail, Assigned: rnons)

References

(Regression)

Details

(Keywords: regression)

Attachments

(2 files)

TB errors.txt
28.89 KB, text/plain
Details
Bug 1789958 - Catch OpenFilterList error in MsgIncomingServer to prevent duplicate pop3 messages. r=mkmelin
48 bytes, text/x-phabricator-request
wsmwk
: approval-comm-esr102+
Details | Review

Steps to reproduce:

Just have TB open

Actual results:

I've been getting tons of duplicate emails coming into one account's inbox all day.

I am using Zoho & I logged into Zoho & they originally came in early this morning & around 1:30 pm (last one).

It's now 9:21 pm & they won't stop coming in.

The same emails over & over (3 of them).

Help.

Thanks

Expected results:

Just one email from all 3 senders.

OS: Unspecified → Windows 8.1
Hardware: Unspecified → Desktop

Sorry, I meant to say that in Zoho mail, only one of each came in so I know it's not Zoho.

I've been getting tons of duplicate emails coming into one account's inbox all day.

What changed in your environment

I am using Zoho & I logged into Zoho & they originally came in early this morning & around 1:30 pm (last one).
Sorry, I meant to say that in Zoho mail, only one of each came in so I know it's not Zoho.

"Zoho mail" means the web interface?

How is the account defined in Thunderbird - pop or imap?

Also, what is your exact version number and

Flags: needinfo?(mail)

What changed - I updated TB to 102.2.1

Zoho mail - not, they are a mail server plus they also have their own e-mail client, but when I went to check if I was also getting duplicates there, I wasn't.

POP

Thanks

Flags: needinfo?(mail)

Thanks for reporting, can you get some logs by following bug 1783290 comment 3

Also, check the menu View > Headers > All. Then select the duplicated email, you should see all headers above the message content. There is a UIDL line, tell me the value.

X-UIDL xxxxxxxx

Hi there,

Ok, did you want all 3, or is one good enough?

Here's one & I did check several of the same email & the number is all the same. 1662657095587110001

Why is their a line through the bug comment you want me to give you logs on?

Are the logs going to show private information to everyone?

Thanks a ton

Ok, did you want all 3, or is one good enough?

One is good enough.

Each message has a unique UIDL, duplicate mails have the same UIDL is expected. After fetching a message, TB saves the UIDL state in <ProfileDirectory>/Mail/<pop3-hostname>/popstate.dat. Before another fetch, TB checks popstate.dat to only download new messages.
You can find your ProfileDirectory from Help -> More Troubleshooting Information, then search for Profiles. Let me know if you see 1662657095587110001 in the popstate.dat.

Without logs and popstate.dat, it's hard to know why TB downloads old messages again. The logs may contain your server host, your user name, but you can remove them before uploading.

Component: Untriaged → Networking: POP
Product: Thunderbird → MailNews Core
Summary: Duplicate emails coming in → Duplicate emails coming in (Zoho, POP3)

(In reply to Ping Chen (:rnons) from comment #6)

Ok, did you want all 3, or is one good enough?

One is good enough.

Each message has a unique UIDL, duplicate mails have the same UIDL is expected. After fetching a message, TB saves the UIDL state in <ProfileDirectory>/Mail/<pop3-hostname>/popstate.dat. Before another fetch, TB checks popstate.dat to only download new messages.
You can find your ProfileDirectory from Help -> More Troubleshooting Information, then search for Profiles. Let me know if you see 1662657095587110001 in the popstate.dat.

Without logs and popstate.dat, it's hard to know why TB downloads old messages again. The logs may contain your server host, your user name, but you can remove them before uploading.

Ok a few things:

  1. We tried to go to where you said, but there's 4 different profiles (2 different sets). Which one do we search in?
  2. Then we found one that had Mail & there's 10 different accounts. Which one do we go to?
  3. Logs - I just copied the log for the email that's corrupted & there's a ton of private info in there. Please give me your private uploading area or email so only you can see it, because I'm not giving all of this private info to the entire world.

Thanks

Sorry, forgot to do the SSs before I pressed save.

https://i.imgur.com/zrq9fG9.png

https://i.imgur.com/8q614GR.png

Attached file TB errors.txt —

Ok, I removed everything that needs to be hidden.

Here is the error log.

Ok, I just checked my mail (I hadn't checked it since Friday) & now TB is doing this to every NEW email coming in it looks like.

I had to change the UN in the settings just to get it to stop doing this.

It's just NUTS!

Please fix!

Thanks, very helpful. The problem seems to be
Pop3Client.jsm:309:18: Component returned failure code: 0x80520016 [nsIMsgFilterService.OpenFilterList] MsgIncomingServer.jsm:874

Working on a fix.

You can set mailnews.pop3.jsmodule to false in Config Editor and restart TB, to use the old pop3 module, which should not have this problem.

Assignee: nobody → remotenonsense
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

(In reply to Ping Chen (:rnons) from comment #12)

Thanks, very helpful. The problem seems to be
Pop3Client.jsm:309:18: Component returned failure code: 0x80520016 [nsIMsgFilterService.OpenFilterList] MsgIncomingServer.jsm:874

Working on a fix.

You can set mailnews.pop3.jsmodule to false in Config Editor and restart TB, to use the old pop3 module, which should not have this problem.

Ok, thanks, done.

So a couple of questions:

  1. How come it only happened with one of my zoho accounts?
  2. What update version will the fix be in? I still have to update to the last update.

I hope you guys get the bugs fixed. I had other issues after the update which took a while to get resolved, & so did tons of people which were mentioned in http://forums.mozillazine.org/

Thank you

Keywords: checkin-needed-tb
Target Milestone: --- → 106 Branch

Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/a48515cd0db9
Catch OpenFilterList error in MsgIncomingServer to prevent duplicate pop3 messages. r=mkmelin

Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Keywords: checkin-needed-tb
Resolution: --- → FIXED

Comment on attachment 9294357 [details]
Bug 1789958 - Catch OpenFilterList error in MsgIncomingServer to prevent duplicate pop3 messages. r=mkmelin

[Approval Request Comment]
Regression caused by (bug #): pop3-js
User impact if declined: Duplicate messages are downloaded.
Testing completed (on c-c, etc.): c-c
Risk to taking this patch (and alternatives if risky): No risk.

I think it's safe to skip beta, the patch didn't change any functionality, only added two try..catch, so that popstate.dat can be correctly updated. This may also fix bug 1777387.

Attachment #9294357 - Flags: approval-comm-esr102?

Comment on attachment 9294357 [details]
Bug 1789958 - Catch OpenFilterList error in MsgIncomingServer to prevent duplicate pop3 messages. r=mkmelin

[Triage Comment]
Approved for esr102

Attachment #9294357 - Flags: approval-comm-esr102? → approval-comm-esr102+

Can someone answer my last set of questions please.

Also, this may be similar to this bug (I don't know), but yesterday one of my accounts said my p/w wasn't correct when it was.

I had to copy the same p/w again to both outgoing & incoming before TB would recognize it.

Then today with ALL of my gmail accounts I got an error saying it couldn't connect, so unless gmail was down about 4-5 hours ago, that's another issue & I just checked, it's working now, but I did have other connection issues even before I created this bug report. I had to literally enter in the p/w over & over again before TB would accept it. That wasn't on this version though. I don't remember what version it was.

Thanks

It will be in 102.3.0, out early next week.
The problem had to do with opening the filter rules file for that account (file corrupt/missing for unknown reason). Probably the file was fine for the other accounts so it only happened for this one.

(In reply to Magnus Melin [:mkmelin] from comment #20)

It will be in 102.3.0, out early next week.
The problem had to do with opening the filter rules file for that account (file corrupt/missing for unknown reason). Probably the file was fine for the other accounts so it only happened for this one.

Thanks for getting back to me Melin.

Well we did look at the message filter for that account & it looked fine, but you are saying I didn't do anything to corrupt it, right?

And what about the other connection issues?

I wouldn't think you had some anything to corrupt it no.
No idea about the connection issues, but it would be another bug. If it did say cannot connect, maybe networking issues, or e.g. firewall/antivirus blocking?

(In reply to Magnus Melin [:mkmelin] from comment #22)

I wouldn't think you had some anything to corrupt it no.
No idea about the connection issues, but it would be another bug. If it did say cannot connect, maybe networking issues, or e.g. firewall/antivirus blocking?

No, nothing has changed on my computer today other than I updated the drivers today & added more memory, but the other non gmail account issue happened yesterday.

Ok, this is definitely a bug. I just launched TB again & again I got all of these errors for all of my gmail accounts. https://i.imgur.com/ybZ2leg.png

Michelle, you have good results with 102.3.0?

Flags: needinfo?(mail)

That specific issue seems to be ok although I did get duplicates for 3 emails (different account) for stat counter today. I think that's the first time it's ever happened although I can't remember & it only happened with 3 of the 8 or so domains.

Thanks

Flags: needinfo?(mail)

Hi,

This bug is happening again & I just blamed zoho again forgetting this was happening a year ago. They are blocking my mail. Please fix it again. I'm still on version 102. I don't want to switch the new new version.

Why did this bug start again?

I assume it was b/c of your update in September.

Please help.

Thanks

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: