1865651 - TAGs don't sync the same with version TB115 (UI not immediately updated when a message tag changed on the server; changes only after message select or folder refresh)

Status: RESOLVED FIXED

(Thunderbird :: Mail Window Front End, defect, P2)

Description

[B4net]

Steps to reproduce:

We are a group of 9 people, organizing us with TAGs inside Thunderbird (using same IMAP Mail).
Before update to 115.4.3 version from 102 (I think) the tags were working flawlessly for more than 10 years. Now the syncing of tags somehow changed and that's not good or there is maybe a bug?

Actual results:

If with new version 115 someone change the tag on received e-mail, on other computers using the same IMAP Mail, the tag stays the same, until you click on that e-mail or until a new e-mail arrives. If nothing from that two things happens, than the tags are not synced (I waited for more than 20 minutes and the syncing was not performed).
I also tested and figured out that the tags are "manually" synced also if I click to filter(QuickFilter) the inbox/sent messages with Tags and also if I change from inbox to sent and back to inbox.
That non-automatic syncing is very confusing because all the time you are not sure if you are already tagged or not.

Expected results:

Before if someone tagged an e-mail message, the tag was synced very fast in next seconds, nevertheless at which e-mail were you standing on. That was perfect!
I don't know in which way the TAGs were syncing before but at least click on "Get Messages" should sync the tagging too. In that way you can see faster on latest incoming e-mails if you are already tagged or not.

Comment 1

[gene smith]

I haven't tried to duplicate your issue yet, but it might be helpful to know a few details:

What type of imap server are you using? (ie., dovecot, cyrus, gmail, exchange etc.)
Are the tags user defined "keywords" or are you just using the built-in tags that TB provides?
How are your imap server settings set in TB: Are you using "immediate" server notification; checking for new messages every X minutes?
I think you are saying all 9 users are on the same imap account. So when user A sets a tag on a message, user B is not seeing that tag on that same message?
Are all the messages that your users tag in Inbox?

This may be fixed by the changes in bug 1854586 and are now in beta 120.0b6 or later (not yet in an the ESR 115). A problem occurs if the network is down at the time of TB startup so that connection to the server may not occur until you click "Get Messages" or, as you mention, switch to another folder and then back to Inbox. I'm not sure why the "quick filter" would also help, but it might. But once the connection is established, it should work OK with the tags after that.
If you can try the latest TB beta your issue might be fixed with it (but don't know about you network status before TB startup, so it might not help).

Anyhow, I'll test the tags myself ASAP.

Comment 2

[gene smith]

Reporter B4net, probably no reason to try the beta since I can duplicate this bug now and the beta won't fix the problem.

I have two tb instances looking at the same imap account. I set a built-in tag like "Important" on a message in Inbox of instance A. The message appears in red font. Then I look at instance B at the same message in Inbox and there is no change. I see at least 3 ways to then get the tag to appear correctly on instance B.

1. On instance A, also set the "Star" flag. This causes both the Star and the message at instance B to become red.
2. Or on instance B, select the account and then select back to Inbox. This causes the message to become red.
3. Or in instance B, select a non-Inbox folder and then move back to Inbox. This also causes the message to become red.

Since the "Star" is actually just another imap flag very similar to the tags and appears in the same type of FLAGS idle response, it's interesting that setting Star on a message in either TB instance always sets the Star on the same message of the other instance. But just setting a tag doesn't.

Looking with IMAP:4 log, I don't see anything wrong with the network response that would prevent an instance from knowing to set the message to the appropriate color. Specifically, when instance A sets the "important" tag I see this idle response on instance B:
* 54 FETCH (FLAGS (\Seen nonjunk $label1)) where $label1 is the symbol for the "important" tag. This idle response never sets the message red. But if the response looks like this * 54 FETCH (FLAGS (\Flagged \Seen nonjunk $label1)), after then also setting the Star (Its symbol is \Flagged) on instance A, both the Star and message on instance B become red.

So, this appears to be a display problem and not a problem at the imap protocol level.
But I need to look deeper to be sure.

Comment 3

[B4net]

@Gene smith
IMAP type: I don't know (using outsourced E-mail server with cpanel)
Tags are default ones + a few personalized (total of 12 different)
IMAP settings -> checking for new messages every X minutes
All messages are inside the same folder (inbox)

Another way to wake up tag syncing is if you click to filter(QuickFilter) the inbox/sent messages with Tags.

Comment 4

[B4net]

Attachment: An option to force syncing tags.

An option to force syncing tags. You have to click to filter tagged messages and than unset the filter and immediately all the tagged messages are synced.

Comment 5

[gene smith]

(In reply to B4net from comment #4)

Created attachment 9364966 [details]
An option to force syncing tags.

An option to force syncing tags. You have to click to filter tagged messages and than unset the filter and immediately all the tagged messages are synced.

Yes, I forgot that you originally reported that work-around. And you are correct, it does show the tagged message with the appropriate color. Interesting too that selecting the "tag" filter button doesn't trigger any imap activity but it fixes the display. So this pretty much proves that the problem is in the display/UI somewhere and not an imap command or parsing issue.
Edit: Another work-around is to change the sort order on any column. E.g, click the Date column twice to cause the tag color to appear in the view.

Comment 6

[B4net]

Hi,
as I posted first bug, i don't have experiences about solutions. When can we expect an repair for this? Will I be somehow informed?
Thank you!

Comment 7

[gene smith]

(In reply to B4net from comment #6)

Hi,
as I posted first bug, i don't have experiences about solutions. When can we expect an repair for this? Will I be somehow informed?
Thank you!

You will get emails about this bug to inform you of activities and when it's fixed.

Seems like a fairly basic UI bug. So UI developer(s) need to be informed of this and not sure they have. I'll ping their leader (aleca).
Made a guess on priority/severity settings.

Comment 8

[B4net]

Is here anything new with that bug?
Not refreshing tags even with "Get Messages" is very problematic and that really makes Thunderbird unreliable for group of more users using one IMAP E-mail account.
Could dedicated developer please resolve that as soon as possible?
Thank you!

Comment 10

[Wayne Mery (:wsmwk)]

B4net,
Still seeing this in version 128?

Comment 11

[B4net]

Hi Wayne Mery,

yes the problem still here on version 128.5.2esr (64-bit).
In the last versions I discovered that mouse scrolling (a few scrolls up and back) also re-sync the tags. But that's not safe enough. The Tags should be synced or somehow the display of messages refreshed at least when clicked "Get Messages". Although before that problem the tags were somehow synced automatically in a few seconds.
I please the developers to make that bug more important and repair it. It seems not problematic, but in fact using the tags on IMAP server is not reliable and usable any more,... for more than a year.
Thank you all in advance!

Comment 12

[Wayne Mery (:wsmwk)]

vlad, can you reproduce?

Comment 13

[Ramona Gavrilescu]

Hello Wayne,

I have managed to reproduce the scenario above.
I have tested on 3 laptops using the same IMAP account on all three.
On TB 102.0(20220627231524) if the tag is removed or changed on an email the changes are visible on the other 2 laptops in seconds without any action

On TB 128.6.0esr(20250107005646), 128.2.3esr(20240920043836), 135.0b1( 20250110170549) same IMAP account on all 3 laptops the changes are visible only if you go to another folder and come back and click on tags option, or if open the same exact email on which the tags were removed/changed on a different laptop.
Scrolling up/down, check/ uncheck the Starred option or Get Messages does not show the changes.

Comment 14

[Wayne Mery (:wsmwk)]

Thanks for testing

Comment 15

[ericj]

I am another Thunderbird user whom is using tags in the same manner and am also impacted by this bug.

Comment 16

[gene smith]

I ran into this bug again while looking at another issue. I tested with 155a1 (self-built daily) and still see the problem. So far still unable to determine why tags don't update when they are changed by another tb instance. I'm still almost 100% sure problem is in front-end UI code but can't find where tags get updated. Interesting that all the other items on the thread row get updated automatically such as the star, read/unread, junk etc when changed from another tb client but tags don't.
Note: as already reported above, tags do manually update if

1. Move off folder and re-select folder
2. Toggle "Tags" button in quick filter bar.
3. Scroll message with expected tag off of screen and then back into view. But doesn't always work.

Comment 17

[B4net]

We are a group of people who deal with this issue daily. Tags have lost their core functionality for us: if your colleague in the same inbox can't see that they've been tagged, they end up tagging themselves—which actually untags them, since they were already tagged from another person. It’s a huge problem for us.
Does anyone have the authority to increase the priority of this bug so it finally gets fixed after two years?
Thank you all

Comment 18

[B4net]

Currently using 140.6.0esr release.

Comment 19

[Toby Pilling [:tobyp]]

Attachment: Bug 1865651 - Notify listeners for remote IMAP keyword updates. r=#thunderbird-reviewers

Comment 20

[Toby Pilling [:tobyp]]

I also noticed that if I added a tag in client A and then added a star, both changes were visible in client B. Hope this patch helps.

Comment 21

[Francesco (no longer active, use PM for contact)]

Clearing "Target Milestone" since this isn't ready to land yet.

Comment 22

[gene smith]

I manually applied Toby's comment 19 patch and now works correctly again.
I'm marking this as regressed by bug 1842384 which made the bug that appeared in 115.

Comment 23

[gene smith]

(In reply to Toby Pilling [:tobyp] from comment #20)

I also noticed that if I added a tag in client A and then added a star, both changes were visible in client B. Hope this patch helps.

Without your patch the "star" had no problems appearing in client B when changed on client A. You could also change other things like read/not-read on the message line and it would follow in remote client. Only tag didn't follow without your patch.

Edit: I thought you were referring to behavior with the patch in place. But I think you mean without the patch. So, yes, without the patch when you set a tag with client A and then set the star, both star and tag will auto-update on client B. But if you set the star first and then the tag, only star auto-updates on client B.
Of course, with the patch the order doesn't matter and both auto-update on client B.

Comment 24

[Wayne Mery (:wsmwk)]

(In reply to Francesco from comment #21)

Clearing "Target Milestone" since this isn't ready to land yet.

Francesco, this is in the patch author's wheelhouse, not yours.

Comment 25

[Francesco (no longer active, use PM for contact)]

Wayne, please ensure that patch authors DO SET THE TARGET MILESTONE ALWAYS if you don't want others to interfere. As per my PM, I've fixed it about 60 times, and here are two more that need fixing: https://mzl.la/4wsWGjw. You have the choice: Either your FTEs do the work or you stop criticizing volunteers who do the cleanup.

Comment 26

[Wayne Mery (:wsmwk)]

Either your FTEs do the work or you stop criticizing volunteers who do the cleanup.

Regardless, this isn't in your wheelhouse.

Comment 27

[gene smith]

One other note on this feature (changing message properties on client A and seeing it automatically on Client B):
This depends on imap IDLE being supported on the server and enabled on Client B (Server setting "Allow immediate server notifications when new messages arrive" being selected/checked).
If IDLE is not supported on server or is not enabled on Client B, there is no automatic update of star, tags, read/unread or junk. Without IDLE, to update message properties in Inbox you have to wait for next timed message fetch (biff) or manually do "Get Messages". For message not in Inbox you have to select another folder and come back to see an update.

Comment 28

[Toby Pilling [:tobyp]]

(In reply to Francesco from comment #21)

Clearing "Target Milestone" since this isn't ready to land yet.

LOL ok I'll leave it to you to set back then. I'm not great at remembering, so I set which milestone I want it to land in.

Comment 29

[gene smith]

(In reply to B4net from comment #18)

Currently using 140.6.0esr release.

I don't know if this fix can be back-ported to 140esr since there is now a 153esr. But until the fix is released I think Toby suggests a reasonable work-around. See comment 20 and edited comment 23.

So if the user sets the tag first and then sets the star, the tag will auto-update on the other client. The user at the other client can reset the star if they want to which will remove it at all clients.

Comment 30

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/6715d2f83dcf
Notify listeners for remote IMAP keyword updates. r=gds,edicharry

Comment 31

[B4net]

(In reply to gene smith from comment #27)

One other note on this feature (changing message properties on client A and seeing it automatically on Client B):
This depends on imap IDLE being supported on the server and enabled on Client B (Server setting "Allow immediate server notifications when new messages arrive" being selected/checked).
If IDLE is not supported on server or is not enabled on Client B, there is no automatic update of star, tags, read/unread or junk. Without IDLE, to update message properties in Inbox you have to wait for next timed message fetch (biff) or manually do "Get Messages". For message not in Inbox you have to select another folder and come back to see an update.

Hi,
quick reply to your message: all other updates like stars and read/unread work fine—it's only the tags causing issues. Clicking 'Get Messages' doesn't update the tags either.

Comment 32

[Toby Pilling [:tobyp]]

Comment on attachment 9620014 [details]
Bug 1865651 - Notify listeners for remote IMAP keyword updates. r=#thunderbird-reviewers

Uplift Approval Request

• Please state case for uplift consideration and ensure bug severity is set: Longstanding bug present since 115 which impacts organizations, more likely to be using ESR
• User impact if declined: Some orgs can't use TB in their workflows due to this broken feature. See bug comments.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Daily?: Yes
• Has the fix been verified in Beta?: No
• Needs manual test from QA?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Straightforward correction to replace operation directly made on header with similar operation that fires database notification.
• Does the fix cause any migrations to be skipped?: No
• String changes made/needed:

Comment 33

[Corey Bryant]

Comment on attachment 9620014 [details]
Bug 1865651 - Notify listeners for remote IMAP keyword updates. r=#thunderbird-reviewers

[Triage Comment]
Approved for esr153

Old issue but simple fix

Comment 34

[Evandro Baginski [:ebaginski]]

Thunderbird ESR153
https://hg.mozilla.org/releases/comm-esr153/rev/76bfbd3dcf91