Closed Bug 440951 Opened 16 years ago Closed 14 years ago

Unable to remove predefined tags from messages in Gmail IMAP folders (Old Mozilla/Tb stored $LabelX instead of $labelX. Gmail IMAP is case sensitive upon keyword remove)

Categories

(Thunderbird :: Mail Window Front End, defect)

Product:
Thunderbird
Component:
Mail Window Front End
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: flat, Unassigned)

References

(Blocks 1 open bug)

Details

(Whiteboard: workaround: see comment #37) 

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14
Build Identifier: Thunderbird version 2.0.0.14 (20080421)

I have configured my thunderbird to get Gmail using IMAP technology. I tag the emails to mark them depending on priority but then I realized that once they are tagged they cannot be un tagged. The tags work ok on a POP folder though. 

Reproducible: Always

Steps to Reproduce:
1.
2.
3.



The only thing that may have uncommon is that I dragged all my POP messages, some of them already tagged,  from the old POP account into the new gmail IMAP folders. The emails and their tags moved over fine, but now I cannot remove the tags. Please help :D
Thank you!
Summary: Unable to remove custom tags from messageson IMAP folders → Unable to remove custom tags from messages in IMAP folders
Version: unspecified → 2.0
Confirming on linux/trunk.
However, if I tag a message in the gimap inbox with two tags, and in the All mail folder remove the tags and add one other, the mail in inbox will have only the single tag.
Status: UNCONFIRMED → NEW
Ever confirmed: true
OS: Windows XP → All
Hardware: PC → All
Summary: Unable to remove custom tags from messages in IMAP folders → Unable to remove custom tags from messages in Gmail IMAP folders
Version: 2.0 → Trunk
The difference is probably due to the different storage methods for tags.  Messages in local folders store the tags in the message header, but IMAP messages store the tags as flags (keywords) on the server.
When you copy a message from local folders to IMAP, the "X-Mozilla-Keywords" header is included.  The header parser recognises it, and sets the tags. 
When you remove a tag from an IMAP message, it does not edit the header, as it would for a local message.  So, the tag gets reset when the header is parsed again.

(In reply to comment #3)
> When you copy a message from local folders to IMAP, the "X-Mozilla-Keywords" header is included.

FYI. This problem is already reported to Bug 426651.

(In reply to comment #2)
> However, if I tag a message in the gimap inbox with two tags,
> and in the All mail folder remove the tags and add one other,
> the mail in inbox will have only the single tag. 

I could observe similar/funny phenomenon with Gmail IMAP.
  1. Add several tags to a mail in a folder (other than [Gmail]/xxx).
     The mail doesn't have X-Mozilla-Keys: header.
  2. In "[Gmail]/All Mail", only last tag is set for the mail.
  3. Copy the tagged mail in the folder to Inbox => only last tag is displayed.
  4. "Rebuild Index" of the folder => only last tag is displayed.
     Remove tag => an other tag which was added at step 1 is displayed
     => Remove tag => an other tag which was added at step 1 is displayed
     => ...
  5. Add other several tags to the mail in "All Mail"
     => Added tags are not displayed at the other mail folder
     => Added tags are displayed when "Rebuild Index" of the other mail folder
Even though above problems exist in Gmail IMAP, "remove of tag cleanly" could be done, with combination of "remove a tag"(not "0 Remove All Tags") and "Rebuild Index" and "remove of tag at each mail folder".

To Magnus Melin:
I think problem you saw is an variation of above issues.
Could you open separate bug for "tagging in Gmail IMAP"? 
(In reply to comment #0)
> I cannot remove the tags.

To xavier roig(bug opener):

(Q1) Which is your operation of "remove tags"?
  (a) "0 Remove All Tags" in context menu
  (b) "Uncheck of a tag" for tag checked in context menu
(Q2) What phenomenon does your "cannot remove the tags" mean?
  - Tag displayed in message header box(or tags displayed at tag column in
    thread pane) won't disappear when you execute above operation
    of (a) or (b).
  - Disappears once, but appears again after some operations.
(Q3) Problem of "I cannot remove the tags" occurs on any tags?
     Or only when tags in X-Mozilla-Keys: header?
     (copied mail from POP3. Tags added at local mail folder for POP3)
> into the new gmail IMAP folders
(Q4) Which folder? Inbox? "All Mail" folder under "[Gmail]" folder?
(Q1) (a) and also by pressing the numer for the tag that is already selected, which usually cancels it out. 
(Q2) The tag disapears doing the above operations but reapears when I select another message or do another operation.
(Q3) It occurs on any tags
(Q4) All IMAP "Gmail" folders
(In reply to comment #7)
> (Q4) All IMAP "Gmail" folders

To xavier roig(bug opener):
Tagging at "[Gmail]/All Mail", and tagging at another mail folder(excluding [Gmail]/Trash & [Gmail]/Spam"), and tagging at other mail folders, for a same mail, are involved?

Please note next:
 - "Mail folder" in Gmail IMAP is "Label" in Gmail via Web
 - Gmail holds only single copy of a mail
   - A mail is held in "[Gmail]/All Mail" only (can be said real mail folder)
   - All other Gmail IMAP folders (excluding [Gmail]/Trash & [Gmail]/Spam")
     are similar to "Virtual Folder" in Tb
   - Tags are added to the single copy of a mail
 - So, "add/remove of a tag" on a mail at a folder affects on tags of the mail
   displayed in other mail folders.   
   (Similar to "concurrent tagging of an IMAP mail at multiple PC's)

> (Q2) The tag disappears doing the above operations but reapears when I select
> another message or do another operation.

Does it mean next?
  1. Tag of "X" was removed from a mail at a mail folder.
  2. Tag of "X" re-appeared on the mail in the mail folder after a while,
     even though tagging of the mail at other mail folder is not done.

I couldn't re-produce above, even with Tb 2.0.0.14. But I could observe next.
  1. Add tags of "X", "Y", "Z" to a mail at a mail folder.
  2. Re-build Index
     => Only tag of "Z" is displayed
  3. Remove "Z" => "Y" appears
  4. Remove "Y" => "X" appears
  This problem is not observed with Tb latest trunk.
  I think it's tag search issue and Bug 439548 & Bug 433175 has fixed it.
  See Bug 433175 Comment #21.
If multiple tags were added to a mail at local mail folder and copied(or moved) to IMAP folder, similar situation can occur.

To xavier roig(bug opener):
Multiple tags were added to a mail?
Can you see problem with latest Tb Trunk? (Bug 439548 & Bug 433175 are fixed)


Another question.
"Offline Use" is enabled for mail folders?
"Offline Mode"(Work Offline) is involved?
(Remove tag while offline, and tag reappears upon online)
To xavier roig(bug opener):

Because of design/spec of Gmail IMAP, "Tag of X re-appears after a while" can occur, if single copy of a mail in multiple folders & tagging at multiple folders, with Tb 2.0.0.14.
(Scenario)
 1. Copy a mail(no tag) to both of folder F1 and folder F2
 2. At F1, add "Important"
 3. At F1, remove "Important"
 4. At F2, add "Important"
 5. Click F1 => removed "Important" at step 3 re-appears
Your are talking about this phenomenon?

FYI.
Funny phenomenon in tagging with Gmail IMAP in comment #8(and comment #4) can easily be observed.
 1. Create two mail folders(say F1, F2)
 2. Copy a mail(no tag) to both of F1 & F2
 3-1. Click F1, add tag of "Important" at F1
 3-2. Click F2 => "Important" appears on mail in F2
 4-1. Click F1, add tag of "Later" at F1 => Important, Later
 4-2. Click F2 => "Important" disappears, "Later" appears
      (tag search issue. already resolved with Tb Trunk) 
 5. Remove tag of "Later" at F2 => "Later" disappears, "Important" appears
    Remove tag of "Important" at F2 => "Important" disappears
 6. Click F1 => Important & Later are still displayed at F1
    (Same as flag/keyword re-sync issue when concurrent tagging by multiple PC)
    (Still can be observed with latest Tb trunk)
New question.
> Bug summary : Unable to remove *custom tags*
The "custom tags" includes following pre-defined tags?
   Important, Work,    Personal, To Do,   Later
  ($label1,   $label2, $label3,  $label4, $label5 in prefs.js definition)
Or tags you manualy added only?

When tags(of Tb2, "label" till Tb 1.5) were added initially at local mail folder?
(Which version of Tb added tag(or label) when a mail is held in local mail folder)
  - Tb 1.5 saved label number in X-Mozilla-Staus(or Status2).
    And is not removed upon migration from "label" to "tag"
    by version up from Tb 1.5 or former to Tb 2.0.
    This produces "revive of removed tag" issue.
  - Funny phenomenon, transition of Later->Work->Important, seems to be
    a result of remaining "label" logic, even after change to "tag".
         
I've opened Bug 450246.
In the bug, Tb's problem relates to Tb's special tag of $label1 to $label6(former "label" of Tb) is not involved. And the problem explained in Bug 450246 is easily re-produced with Gmail IMAP because of special design/spec of Gmail IMAP.

To xavier roig(bug opener):

Question again.
Is Tb's special tag of $label1 to $label5(former "label" of Tb. Important, Work,    Personal, To Do, Later) is involved in your problem?
If yes, multiple special tag of $label1 to $label5 is involved in your problem?
If no special tag of $label1 to $label5 is involved in your problem, or if only one special tag of $label1 to $label5 is involved in your problem, same problem as Bug 450246 I've reported?
I also am having a similar problem:
I am using TB 2.0.0.19
Problem occurs with Gmail IMAP
I reinstalled TB 2.0.0.19 and problem persists only with the few email with problems-6 total.
I deleted all of the default tags and created 5 of my own.  On a few emails that I tagged using number 1, they became tagged as "Important". I tagged about 8 emails in the same session and only 6 email have this problem, 5 email I tagged with the number 2: 2 tagged as expected and 3 became tagged as "Work".  If I try to retag them as 1 or 2 or remove the tag, it appears changed until I view a different folder, then switch back to Gmail's inbox.  If I add more tags, the "Important" tag is still there.  I went into the config editor and the only the tags I created with the values I wanted were listed.  However, I looked at the mailnews.labels and the 5 default labels were there.  I don't know much about labels, but when I changed the value of the label "Important" to "Attention" the email that had the "Important" tag that I can't remove changed to "Attention".  I originally tagged these emails using the number pad (the way I always tag emails) and I have tried to get rid of the tag using 0 and right click remove all tags.  Has a bug like this been reported yet?  If so, I am sorry for the post.  Also, is there any way I can remove these labels?
Confirmed on current TB3 nightly.
As a note:
Comment 13 states that this is occurring with *all* tags, not just custom tags.  This is the case for me as well.  Propose that the bug title be changed to reflect this.
(In reply to comment #14)
> Confirmed on current TB3 nightly.

Which phenomenon in this bug with what "steps to reproduce" are you confirming with Tb3 nightly?  
I couldn't observe funny phenomenon I described in my comment #10 with Tb3 nightly(I used 2009/11/12 build).
I couldn't observe next issue too with Tb3 nightly.
  tag of mail in Gmail IMAP folder is not removed
  if tag is written in X-Mozilla-Keys: header(copied from local mail folder)

George Nassar, can you attach (a) tag definitions in prefs.js, (b) detailed steps to reproduce and description of phenomenon, (c) screen shots, and (d) IMAP log?
To George Nassar:
Please note next.
(i) If multiple tags are added to a mail, sufficient width of "Tag" column
    is required to see all tags at thread pane.
(ii) Order of flags(keywords) returned by IMAP is random.
     Order of flags(keywords) returned by Gmail IMAP is different
     between mail in a Gmail IMAP folder(Gmail Label) and same mail in
     "[Gmail]/Alll Mail". So, following can happen.
       mail in Inbox                   : tag-1, tag-2, tag-3
       same mail in "[Gmail]/All Mail" : tag-3, tag-2, tag-1
(iii) Delay of tag data propagation may occur at Gmail / Gmail IMAP.
      So following can happen temporary.
      1. mail in Inbox                   : tag-1, tag-2, tag-3
      2. same mail in "[Gmail]/All Mail" : tag-3, tag-2, tag-1
      3. remove tag-2 at "[Gmail]/All Mail"
      4. tag-1, tag-2, tag-3 is still displayed at Inbox.
Currently, replicable on some Gmail IMAP accounts:
1) mark an email with tag 1, 2 or 3 (not tested with others; tags left at default)
2) attempt to remove tag, via either unselecting the specific tag or using "remove all tags"
3) leave and re-enter the folder

Expected: message appears without the tag
Actual: message appears still tagged

However -- this seems to happen only with certain accounts, but not with others.  Some accounts will remove tags fine, some will not remove them.

Therefore, I thought it was a profile issue -- but it's not.  I created a brand new profile, and added two accounts -- one that had had the problem, one that had not -- into the profile.  The account that had the problems kept having the problems, and the account that didn't still worked fine.

Further examination seems to point to the problem occurring exclusively in GMail accounts tied to Google Apps for Domains, for whatever reason.  Have now tried with two Google Apps accounts and both have had the problem, and two regular Gmail accounts neither of which have had the problem.

prefs.js tags section:
user_pref("mailnews.tags.$label1.color", "#FF0000");
user_pref("mailnews.tags.$label1.tag", "Important");
user_pref("mailnews.tags.$label2.color", "#FF9900");
user_pref("mailnews.tags.$label2.tag", "Work");
user_pref("mailnews.tags.$label3.color", "#009900");
user_pref("mailnews.tags.$label3.tag", "Personal");
user_pref("mailnews.tags.$label4.color", "#3333FF");
user_pref("mailnews.tags.$label4.tag", "To Do");
user_pref("mailnews.tags.$label5.color", "#993399");
user_pref("mailnews.tags.$label5.tag", "Later");
user_pref("mailnews.tags.version", 2);

IMAP log of attempting to remove a tag (failed):

 1513092880[7f965fb7e450]: 5b5b9800:providentdata.com:S-INBOX:SendData: 34 IDLE
 1513092880[7f965fb7e450]: ReadNextLine [stream=59950d70 nb=10 needmore=0]
 1513092880[7f965fb7e450]: 5b5b9800:providentdata.com:S-INBOX:CreateNewLineFromSocket: + idling
+1513092880[7f965fb7e450]: 5b5b9800:providentdata.com:S-INBOX:SendData: DONE
+1513092880[7f965fb7e450]: ReadNextLine [stream=59950d70 nb=33 needmore=0]
+1513092880[7f965fb7e450]: 5b5b9800:providentdata.com:S-INBOX:CreateNewLineFromSocket: 34 OK IDLE terminated (Success)
+1513092880[7f965fb7e450]: 5b5b9800:providentdata.com:S-INBOX:ProcessCurrentURL: entering
+1513092880[7f965fb7e450]: 5b5b9800:providentdata.com:S-INBOX:ProcessCurrentURL:imap://george@providentdata.com:993/customKeywords%3EUID%3E/INBOX%3E5295%3E%3E$label2:  = currentUrl
+1513092880[7f965fb7e450]: 5b5b9800:providentdata.com:S-INBOX:SendData: 35 uid store 5295 -FLAGS ($label2)
+1513092880[7f965fb7e450]: ReadNextLine [stream=59950d70 nb=47 needmore=0]
+1513092880[7f965fb7e450]: 5b5b9800:providentdata.com:S-INBOX:CreateNewLineFromSocket: * 4121 FETCH (FLAGS (\Seen $Label2) UID 5295)
+1513092880[7f965fb7e450]: ReadNextLine [stream=59950d70 nb=15 needmore=0]
+1513092880[7f965fb7e450]: 5b5b9800:providentdata.com:S-INBOX:CreateNewLineFromSocket: 35 OK Success
+1513092880[7f965fb7e450]: 5b5b9800:providentdata.com:S-INBOX:SendData: 36 IDLE
+1513092880[7f965fb7e450]: ReadNextLine [stream=59950d70 nb=10 needmore=0]
+1513092880[7f965fb7e450]: 5b5b9800:providentdata.com:S-INBOX:CreateNewLineFromSocket: + idling

What should I take screen shots of?  Seemed pretty straightforward -- not hidden within the UI.  Messages are clearly highlighted when they're tagged and not when they're not.
For your added notes:
1) and 2) are issues affecting visibility of tags.  I am having no problem seeing the tags.  The problem is with seeing them when they *shouldn't* be there -- AKA removing the tags and having the remove fail, visible next time the listing gets pulled.
3) is not a concern.  All operations are occurring in the Inbox.  And enough time has passed where there is no delay concern.  I have literally dozens of emails which I've tried to remove the tags from over the past few weeks, that are still tagged.
(In reply to comment #18)
> +1513092880[7f965fb7e450]: 5b5b9800:providentdata.com:S-INBOX:SendData: 35 uid
> store 5295 -FLAGS ($label2)
> +1513092880[7f965fb7e450]: ReadNextLine [stream=59950d70 nb=47 needmore=0]
> +1513092880[7f965fb7e450]:
> 5b5b9800:providentdata.com:S-INBOX:CreateNewLineFromSocket: * 4121 FETCH (FLAGS
> (\Seen $Label2) UID 5295)

I wonder if Gmail's flags are case-sensitive?  TB is trying to remove $label2, but the message carries flag $Label2.   RFC3501 is silent on this, but implies that case is signficant.
I've been told that the RFC explicitly says case is not significant for keywords, though the text in question is really talking about the built-in keywords. But I suppose what's important here is what gMail is doing. George, you could try shutting down TB, and editing prefs.js by hand, changing the mailnews.label.$label1-5 prefs to be $Label1-5 and see if that fixes it for you.
I could try to fix the code to maintain the case of the keywords given back to us when removing keywords, except that it might cause issues with the way we're handling case-insensitivity.
I confirmed, from a terminal session, that Gmail does treat the keywords as case-sensitive:
1 uid store 70 +Flags ($Label2)
* 15 FETCH (FLAGS (\Seen $Label2) UID 70)
1 OK Success
2 uid store 70 -Flags ($label2)
* 15 FETCH (FLAGS (\Seen $Label2) UID 70)
2 OK Success
3 uid store 70 -Flags ($Label2)
* 15 FETCH (FLAGS (\Seen) UID 70)
3 OK Success
Well, interestingly, I don't think the problem is Gmail treating keywords as case-sensitive -- it seems to be *adding* initial caps.  And only on Google Apps accounts...
For the testing I did, I used emails that hadn't been tagged.  I added the tag, and then tried to remove it.  If it was merely an issue of case sensitivity, one would think the case of the label would've been the same when added and removed.

So I did a little more logging, and what I got back was *bizarre.*

Regular gmail account, tagging works as expected.  When it appears in the log, it's as $label2 (the one I'm using now, for testing.)

On that Google Apps account:

-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:ProcessCurrentURL: entering
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:ProcessCurrentURL:imap://george@providentdata.com:
993/customKeywords%3EUID%3E/INBOX%3E5299%3E$label2%3E:  = currentUrl
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:SendData: 42 uid store 5299 +FLAGS ($label2)
-1125124336[7fe7c287fbc0]: ReadNextLine [stream=dc421ed0 nb=57 needmore=0]
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:CreateNewLineFromSocket: * 4125 FETCH (FLAGS (\See
n \Answered $label2) UID 5299)
-1125124336[7fe7c287fbc0]: ReadNextLine [stream=dc421ed0 nb=15 needmore=0]
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:CreateNewLineFromSocket: 42 OK Success
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:SendData: 43 IDLE
-1125124336[7fe7c287fbc0]: ReadNextLine [stream=dc421ed0 nb=10 needmore=0]
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:CreateNewLineFromSocket: + idling
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:SendData: DONE
-1125124336[7fe7c287fbc0]: ReadNextLine [stream=dc421ed0 nb=33 needmore=0]
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:CreateNewLineFromSocket: 43 OK IDLE terminated (Su
ccess)
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:ProcessCurrentURL: entering
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:ProcessCurrentURL:imap://george@providentdata.com:993/customKeywords%3EUID%3E/INBOX%3E5299%3E%3E$label2:  = currentUrl
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:SendData: 44 uid store 5299 -FLAGS ($label2)
-1125124336[7fe7c287fbc0]: ReadNextLine [stream=dc421ed0 nb=57 needmore=0]
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:CreateNewLineFromSocket: * 4125 FETCH (FLAGS (\Seen \Answered $Label2) UID 5299)
-1125124336[7fe7c287fbc0]: ReadNextLine [stream=dc421ed0 nb=15 needmore=0]
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:CreateNewLineFromSocket: 44 OK Success
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:SendData: 45 IDLE
-1125124336[7fe7c287fbc0]: ReadNextLine [stream=dc421ed0 nb=10 needmore=0]
-1125124336[7fe7c287fbc0]: be7ab000:providentdata.com:S-INBOX:CreateNewLineFromSocket: + idling

So right after adding $label2 to FLAGS, the fetch gives back \Seen \Answered $label2 -- but then after trying to *unset* $label2, the fetch gives back \Seen \Answered $Label2.

I then checked to see if the unset request matters, and it doesn't.  Set $label2, and the first fetch comes back with "$label2"; any future operation (I tested by toggling the new email indicator, so sending -\Seen) returns the label with initial caps, "$Label2".

Now, I will test the idea in comment 20 to see if that successfully removes the initial-caps label from emails.
Is there any place that the tags should be changed other than prefs.js?  I think I'm doing something wrong -- I changed all the tag entries to use $Label(n) instead of $label(n) and now not only do I not see anything tagged at all (where I thought I'd still see the "uppercase tagged" emails as tagged), but I cannot tag or untag items any longer -- no tags appear in the tag drop-down, and hitting 1-5 does nothing.
no, prefs.js is the place to change - maybe we've hardcoded those tags to some extent. You should just set them back to $label1-5 - sorry that didn't work.
OK, thank you. I guess what I was trying to test was confirmed by comment 23 anyway -- that it would've properly removed the label if it were case-correct.

So the only question that remains, I guess, is why does Gmail seem to be adding initial caps to the label?  We're sending it to them correctly, right?

And is this just a horribly frustrating upstream problem that they need to solve, or is Google "big/important" enough where some kludge gets put in to have it deal with the tag problem? (Say, push a remove of both the lowercase and initial-case tags when a tag removal is called for, which would seem to be OK considering that our tags should be case-insensitive as per comment 21 anyway?)
(In reply to comment #27)

> So the only question that remains, I guess, is why does Gmail seem to be adding
> initial caps to the label?  We're sending it to them correctly, right?

I suspect to make it look nicer, so perhaps they're surfacing it to the user. Yes, we're sending it correctly.
 
> And is this just a horribly frustrating upstream problem that they need to
> solve, or is Google "big/important" enough where some kludge gets put in to
> have it deal with the tag problem?

I'll ping the gmail folks and see if they can fix it on their end. Otherwise, I'll have to investigate the kludge approach, though it may not be possible to keep all the different servers happy.
I did some more testing and discovered that Gmail preserves the case used the first time the keyword was defined to the account. 

I believe that Gmail maintains a keyword table per account and uses a case-insensitive search to find the entry when adding a label, but case-sensitive when removing.
From what I understand from Google folks, Brian is right. Also, if I understood correctly, the table is not cleared, so that case is permanent. So until either Thunderbird works around this, or gMail makes imap labels case-insensitive, you're unfortunately stuck. 

Thunderbird only ever looks for the lower case version of the label when looking in prefs, which is why my suggestion of changing the prefs didn't work.

Do you happen to know how the $Label1 was stored on a message in the google apps account folder? I'm pretty sure TB only sets $label1.
Yes, what he says makes sense.  Hopefully it can be fixed upstream, as that seems the cleanest solution.

I have been using this account with Thunderbird since early in 2.0.  If I recall correctly, somewhere around 2.0.0.13 or 14 is when this problem started cropping up.  Could it be that initial caps were used early on in the label implementation?

I've never used labels with anything other than TB.  In fact, the only other client I've ever used with Gmail other than the web interface has been TB.

I figure worst case, if Google can't/won't fix this, I can't imagine there would be a myriad of other varieties of capitalizations for labels, so the above kludge would seem to be a simple enough fix.  This problem has been reported in various ways often enough where this is bound to help a number of those folks.  It may be that the common thread among all us having the problem is that we started using tags early on.  It'd be nice to be able to support those of us that did so, if it's not a horribly intrusive fix.

(Still rather see Google comply with the RFC, though. Fingers crossed!)
My understanding is that Google plans to fix this, but I'm a little bothered that this wasn't caused by some other imap client. I don't have an explanation for that.
Reading through other bugs -- could it be that I started experiencing this bug at around the time the patch for bug 355205 landed?  Looking at the patch code, the "one at a time" tag removal it implements would potentially fail due to the problem we've noted above, but the prior "all at once" tag removal would've likely cleared the tags regardless of case.

I ask this because that would mean that it is very possible that the initial-uppercase labels were created on the Gmail account even earlier than 2.0.0.13-14 like I mentioned before, and I only started noticing that was a problem when we went to the remove-one-by-one method for clearing tags.  (Heck -- thinking about it, it seems likely that I may have been using some of these email accounts as early on as TB1.5... hard to say whether it was these specific accounts giving problems, as that was too long ago.)
Summary: Unable to remove custom tags from messages in Gmail IMAP folders → Unable to remove custom tags from messages in Gmail IMAP folders (somehow $LabelX was set instead of $labelX. Gmail IMAP is case sensitive on keyword)
Tb 1.5 or former seems to have used keyword of $Label1 to $Label5 for "Label".
> http://deflexion.com/2006/05/server-side-message-labels
$Label1 to $Label5 is seen in following source.
> http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapServerResponseParser.cpp#1814
> 1835     else if (!PL_strncasecmp(fNextToken, "\\Draft", 6))
> 1836       fSettablePermanentFlags |= kImapMsgDraftFlag;
> 1837     else if (!PL_strncasecmp(fNextToken, "$Label1", 7))
> 1838       labelFlags |= 1;
As PL_strncasecmp is used for both \Draft and $Label1, "$Label1" with upper "L looks to be used before transition from "Label" to "Tag" by Tb 2.
Summary: Unable to remove custom tags from messages in Gmail IMAP folders (somehow $LabelX was set instead of $labelX. Gmail IMAP is case sensitive on keyword) → Unable to remove predefined tags from messages in Gmail IMAP folders (somehow $LabelX was set instead of $labelX. Gmail IMAP is case sensitive upon keyword remove)
(In reply to comment #35)
> $Label1 to $Label5 is seen in following source.
> > http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapServerResponseParser.cpp#1814
> > 1835     else if (!PL_strncasecmp(fNextToken, "\\Draft", 6))
> > 1836       fSettablePermanentFlags |= kImapMsgDraftFlag;
> > 1837     else if (!PL_strncasecmp(fNextToken, "$Label1", 7))
> > 1838       labelFlags |= 1;
> As PL_strncasecmp is used for both \Draft and $Label1, "$Label1" with upper "L
> looks to be used before transition from "Label" to "Tag" by Tb 2.

That's not relevant w.r.t. the keywords we set on the server - strncasecmp does a case-insensitive compare, so it doesn't matter if we use "$Label1" or "$laBeL1" in the compare. But I believe you're right and 1.5 might have used $Label1 when setting keywords.
Following is IMAP log of Mozilla 1.4.0, with gmail.imap.com.
(lower $label2, $label5 is added by Tb 2 or Tb 3)

(1) Remove all labels(==0)
 
> SendData: 11 uid store 11623 -Flags ($Label1 $Label2 $Label3 $Label4 $Label5) 
> CreateNewLineFromSocket: * 1 FETCH (FLAGS (\Seen $label2 $label5 NonJunk) UID 11623) 
> CreateNewLineFromSocket: 11 OK Success 
Because $label5 exists, "Later" is displayed after "remove all".

(2) Add label of "Important"(==1)

> SendData: 10 uid store 11623 +Flags ($Label1) 
> CreateNewLineFromSocket: * 1 FETCH (FLAGS (\Seen $label2 $label5 NonJunk $Label1) UID 11623)

=> $Label1 is returned by Gmail IMAP
 
> CreateNewLineFromSocket: 10 OK Success 
> SendData: 11 uid store 11623 -Flags ($Label2 $Label3 $Label4 $Label5) 
> CreateNewLineFromSocket: * 1 FETCH (FLAGS (\Seen $label2 $label1 $label5 NonJunk) UID 11623) 

=> $label1 is somehow returned by Gmail IMAP at this step.
   Possibly because other mails already have $label1.

> CreateNewLineFromSocket: 11 OK Success
Because $label5 exists, "Later" is always displayed.

Removal of $Label1 to $Label5 by old mozilla/thunderbird may be a recovery procedure.
Summary: Unable to remove predefined tags from messages in Gmail IMAP folders (somehow $LabelX was set instead of $labelX. Gmail IMAP is case sensitive upon keyword remove) → Unable to remove predefined tags from messages in Gmail IMAP folders (Old Mozilla/Tb stored $LabelX instead of $labelX. Gmail IMAP is case sensitive upon keyword remove)
Just to clarify: are we waiting on Gmail to fix this?  If so, did we by any chance get a rough ETA from them for this?
My understanding is that Gmail is planning to fix this before we're planning to ship our next major version of TB, and we're planning on shipping our next version of TB next April. No promises, but that's my current understanding.
To all people suffering from this bug currently:

As I wrote in comment #37, Mozilla 1.4 removes $Label1 to $Label5 by "remove all Labels (0)". If this bug is critical for you currently, de-compress "Mozilla 1.4.x release build"(if MS Win, UNZIP win32.zip build, not installer build), define Gmail IMAP acount, and execute "remove all Labels" for all mails in [Gmail]/All Mail", "[Gmail]/Trash", and "[Gmail]/Spam", please.
As is stated in comments #29 and 30, Gmail preserves the case of the keywords used in its service, though it matches them case-insensitively for a lookup or add.

So if this is the case, it would seem like your process would not solve the problem.  At best, it would successfully remove the tags as a one-time thing, but if you were to start using tags again after that, the problem would still exist -- you'd be unable to remove them without firing up Mozilla 1.4 again.

So again, it's Google fixes the problem on their end, or we put in a kludge to remove both initial caps and non-caps versions of keywords.

I am obviously biased on this, but since, as I mentioned, there have been a number of bugs filed that would be solved by this problem, some under TB, some under comm-central, and more folks at the forums who have also noted the problem, this quick kludge would probably help a lot of people.
Sorry for the idiot question here - I am trying to use tags for GTD and have the problem  you all are talking about, removal of tags not working.
Does this mean we can't use tags at all globally in TB3 or is there an alternative?
Wiping prefs.js and starting over? 
Not clear if we can do anything here but stop using and wait.
Whiteboard: workaround: see comment #37
I think Google may have fixed this problem upstream as of a couple of days ago.
Testing from a terminal session confirms that removing flags is now case-insensitive in Gimap.
yay, thx for the update and testing.
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.