1835891 - create gmail imap account results in incorrect folder list

Status: NEW

(Thunderbird :: Folder and Message Lists, defect, P2)

Description

[Anje]

Installed beta 114.05b
Create imap gmail account with no problem - excellent.

Folder Pane shows:
Inbox
Drafts
Sent Mail
Archived
Spam
Bin
INBOX
-- Mozilla Firefox
-- Samsung All 4
Online Orders
Starred
Testing

This is incorrect and not desired.

Expected:
Inbox
-- Mozilla Firefox
-- Samsung All 4
Drafts
Sent Mail
Archived
Spam
Bin
Online Orders
Starred
Testing

Checked the profile in 'ImapMail' > imap.gmail.com
I see:
[Gmail].sbd
[Gmail].msf
Archives.msf
Drafts.msf
INBOX
INBOX.msf
msgFilterRules.dat
Sent.msf

Note: There are no such gmail folders called 'Sent' and 'Archives' and I suspect this 'Drafts.msf' file is also wrong. It seems Thunderbird has created some files which would normally appear in a pop account.

Expected
No [Gmail].sbd folder and no [Gmail].msf file.
No false folders that normally appear when creating a pop mail account
All the items now found in [Gmail]sbd - see below to be listed under imap.gmail.com

In 'ImapMail' > imap.gmail.com > [Gmail].sbd
I see:
INBOX.sbd
Archived
Archived.msf
Bin
Bin.msf
Drafts
Drafts.msf
INBOX.msf
Online Orders.
Online Orders.msf
Sent Mail
Sent Mail.msf
Spam.msf
Starred
Starred.msf
Testing.msf

Expected
All the above in imap.gmail.com NOT in [Gmail]sbd

In 'ImapMail' > imap.gmail.com > [Gmail].sbd > INBOX.sbd
I see:
Mozilla Firefox
Mozilla Firefox.msf
Samsumg All 4
Samsumg All 4.msf

This was expected - but INBOX.sbd should be in imap.gmail.com and NOT in [Gmail].sbd

I want to remove the [Gmail],sbd - Tried:

• In Account Settings > Server Settings
• click on 'Advanced' button
• IMAP server directory is blank
• IMAP server directory type : [Gmail]
• click on 'OK'

Exit Thunderbird and restart Thunderbird.

Folder Pane Results:

• Every folder is duplicated except 'Inbox' - Not expected.
• The folders in the italic INBOX remain in that folder, however, they also appear as normal folders - not subfolders and not subfolders to the 'Inbox' which is where they are supposed to be. Neither expected.

** In profile folders... 'ImapMail' > imap.gmail.com**

I now see most of the correct files and folders I would expect to see.
The [Gmail].sbd has gone.
I see :
INBOX.sbd
Archived
Archived.msf
Archives.msf - this is still wrong at this point - I will need to delete this file
Bin
Bin.msf
Drafts.msf - I will need to delete this file
Drafts-1 - I will need to rename to remove the number
Drafts-1.msf - I will need to rename to remove number
INBOX
INBOX.msf
msgFilterRules.dat
Online Orders.
Online Orders.msf
Sent Mail
Sent Mail.msf
Sent.msf - this is still wrong at this point - I need to delete this file
Spam.msf
Starred
Starred.msf
Testing.msf

• Exit Thunderbird and delete wrong files and rename files as noted above.

• Start Thunderbird

Result
All folders display correctly.
The Inbox now displays correct subfolders.
The italic INBOX has gone.

Expected

When I edited - IMAP server directory type : [Gmail]
And restarted Thunderbird
All folders to be correct not duplicated.
I should not need to exit Thunderbird, edit files in profile and restart again.

Can someone please check to see if they can replicate?
I've removed account, added again and managed to get identical results each time.
It needs a double restart to remove duplicate folders and get correct arrangement.

Comment 1

[Anje]

Windows 10 OS
This is new install of beta 114.05b, so I'm presuming it would have the updates as per bug 1818592
It is also a new beta profile name folder with one gmail imap account newly created.
Basically I discovered
Did not expect Folder Pane folders and subfolders to display incorrectly.
In profile - ImapMail/ imap.gmail.com - Did not expect creation of non gmail files - eg: Sent, Drafts, Archives
Did not expect all files and folders to be in a imap.gmail.com/[Gmail].sbd folder.
Expected - IMAP server directory type : [Gmail] to be set up by default - it was not. Hence incorrect display in Folder Pane.
Expected - when I edited the above to say [Gmail] and restarted Thunderbird, all folders in Folder Pane to be correct as per profile files.
Did not expect a complete duplicate of folders and a second exit and restart to be required to fix display.

Comment 2

[Wayne Mery (:wsmwk)]

Please check 115.0b4 and 115.0b5

Comment 3

[Anje]

Attachment: 115.0b6 - create gmail imap.jpg

Beta 115.0b6
REmoved gmail imap account.
Removed saved passwords.
Created new existing mail account - gmail - imap.

Image shows what files and folders were created.

Folder: imap.gmail.com
Has [Gmail].sbd and [Gmail].msf - Not Expected.
Has Archives.msf, Drafts.msf and Sent.msf - Not expected and not imap folders.
It's as if account has auto created folders you would expect to see in a pop account.
Has INBOX which is expected and contains all Inbox emails.

Folder: [Gmail].sbd contains all the files and folder I expected to see in imap.gmail.com folder.
Folder: INBOX.sbd contains correct files, but display does not show them as subfolders to Inbox, they are in INBOX.

Not shown in above image but in 'Account Settings' > 'Server Settings' > 'Advanced'
Actual : IMAP server directory: blank
Expected: IMAP server directory : [Gmail]

Comment 4

[Anje]

Attachment: 115.0b6 - create gmail imap 2.jpg

Image shows:
Set the following:

• 'Account Settings' > 'Server Settings' > 'Advanced'

• IMAP server directory : [Gmail]

• Exit Thunderbird.

• Start Thunderbird

• Folder Pane shows all folders, But Two folders - Mozilla Firefoz and Samsung All 4 are not shown as subfolders of Inbox - Not Expected.

• Check profile folders

• [Gmail].sbd has been removed - Expected.

• All imap files and folders now in imap.gmail.com - Expected.

• Files - Archives.msf, Drafts.msf and Sent.msf - need to deleted as they are not imap files.

• Edited name of Drafts-1 and Drafts-1.msf to Drafts and Drafts.msf, so now correct. These files are imap files, but obviously got additional number because the non imap incorrectly created Drafts forced the filename with number.

As Folder Pane is still incorrect:

• Exit Thunderbird and restart Thunderbird for a second time
• Result - Folder Pane now correct.

The above is a lot of file fixing which should not be required when creating a gmail imap account.

When creating new existing mail account for gmail
Expected this setting to be auto set so no [Gmail].sbd folder in profile folders.

• 'Account Settings' > 'Server Settings' > 'Advanced'
• IMAP server directory : [Gmail]

Expected - No files that are not imap files to be in imap.gmail.com

Hope the info with images helps to show the weirdness which is occuring.

Comment 5

[Anje]

For clarity:
This is not a 'Unified folder' view
Settings : View > Folders > All

Comment 6

[Wayne Mery (:wsmwk)]

Do you have similar results if create a new account today?

As Folder Pane is still incorrect:
Exit Thunderbird and restart Thunderbird for a second time
Result - Folder Pane now correct.

bug 1843048 ?

Comment 7

[Wayne Mery (:wsmwk)]

(In reply to Wayne Mery (:wsmwk) from comment #6)

Do you have similar results if create a new account today?

As Folder Pane is still incorrect:
Exit Thunderbird and restart Thunderbird for a second time
Result - Folder Pane now correct.

bug 1843048 ?

I meant Bug 1843637

Comment 8

[Anje]

beta 116.0b5
Removed everything and created as new.
Identical results regarding what is seen in Folder Pane and what is seen in Profile folders.

Still had to set the following because it is left blank on account creation

'Account Settings' > 'Server Settings' > 'Advanced'
IMAP server directory : [Gmail]

and delete the incorrect non imap 'pop' 'ish folders that were incorrectly created in profile.
and restart twice for get the Folder Pane view corrected - note after fixing the profile folders - collapsing the account name folder and expanding again did not force the folders called :

Mozilla Firefox
Samsumg All 4

to be correctly reset as subfolders to Inbox - although at this point they are correct in the profile folders. A second restart did reset them as subfolders.

So nothing has changed.

Comment 9

[Anje]

I'm not sure my issue is identical to bug 1843637 as I am seeing all folders in Folder Pane - I do not have deep nested set of folders.

This time after setting 'IMAP server directory : [Gmail]' and a restart - I tried collapse/expand of chevron twisty to left of account name folder to see if it refreshed all the folders correctly but it did not have any effect - hence a second restart attempted which, as usual, resolved the display and put them as subfolders to 'Inbox'.

So I am seeing several notable issues when creating a gmail imap account.

• Non imap files created in imap.gmail.com profile folder
• Incorrect creation of [Gmail],sbd in imap.gmail.com profile folder
• 'IMAP server directory : [Gmail]' not set by default which causes the problem of [Gmail].sbd in profile and the creation of NoSelect INBOX folder in Folder Pane which now contains all Inbox subfolders.
• Requirement to do restart Twice to force Folder Pane to display correctly after setting 'IMAP server directory : [Gmail]'

Comment 10

[Anje]

Would this bug 1818592 be relevant?
https://hg.mozilla.org/releases/comm-beta/rev/035b1b71951f

It says fixed in 102*, but beta ?

Comment 11

[gene smith]

From comment 9:

I'm not sure my issue is identical to bug 1843637 as I am seeing all folders in Folder Pane - I do not have deep nested set of folders.

This time after setting 'IMAP server directory : [Gmail]' and a restart - I tried collapse/expand of chevron twisty to left of account name folder to see if it refreshed all the folders correctly but it did not have any effect

Bug 1843637 mainly affects users who un-check Advance Imap server setting "Show only subscribed folders". If you have advanced imap setting at default the initial folder discovery should work OK.

However, that bug also has a proposed patch to fix collapse/expand. With 115.0 release expand no longer re-triggers imap folder discovery and I've submitted a patch (awaiting review) to fix that in Bug 1843637.

Comment 12

[gene smith]

With 116, I created a new profile and configured one of my gmail account in it. I don't see any problems other than Archive.msf, Sent.msf and Drafts.msf appearing at the top level. I have a subfolder of Inbox and it appears and I don't see "INBOX" (all caps) in the folder pane, just Inbox at the top like it should with its subfolder.
I personally registered my disagreement with not showing the [Gmail] "folder" in the folder pane, e.g. bug 1816822 comment 11. However, it's only a display change on folder pane and otherwise [Gmail] still exists at gmail server and internally in TB. For example, if you look at gmail subscription you still see [Gmail] and any folder under it. Also, if you copy or move a message or folder to a gmail account you will see [Gmail] in the pop-up/expanding selection tree. It also shows up (with escape chars in place of the []s) in the right click folder properties for folders located inside [Gmail].
Another bad thing about hiding [Gmail] is if you had a folder with the same name at top level and under [Gmail] when you created it with <=102, it will appear as a duplicate folder with 115, which is the subject of bug 1816822 resolved as WONTFIX.

Comment 13

[Anje]

[Gmail] still exists at gmail server

Yes - via webmail any top level label I create I have to manually add [Gmail]/ prior to name eg: [Gmail]/Test - gmail does not do it automatically. But I do not need to do it for any subfolders created within a created top level label because gmail does manage to add [Gmail] automatically, even though it does not display it.
eg: [Gmail]/Test/Go does not exist in the gmail folder tree view, but it does exist in the copy label to another label list.
It is just [Gmail]/Test and that 'Go' label is displayed an indented subfolder eg: > Go

but I do need to do it if within a top level default folder
eg:
[Gmail]/INBOX/Mozilla
[Gmail]/Mozilla

So even Gmail webmail does not have the same rule applied generically.

The [Gmail] needs to be manually added/edited within webmail otherwise you cannot see labels as folders you would like to subscribe to see. I have had to advise people to access webmail account and alter the labels they created in webmail just to see them in Thunderbird.

In Thunderbird
Set 'IMAP server directory : [Gmail]'
user_pref("mail.server.server1.server_sub_directory", "[Gmail]");
Other than the above preference setting then I do not see that [Gmail] reference anywhere else in Thunderbird nor in Profile etc which is precisely what is required and desired in order to get a correct folder display.

With respect to all of your comments - I am not seeing [Gmail] because it does not exist after setting 'IMAP server directory : [Gmail]'.

if you look at gmail subscription you still see [Gmail] and any folder under it.

No [Gmail], just a list of folders in alphabeical order but I do see an italic NoSelect INBOX showing correct subfolders, but then it does not show 'Inbox' as a folder I can subcribe to see. It is probably assumed by default that Inbox is auto subscribed and therefore cannot be selected as subscribed or unsubscribed.

[Gmail] does not display in Account Settings > Synchronisation & Storage because I have folders set up correctly after going through the performance described above. Therefore the display is identical to Folder Pane.

Also, if you copy or move a message or folder to a gmail account you will see [Gmail] in the pop-up/expanding selection tree.

Only in webmail - Not in Thunderbird, I see just the correct account name and all the correct folders in the correct locations.

It also shows up (with escape chars in place of the []s) in the right click folder properties for folders located inside [Gmail].

No, not in my case.
eg: folder properties - imap://[e-mail]/INBOX/Mozilla%20Firefox

bug 1816822 seems to be part of this bug - when I set 'IMAP server directory : [Gmail]' and do restart all subfolders appear as top level until I do a second restart which then fixes the problem and correctly positions the subfolders under the correct top level folder.

Comment 14

[Anje]

The main problem occurs because for imap gmail account - 'IMAP server directory : [Gmail]' is not automatically set meaning the Folder Pane etc is incorrect.
The additional none imap files do not effect performance but should not be created, so clearly something is not correct.

Comment 15

[gene smith]

The main problem occurs because for imap gmail account - 'IMAP server directory : [Gmail]' is not automatically set meaning the Folder Pane etc is incorrect.

The problem with doing that automatically on gmail account creation is that any existing top level folder (on same level as Inbox) can't be seen in TB and can't be subscribed in TB. I just tried it and that is the case.

The additional none imap files do not effect performance but should not be created, so clearly something is not correct.

Yes they cause no harm that I see but don't know why they now appears. (I think you are referring to Archives.msf , Sent.msf etc.)

I still think if gmail.com imap server reports back a non-selectable folder [Gmail] (it's not really a "ghost folder") then TB should show it as a valid folder and NOT always hide it. If the user really wants to hide it they can set mail.server.serverX.server_sub_directory = [Gmail] in prefs or advanced imap setting. Another option would be on the account setup screen only for gmail to prompt to hide [Gmail] folder when creating the account and set mail.server.serverX.server_sub_directory accordingly. (I've never looked to see how other email clients handle hiding or showing [Gmail] folder.)

Comment 16

[Wayne Mery (:wsmwk)]

Please post a short update of current beta behavior

Comment 17

[Anje]

Using beta 119.0b1

Exactly the same as previously mentioned with minor alteration.
Drafts-1 was not created
Drafts-1.msf was not created
Archives.msf was not created

Sent.msf was created so needed to be deleted from profile gmail account.

Note: I had renamed 'Archives' to 'OldStore' and it auto set up to use as Archives folder which I did not expect .
But all the other alterations had to be made and the weird need to restart twice to get Inbox subfolders to appear under Inbox.

Comment 18

[Wayne Mery (:wsmwk)]

I'm wondering if https://support.mozilla.org/en-US/questions/1426444 is a variation of this

Comment 19

[gene smith]

(In reply to Wayne Mery (:wsmwk) from comment #18)

I'm wondering if https://support.mozilla.org/en-US/questions/1426444 is a variation of this

I'm not sure. I think Anje is setting up a new gmail imap account. The support user just has an existing account (only for gmail?) that now shows the folders in an unexpected order and he sees several folders with the same name at the same level after update from 102 to 115.
Anyhow, I need to read Anje's description again and hopefully get more info from support user.

Comment 20

[gene smith]

The support user was able to fix the issue by, more or less, following Anje's suggestion here: bug 1816822 comment 23.
Otherwise, I'm pretty sure the issue of this bug is caused by hiding the [Gmail] and other "non-selectable" folders in gmail accounts introduced in 115. The only solution to this is to back out that change (rejected as WONTFIX at bug 1816822) or to provide users with a pref to disable the hiding of non-selectable gmail folders as I proposed and submitted a patch for, also at bug 1816822.