800035 - Users should not be forced to use special folders returned by XLIST or LIST (SPECIAL-USE)

Status: NEW

(MailNews Core :: Networking: IMAP, defect)

Description

[David Lechner (:dlech)]

I have come across a number of bug reports that have to do with users being forced to use the special folders returned by XLIST. This will also be an issue when SPECIAL-USE is implemented.

This bug will be for the actual fix and to track all of the related bugs.

Here are some comments from a related bug: 
------------------------------------------------------------------------------

Bug 533140 Comment 37 David Lechner (:dlech) 2012-09-15 19:09:58 CDT

I have a proposed solution to this. I would like some feedback before I attempt a patch.

...

We could add a 4th option to "Account Settings... > <Account> > Server Settings > When I delete a message:"

Current options are:
- Move it to this folder (selection is ignored if server supports XLIST)
- Just mark it as deleted (setting recommended by google)
- Remove it immediate

New option would be:
- Use server specified folder

The new option would be selected by default for servers that support XLIST and be disabled for servers that do not support XLIST. Functionally it would work the same as "Move it to this folder" does now.
If the server supports XLIST and "Move it to this folder" is selected, then the value returned by XLIST will be ignored and the user specified folder will be used.

This could be a good thing for other IMAP users (not just GMail) that want to bypass a server specified Trash folder.

...


Bug 533140 Comment 39 Irving Reid (:irving) 2012-09-20 10:59:38 CDT

My preference for the options would be

1) Mark message as deleted
2) Move to the server's Trash folder
3) Move to this folder: <select here>
4) Remove immediately

... with the wording cleaned up by UI review
------------------------------------------------------------------------------

So, my proposal now is to not only do this for the Trash folder, but also for Drafts, Sent, Archive and Junk folders.

Comment 1

[Vincent (caméléon)]

I should say that I am not convince that we should do that in Thunderbird core. I'd prefer we keep it simple, and instead of adding more options, trying to clear some of the existing one.
In fact we can have only one option:
1) Move it to the trash folder

If the server indicate the folder name, then no problem.
If it is not the case, then the user has to select it manually

Comment 2

[Ben Bucksch (:BenB)]

See also bug 558659 comment 40, TODO:
4. Folder prefs: Figure out how when to honor server flags and when user override. User override should be possible, but only as intentional act, not by merely clicking "OK" on a pref dialog that has the folder pref on it. That would solve the Junk folder case, and also prevent the other folders to be accidentally changed, too. Either:
4.a. Make the pref UI so that it has "default" (not a concrete folder) as default setting, or a checkbox (default off) to enable the folder selector, 
4.b. Make the pref UI show the (concrete) server-detected folder by default, and not save the folder to prefs unless it's different from the default. I'm leaning towards this solution.

Comment 3

[Magnus Melin [:mkmelin]]

(In reply to David Lechner (:dlech) from comment #0)
> Current options are:
> - Move it to this folder (selection is ignored if server supports XLIST)
> - Just mark it as deleted (setting recommended by google)
> - Remove it immediate
> 
> New option would be:
> - Use server specified folder

Lets not mix apples and oranges, the delete model is one thing, which folder you want to use (if any) is another.

Is there really much value in allowing users to change which the special folders are? I don't think so myself, but of course in case there are problems with servers supporting it badly etc, there's value in being able to turn it off completely (per account).

Comment 4

[WADA:World Anti-bad-Duping Agency]

Other than "Trash at Server Settings" can be freely selected at Copies&Folders and Junk Settings.
Adding "trash" in bug summary in order to avoid misunderstanding ad confusion.

Comment 5

[Ben Bucksch (:BenB)]

re comment 4:
No, the bug applies to all folders. See comment 2. Currently, a mere action of opening the prefs dialog and clicking OK would replace the folders provided by XLIST and use the Thunderbird's defaults, rendering the whole feature moot. That's just broken and must be fixed, and it applies to all folder types we have UI pref for.

re comment 3:
> Is there really much value in allowing users to change which the special folders are?

Yes. Users must be able to organize their own space, not be forced by the provider. For example, I hate the "OUTBOX" (all uppercase) that some French ISPs use, or even "Lettre envoyee". If I never use their Webmail, it doesn't matter what they set as default. Also, I might want Thunderbird to cooperate with another IMAP client that does not support XLIST, and uses other folders than the provider.

Comment 6

[WADA:World Anti-bad-Duping Agency]

(In reply to Ben Bucksch (:BenB) from comment #5)
> Currently, a mere action of opening the prefs dialog and clicking OK would replace the folders provided by XLIST and use the Thunderbird's defaults, rendering the whole feature moot.
> That's just broken and must be fixed, and it applies to all folder types we have UI pref for.

It's phenomenon only when (A) '"?????" Folder on: [ existent account ]' is used at Copies&Folders/Junk Setting and folder of "?????" doesn't exist, isn't it.

If (B) "Other: [ Extent Mail Folder ]", existent mail folder only is shown in folder list, and selected folder is used as Special Folder, as expected.
If (C) '"?????" Folder on: [ existent account ]' is used but folder of "?????" already exists, "?????" is used as Special Folder, as expected.

By (B)/(C), flag of special folder is set as expected in msgFolder.flags, although special folder flag of [Gmail]/xxx is not removed by simple operation of (B)/(C).

So, as for Copies&Folders/Junk Setting of Gmail IMAP account(XLIST is used), I think calling it "forced to use special folders returned by XLIST" is too strong, although (A) is surely called "just broken".

(A) is perhaps a result by changes for problems(account setting change is not saved, etc.) when wrong Junk folder is set in prefs.js(for example, non-existent folder at Others:, because the folder is deleted while Tb is not running).

Ben Bucksch, when other IMAP with XLIST or LIST (SPECIAL-USE), is any "special folder returned to XLIST" forced to use even when (B)/(C)?

Note-1:
To remove special icon from [Gmail]xxx, following is needed.
  At "Other:" of Copies&Folders, select [Gmail]/xxx as special folder.
  Close account setting, re-open account setting.
  At "Other:" of Copies&Folders, select other existent mail folder.
  Close account settings => icon of [Gmail]/xxx is change to ordinal one.


Trash case is different. Phenmenon in trash case is; 
- When trash selection is changed from [Gmail]/Trash to TrashX,
  icon of [Gmail]/Trash is changed from trash-can-icon to ordinal one,
  and icon of TrashX is changed to trash-can-icon.
  However, icon of [Gmail]/Trash is changed to trash-can-icon sooner or later,
  and icon of TrashX is changed to ordinal one.
i.e. There is no way to use other than [Gmail]/Trash as trash folder. 

This is true problem called "forced to use special folders returned by XLIST".
This is comment #0 and comment #1.

Note-2:
In addition to above, followin is oserved on trash.
- When "Move to trash" model and [Gmail]/IMAP is selected,
  if IMAP delete model is changed to "Just mark it as deleted" or "Remove
  immediately", icon of [Gmail]/Trash is changed to ordinal icon.
  Folder-flag of "Trash" is actually removed from msgFlder.flag of [Gmail]/Trash.
In this case, "Empty Trash" capability is lost.

Because relevant settings is different,
- Copies&Folder setting is held in 
    mail.identity.idX.???_folder_picker_mode
    mail.identity.idX.???_folder (folder URL)
- Junk Settings is held in 
    mail.server.serverN.spamActionTargetAccount (account URL)
    mail.server.serverN.spamActionTargetFolder (folder URL, for Others: setting)
- Trash name is held in
    mail.server.serverN.trash_folder_name (folder name string)
different changes for each feature is perhaps required.

Comment 7

[Ben Bucksch (:BenB)]

WADA, I've been working on this feature and spent a lot of time on it. What needs to be done is clear and listed in comment 2.

Comment 8

[WADA:World Anti-bad-Duping Agency]

(In reply to Ben Bucksch (:BenB) from comment #7)
> WADA, I've been working on this feature and spent a lot of time on it. What
> needs to be done is clear and listed in comment 2.

I see.

After fix of Bug 798663(status-thunderbird-esr17:fixed, landed on Tb 12.0.2) which resolved problem of Bug 815087, Tb 17.0.2 detects "Gmail IMAP or not" correctly, so problem of this bug was exposed to users who are using Gmail IMAP for long time with mail.server.serverN.hostname!=imap.gmail.com.
(a) Trash case : bug 829185
    hostname=imap.googlemail.com case.
    By auto-config, hostname=imap.googlemail.com is used, perhaps by definition
    order in ISPDB.
    So, complaint by users may increase after Tb 17.0.2.
(b) Archive case : bug 705491 comment #13
    Although other than [Gmail]/All Mail can be selected at UI and is used,
    "Archives Options..." is disabled and "archive to .../yy/mm" is impossible.

So, there are at least 4 variants in poblems caused by this bug, which needs solution of your comment #2.
(1) Trash via Server Settings
(2) Junk via Junk Settings
(3) Archive via Copies&Folders/archive folder ([Gmail]/All Mail relevant problem)
(4) Drafts/SentMail via Copies&Folders
Are we better to dup such bugs/problems to this bug?
Or it's better to keep them for each case?

By the way, SpecialUse is defined as follows. Should we pay attension to Queue also?
> http://mxr.mozilla.org/comm-central/source/mailnews/base/public/nsMsgFolderFlags.idl#106
> const nsMsgFolderFlagType SpecialUse = Inbox|Drafts|Trash|SentMail|
>                                        Templates|Junk|Archive|Queue;

Comment 9

[WADA:World Anti-bad-Duping Agency]

For following.
(b) Archive case : bug 705491 comment #13
    Although other than [Gmail]/All Mail can be selected at UI and is used,
    "Archives Options..." is disabled and "archive to .../yy/mm" is impossible.
This was also not "UI and XLIST" issue. This was "intentional forcing of something which is not directly related to XLIST" in order to avoid user's confusions or unwanted bug opens by general users what is caused by particularity of Gmail IMAP.

I incidently saw following code.
> http://mxr.mozilla.org/comm-central/source/mail/base/content/mailWindowOverlay.js#1683
Regardless of archive related settings, if isGMailServer=true, Tb currently forces archive_granularity=0. "Prohibiting 'Archive Setting...' in UI" is to avoid user's further confusions due to above logic.
This is similar to "forcing [Gmail]/Trash of Gmail IMAP".

"Forcing [Gmail]/Trash problem of Gmail IMAP" and "Forcing archive_granularity=0 problem of Gmail IMAP" are never directly related to "folder is specified in XLIST or LIST Special" nor "UI for folder selection". These two forcing problems are by particularity of [Gmail]/Trash and [Gmail]/All Mail in Gmail/Gmail IMAP.

I think above two forcing problems and following general "UI/XLIST or LIST Special/Folder Selection" related issues are better processed separately,
  (a) Folder Selection, defaulting, falling back etc. at Copie&Folders
      and Junk Settings, when XLIST or LIST Special is used.
  (b) Trash folder selection what is similar to Copie&Folders/Junk Settings.
because current phenomenon when folder is set in XLIST is roughly as follows;
(i) Tb's default of top level Archives, Drafts, Sent, Junk, Trash
    doesn't usually exist at IMAP server if Gmail.
(ii) Folders for them at server are set in XLIST by Gmail.
    (Note: If top level deault folder is created before Tb starts,       )
    (      Tb currently uses it, even when XLIST is returned from server,)
    (      because defaul is these top level folders of the account.     )
(iii) Users usually don't want automatic creation of Tb's default special folder
    on Gmail server.
(iv) So, current Tb tries to fall back to folder specified in XLIST,
    if default top level folder is not found.
    And, because folder specified in XLIST exists at server, it's used as
    special folder in Tb, and is usually set in Others: of Copies&Folders or
    Junk Settings.
    (I can't consider this is "forcing".)
(v) Because of never "forcing", user can request any folder at Copies&Folders and
    Junk Settings, and Tb uses requested folder by user as special folder in Tb.
(vi) If user deletes folder which user requested at Copies&Folders or Junk
    Settings while Tb is not running or Tb doesn't open the folder,
    by mechanism of (iv), Tb uses folder specified in XLIST.
    (I can't consider this is "forcing" too.)

Comment 10

[Kartik Subbarao]

I recently discovered this problem and wanted to share a workaround that preserves key parts of my preferred use model:

(1) I can press the Delete key on the keyboard, or the delete button in the GUI, to remove mail from my Inbox (or other folders), while preserving them in a folder (All Mail) where messages are not automatically deleted after 30 days.

(2) I can periodically purge old messages at a timeframe of my choosing, in a simple way.

I'm achieving (1) as follows:

In Thunderbird:

Account Settings->Server Settings->When I delete a message: "Remove it immediately"

"Remove it immediately", in the context of Gmail, doesn't have the same outcome as it might on other conventional IMAP servers. It does set the IMAP "Deleted" flag on the message, but in Gmail, all that effectively does is only remove it from the current folder. The message remains accessible in Gmail's "All Mail" folder.

In Gmail:

Settings->When I mark a message in IMAP as deleted: "Auto-Expunge on"

This just simplifies things for my use case by ensuring that Gmail's actions are immediately synced up with Thunderbird's actions.

References:

https://support.google.com/mail/answer/77657
https://support.google.com/mail/answer/78755
https://support.google.com/mail/answer/78892

I'm achieving (2) with the following Google Apps Script:

https://script.google.com/d/1xWTKAwhyul0SVGYrhWdHSMX1wylsfemcjmrsUX6NwxE1Jzj9_M53uJEG/edit?usp=sharing

This script automatically moves messages in "All Mail" older than 3 years into Gmail's Trash folder, where they will be then subject to Gmail's autodeletion policy. It excludes unread messages, Inbox messages, and messages that are in any of my user folders.

With Resources->Current project's triggers, I have set this to automatically run every month.

Comment 11

[Kartik Subbarao]

Sorry, I meant to post the previous comment to bug 533140. Looks like I can't delete it in Bugzilla :-(