1867091 - The menu item "Get selected messages" does not work at all - neither in the main nor in the context menu

Status: RESOLVED FIXED

(Thunderbird :: Folder and Message Lists, defect)

Description

[artem.semenov]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0

Steps to reproduce:

POP3, "Fetch headers only" mode.
After update to version 115.* it's impossible to use "Get selected messages" neither in the main nor in the context menu. In the main menu the item is always disabled, in the context menu (if at least two messages are selected in the message list) the item is always active, but does not cause any actions.

Actual results:

Nothing happens.

Expected results:

The menu item should be available if there are selected messages and perform its function.

Comment 1

[Wayne Mery (:wsmwk)]

Please specify your full version.
Does it also happen after Help > Troubleshoot Mode?
Any errors in developer > error console at startup?

Comment 2

[artem.semenov]

Attachment: developer > error console

Comment 3

[artem.semenov]

(In reply to Wayne Mery (:wsmwk) from comment #1)

Please specify your full version.
Debian 11, Thunderbird 115.5.0 64-bit (same behavior in 115.4.1).

Does it also happen after Help > Troubleshoot Mode?
Yes.

Any errors in developer > error console at startup?
The log is attached above.

Comment 4

[artem.semenov]

(In reply to Wayne Mery (:wsmwk) from comment #1)

Please specify your full version.

Debian 11, Thunderbird 115.5.0 64-bit (same behavior in 115.4.1).

Does it also happen after Help > Troubleshoot Mode?

Yes.

Any errors in developer > error console at startup?

The log is attached above.

Comment 5

[amichai2]

Same issue on Ubuntu 22.04 Thunderbird 115.6.0:

• I had a backlog of about 7K POP messages.
• TB couldn't handle it, silently stopping the download (of full messages) about 650 messages in. No indication of how or why it fails, just silently stops in the middle.
• Could not find any setting to download in limited batch sizes instead of all in one go, which would have been a good workaround.
• Every new email check started from scratch, so I ended up having those first 650 messages duplicated about 10 times each (every 10 minute check plus a few manual checks).
• I changed the account setting to fetch headers only. This time it made it to the end and got all 7K message headers. This also stopped the duplicate downloads, although the 7K message still remain on server until bodies are downloaded.
• Tried downloading the bodies, but could only do it one at a time by clicking the link displayed instead of the body inside each message. Takes a few seconds each, so not feasible for thousands of messages.
• File -> Offline -> Get Selected Messages is always disabled/grayed.
• Right-click -> Get Selected Messages is enabled (when two or more messages are selected) but does nothing. Bodies are not loaded, no error message, no success message, no nothing.
  Now I'm stuck with 600x10 duplicate messages with bodies, and another 6K messages with no bodies that I cannot view/download/search in reasonable time.

Comment 6

[artem.semenov]

Maybe it will be helpful.
The problem is that the condiction gViewWrapper.dbView.selectedCount > 0 is always false:

commandController.registerCallback(
  "cmd_downloadSelected",
  () =>
    gViewWrapper.dbView.doCommand(
      Ci.nsMsgViewCommandType.downloadSelectedForOffline
    ),
  () =>
    gFolder &&
    !gFolder.isServer &&
    MailOfflineMgr.isOnline() &&
    gViewWrapper.dbView.selectedCount > 0
);

Yesterday I built version 115, then I removed this condition (gViewWrapper.dbView.selectedCount > 0) in about3Pane.js. After that the menu item ("Get selected messages") is enabled and the action works properly.

Comment 7

[Hartmut Welpmann [:welpy-cw]]

(In reply to artem.semenov from comment #6)

Yesterday I built version 115, then I removed this condition (gViewWrapper.dbView.selectedCount > 0) in about3Pane.js. After that the menu item ("Get selected messages") is enabled and the action works properly.

Thanks for the pointer, that should be gViewWrapper.dbView.numSelected > 0. But there seems to be something else not quite right: Did you also notice that the header-only (dummy) messages didn't get removed after the complete messages have been downloaded?

Comment 8

[artem.semenov]

(In reply to Hartmut Welpmann [:welpy-cw] from comment #7)

Did you also notice that the header-only (dummy) messages didn't get removed after the complete messages have been downloaded?

The situation is more complicated. What you described doesn't happen if you just receive the rest of the messages - header-only (dummy) messages will disappear. However, if you run "Repair Folder" after that (from the folder context menu "Properties") without first compressing the folder ("Compact" from the folder context menu), the header-only (dummy) messages appear in the list again. There will be both complete and dummy messages. I didn't write about it because the program has been behaving like this for a long time. And I have already accepted this as an inevitable evil. :)

Comment 9

[amichai2]

Is this typo bug something that I can easily patch locally, or do I need to wait for the next release?

Comment 10

[Hartmut Welpmann [:welpy-cw]]

(In reply to artem.semenov from comment #8)

(In reply to Hartmut Welpmann [:welpy-cw] from comment #7)

Did you also notice that the header-only (dummy) messages didn't get removed after the complete messages have been downloaded?

The situation is more complicated. What you described doesn't happen if you just receive the rest of the messages - header-only (dummy) messages will disappear. However, if you run "Repair Folder" after that (from the folder context menu "Properties") without first compressing the folder ("Compact" from the folder context menu), the header-only (dummy) messages appear in the list again. There will be both complete and dummy messages. I didn't write about it because the program has been behaving like this for a long time. And I have already accepted this as an inevitable evil. :)

When downloading a single body via the link in the dummy message, I can reproduce this behavior exactly. But downloading bodies by "Get selected messages" didn't remove the dummy messages afterwards, even compacting didn't help. I tested this in in 115 and Daily, but this appears to have worked in 102.

Comment 11

[Hartmut Welpmann [:welpy-cw]]

(In reply to amichai2 from comment #9)

Is this typo bug something that I can easily patch locally, or do I need to wait for the next release?

You could try this:

1. Select the messages you want to download.
2. Open the Error Console (Ctrl-Shift-J)
3. Paste document.getElementById("tabmail").currentAbout3Pane.gViewWrapper.dbView.doCommand( Ci.nsMsgViewCommandType.downloadSelectedForOffline) and press Enter.

Comment 12

[Hartmut Welpmann [:welpy-cw]]

(In reply to amichai2 from comment #5)

Same issue on Ubuntu 22.04 Thunderbird 115.6.0:

• I had a backlog of about 7K POP messages.
• TB couldn't handle it, silently stopping the download (of full messages) about 650 messages in. No indication of how or why it fails, just silently stops in the middle.
• Could not find any setting to download in limited batch sizes instead of all in one go, which would have been a good workaround.
• Every new email check started from scratch, so I ended up having those first 650 messages duplicated about 10 times each (every 10 minute check plus a few manual checks).

Gene, do you have any insights?

Comment 13

[Hartmut Welpmann [:welpy-cw]]

(In reply to amichai2 from comment #5)

Same issue on Ubuntu 22.04 Thunderbird 115.6.0:

• I had a backlog of about 7K POP messages.
• TB couldn't handle it, silently stopping the download (of full messages) about 650 messages in. No indication of how or why it fails, just silently stops in the middle.
  [...]
  Now I'm stuck with 600x10 duplicate messages with bodies, and another 6K messages with no bodies that I cannot view/download/search in reasonable time.

Just another idea: Switch "Fetch headers only" off. Remove (and possibly backup) the files Inbox, Inbox.msf and popstate.dat from the server's folder under the Mail folder in your profile directory. Then try again getting all the messages.

Comment 14

[Hartmut Welpmann [:welpy-cw]]

(In reply to Hartmut Welpmann [:welpy-cw] from comment #10)

But downloading bodies by "Get selected messages" didn't remove the dummy messages afterwards, even compacting didn't help. I tested this in in 115 and Daily, but this appears to have worked in 102.

This actually depends now on the "Leave messages on server" setting. If disabled, the selected dummy headers are purged afterwards by nsPop3Sink::CheckPartialMessages.

Comment 15

[Hartmut Welpmann [:welpy-cw]]

Attachment: Bug 1867091 - Fix "Get selected messages" command. r=benc,mkmelin

Comment 16

[gene smith]

(In reply to Hartmut Welpmann [:welpy-cw] from comment #12)

Gene, do you have any insights?

Not really, other than to confirm the bug described in comment 0. But looks like you have a proposed fix...

Comment 17

[amichai2]

(In reply to Hartmut Welpmann [:welpy-cw] from comment #11)

3. Paste document.getElementById("tabmail").currentAbout3Pane.gViewWrapper.dbView.doCommand( Ci.nsMsgViewCommandType.downloadSelectedForOffline) and press Enter.

Thanks, I tried this on one small folder first, and it did download all the bodies, but they are now all duplicates - there is one with and one without the body. Dunno if it's related but just thinking out loud - the messages in this folder are automatically moved there by filters on the inbox, so maybe when it downloads it with body the new message goes to the inbox first, is not recognized there as a duplicate, and then is moved back to the target folder where there are now both copies? Or maybe that's unrelated and it's just a bug :-)

Comment 18

[Hartmut Welpmann [:welpy-cw]]

(In reply to amichai2 from comment #17)

(In reply to Hartmut Welpmann [:welpy-cw] from comment #11)

3. Paste document.getElementById("tabmail").currentAbout3Pane.gViewWrapper.dbView.doCommand( Ci.nsMsgViewCommandType.downloadSelectedForOffline) and press Enter.

Thanks, I tried this on one small folder first, and it did download all the bodies, but they are now all duplicates - there is one with and one without the body. Dunno if it's related but just thinking out loud - the messages in this folder are automatically moved there by filters on the inbox, so maybe when it downloads it with body the new message goes to the inbox first, is not recognized there as a duplicate, and then is moved back to the target folder where there are now both copies? Or maybe that's unrelated and it's just a bug :-)

Even without any filtering involved, dummy messages aren't replaced by completed ones, at least if "Leave messages on server" is enabled, so that's definitely a regression.

Try sorting the messages by "Order received". This should make it relatively easy to remove the empty messages.

Comment 19

[amichai2]

Even if sorted, manually deleting every second email doesn't scale to thousands of emails...
"Leave messages on server" is not enabled and never was.

Comment 20

[Hartmut Welpmann [:welpy-cw]]

(In reply to amichai2 from comment #19)

Even if sorted, manually deleting every second email doesn't scale to thousands of emails...

When sorted by "Order received" (not by "Date" or "Received"), all of the incomplete messages should appear en bloc, if I am not mistaken.

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/e4a1500b5534
Fix "Get selected messages" command. r=BenC

Comment 22

[Martin Schröder [:mschroeder]]

Backout happened: https://hg.mozilla.org/comm-central/rev/77d77193b439f2a32410ef965701100c61e5a48c

Comment 23

[Hartmut Welpmann [:welpy-cw]]

Apologies for this careless mistake, something got lost in refactoring. The updated revision can be built.

Comment 24

[John Bieling (:TbSync)]

Please attach a try run with the following settings: "try: -b do -p all -u all"

Comment 25

[John Bieling (:TbSync)]

Removing the check-in flag, to prevent this being landed before a clean try build has finished.

Comment 26

[Hartmut Welpmann [:welpy-cw]]

Thank you, John. Here is the try run. I'm not sure what to make of the failed mochitests under Windows, but they appear to have happened before and I don't see an obvious connection to my patch.

Comment 27

[John Bieling (:TbSync)]

Thanks for the full try, looks good!

Comment 28

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/30b899294372
Fix "Get selected messages" command. r=BenC

Comment 29

[Hartmut Welpmann [:welpy-cw]]

Comment on attachment 9378558 [details]
Bug 1867091 - Fix "Get selected messages" command. r=benc,mkmelin

[Approval Request Comment]
Regression caused by (bug #): bug 1830600 and bug 1789774
User impact if declined: "Get selected messages" does not work at all, so POP3 "Fetch headers only" mode is practically useless.
Testing completed (on c-c, etc.): yes
Risk to taking this patch (and alternatives if risky): rather low

Comment 30

[Wayne Mery (:wsmwk)]

Comment on attachment 9378558 [details]
Bug 1867091 - Fix "Get selected messages" command. r=benc,mkmelin

[Triage Comment]
Approved for esr115

Comment 31

[Rob Lemley [:rjl]]

Thunderbird 115.10.0:
https://hg.mozilla.org/releases/comm-esr115/rev/8a66a4e480c6