1908761 - Quick Filter stalls computer and takes 10-20 seconds to display results when using certain sort types.

Status: RESOLVED FIXED

(Thunderbird :: Folder and Message Lists, defect, P1)

Description

[Noah]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0

Steps to reproduce:

Go to folder with 20k+ email and use quick filter

Actual results:

Thunderbird stalls and windows shows as not responding for 10-20 seconds until it is able to display results. Performance capture here: https://profiler.firefox.com/public/kysamrkxp4d55z9p3fqyyqj1491njepjc9f0s7r/calltree/?globalTrackOrder=01&thread=0&v=10

Expected results:

In all prior versions the quick filter was almost instant with no stalling.

Comment 1

[Wayne Mery (:wsmwk)]

NughShwark's profile is good https://profiler.firefox.com/public/kysamrkxp4d55z9p3fqyyqj1491njepjc9f0s7r/calltree/?globalTrackOrder=01&thread=0&v=10

Lots of jank https://share.firefox.dev/4d6wy4f in the flame graph.

Comment 2

[Wayne Mery (:wsmwk)]

Noah, what sort column and order is being used on this folder of 22k messages?

Comment 3

[Noah]

The displayed columns, in order, are: Thread (set to unthreaded), read status, attachments, correspondents, date and size.
Folder is sorted by date with newest at the bottom.

Let me me know if you need more info.

Comment 4

[Martin Giger [:freaktechnik]]

In the profile we can see that this involves JS -> C++ -> JS logic. And we can also see that we're spending about 500ms per call in parseDecodedHeader, and half of that in getHeaderTokens. There are also two massive GCs during the 10s we spend sorting in nsMsgDBView::Sort (which calls the previously mentioned parseDecodedHeader with GetCollationKey, which uses IsOutgoingMsg that leads us to try and decode the mime header).

I'm not familiar enough with this part of code to go any further than that, but maybe we should avoid trying to parse the headers for the code path we use for sorting of messages? Surely we already have the information in the DB?

Comment 5

[Hartmut Welpmann [:welpy-cw]]

Noah, can you please try the following workaround and report back:

1. Select column "Order received" to display.
2. Click on "Order received" column header.
3. Click on "Date" column header.

This should be the fastest combination of primary and secondary sort types. Regarding the performance profile Martin is talking about, my guess is that the secondary sort was (unintentionally) set to by "Correspondent", which currently has the most severe performance issues.

Comment 6

[Francesco]

See my comment in bug 1908758 comment #4.

Comment 7

[R de Wit]

(In reply to Hartmut Welpmann [:welpy-cw] from comment #5)

Noah, can you please try the following workaround and report back:

1. Select column "Order received" to display.
2. Click on "Order received" column header.
3. Click on "Date" column header.

Since I'm affected by this bug too, I tried out a similar approach and first clicked on the "Size" column (didn't want to add the "Order received" column") and then on the "Date" column and I can confirm that Quick Filter works much, much faster after that. HTH.

Comment 8

[HK-47]

Confirmed that worked. Interestingly after sorting by order received then date, I'm unable to replicate the slowdown by any means including going back to my prior "received" sort, so this looks like a permanent fix.

Comment 9

[HK-47]

Confirmed that worked. Interestingly after sorting by order received then date, I'm unable to replicate the slowdown by any means including going back to my prior "received" sort, so this looks like a permanent fix.

Comment 10

[HK-47]

(In reply to HK-47 from comment #9)

Confirmed that worked. Interestingly after sorting by order received then date, I'm unable to replicate the slowdown by any means including going back to my prior "received" sort, so this looks like a permanent fix.

Don't know why it double-posted, please delete the dupe.

Wanted to add that I made the change via the system menu "View | Sort By" as I use the vertical cards view. That worked fine.

Comment 11

[Hartmut Welpmann [:welpy-cw]]

Attachment: Bug 1908761 - Simplify and improve handling of secondary sorts in DBViewWrapper. r=#thunderbird-reviewers

To set the secondary sort, DBViewWrapper lets the view sort itself twice in many cases such as
entering/leaving Quick Filter or leaving Grouped By Sort, as well as just changing the sort type
itself. Since nsMsgDBView::Sort() already takes the secondary sort into account, these
additional calls can be removed, which can significantly improve performance depending on the
individual sort types.

When leaving Grouped By Sort, the secondary sort type will always be set to Order Received, which
prevents having a comparably slow sort type such as by From, Recipient, or Correspondents as
secondary sort in place.

Comment 13

[Noah]

(In reply to R de Wit from comment #7)

(In reply to Hartmut Welpmann [:welpy-cw] from comment #5)

Noah, can you please try the following workaround and report back:

1. Select column "Order received" to display.
2. Click on "Order received" column header.
3. Click on "Date" column header.

Since I'm affected by this bug too, I tried out a similar approach and first clicked on the "Size" column (didn't want to add the "Order received" column") and then on the "Date" column and I can confirm that Quick Filter works much, much faster after that. HTH.

Just tried this along with removing the "Correspondents" column and using the recipient (for sent mail folder) and it's snappy again. I had already had it sorted by date prior to adding the order received column and while that didn't appear to change the order vs the date, it works now even after bringing back correspondents.

Thank you all!

Comment 14

[laurence.norah]

Following the workaround steps in this has resolved this issue for me, making TB quickfilter usable again in 128. Thanks!

Comment 15

[Worcester12345]

This is borderline S2. If this is what is shipping out to users now, definitely S2 severity.

Comment 16

[Alessandro Castellani [:aleca]]

Great finding and exploration in this bug, thank you all for your valuable help.
Welpy, do you have any intuition or potential approach on what the performance problem in the Correspondent field is and if you identified any pointer on how to tackle it?

I'm gonna leave this as S3 because Correspondent is not a default sorting order and (I don't think) we changed it during upgrade, so users that haven't changed the default will not be affected, and an easy workaround exists.

Setting P1 so we can try to tackle it before the next point release.

Comment 17

[Hartmut Welpmann [:welpy-cw]]

(In reply to Alessandro Castellani [:aleca] from comment #16)

Welpy, do you have any intuition or potential approach on what the performance problem in the Correspondent field is and if you identified any pointer on how to tackle it?

https://searchfox.org/comm-central/rev/3554a691473aa58c8430d879a2749489194bf96d/mailnews/db/msgdb/src/nsMsgDatabase.cpp#2998-2999,3005
https://searchfox.org/comm-central/rev/3554a691473aa58c8430d879a2749489194bf96d/mailnews/base/src/nsMsgDBView.cpp#466,498-499
https://searchfox.org/comm-central/rev/3554a691473aa58c8430d879a2749489194bf96d/mailnews/base/src/nsMsgDBView.cpp#856,862

These functions, which are needed to process message headers containing addresses, rely on JSMime. Each call to DecodedHeader or EncodedHeader involves some XPConnect overhead, which adds up to a significant amount. For a possible approach see bug 1089299.

Comment 18

[Hartmut Welpmann [:welpy-cw]]

(Essentially what Martin already discovered in comment 4)

Comment 19

[Worcester12345]

(In reply to Alessandro Castellani [:aleca] from comment #16)

I'm gonna leave this as S3 because Correspondent is not a default sorting order and (I don't think) we changed it during upgrade, so users that haven't changed the default will not be affected, and an easy workaround exists.

Not sorting, or even showing "Correspondent" here!

Comment 20

[Hartmut Welpmann [:welpy-cw]]

(In reply to Worcester12345 from comment #19)

(In reply to Alessandro Castellani [:aleca] from comment #16)

I'm gonna leave this as S3 because Correspondent is not a default sorting order and (I don't think) we changed it during upgrade, so users that haven't changed the default will not be affected, and an easy workaround exists.

Not sorting, or even showing "Correspondent" here!

The underlying cause identified in this bug is about sorts by From, Recipient, or Correspondents. Each of them is slower than other sort types, especially when active as secondary sort. So if the workaround provided in comment 5 doesn't have any effect for you, there is another cause.

By the way, I did some tests with a slow debug build. The issue is most obvious when leaving quick filter. These are the rough measurements for about 8000 unthreaded messages in a local folder:

primary sort	secondary sort	without patch	with patch
Date	Corresp.	37s	1s (!)
Subject	Corresp.	52s	17s
Corresp.	Subject	13s	9s

Comment 21

[Worcester12345]

I don't think this is the case for me. I don't even know what a "secondary sort" is or how to turn it on. I don't think I've ever used that.

This is absolutely KILLING use of email. I hope other users here are not having the same thing.

Comment 22

[Worcester12345]

Oh, I wanted to add, this is pretty new. I don't recall it doing this before version 128 came out.

Comment 23

[Francesco]

@Worcester12345: Please note what "secondary sort" is. It's always active, no need to turn it on, and if you ever sorted by a column, you have used it. Here's the story via example. If you first sort by date ascending and then by sender, you will get all the mail sent by a specific person "sorted together" and inside the "group", sorted by date ascending. Date ascending is the secondary sort. If you first sort by date descending and then by subject, you get a different result. This functionality has existed for decades, sadly there is no secondary sort indicator, so many people don't recognise the hidden feature.

IOW: A secondary sort can "hang around" and you don't even notice.

Comment 24

[Worcester12345]

How does one turn this off? Click the column twice?
This is taking MINUTES to just do what took SECONDS a week ago.
It is absolutely killing productivity!!!!!!!!!
This is on a live production system, not "beta", not "nightly".

Comment 25

[laurence.norah]

(In reply to Worcester12345 from comment #24)

How does one turn this off? Click the column twice?
This is taking MINUTES to just do what took SECONDS a week ago.
It is absolutely killing productivity!!!!!!!!!
This is on a live production system, not "beta", not "nightly".

For me, following the instructions in comment 5 of this thread resolved the issue. https://bugzilla.mozilla.org/show_bug.cgi?id=1908761#c5

Comment 26

[Worcester12345]

(In reply to laurence.norah from comment #25)

(In reply to Worcester12345 from comment #24)

How does one turn this off? Click the column twice?
This is taking MINUTES to just do what took SECONDS a week ago.
It is absolutely killing productivity!!!!!!!!!
This is on a live production system, not "beta", not "nightly".

For me, following the instructions in comment 5 of this thread resolved the issue. https://bugzilla.mozilla.org/show_bug.cgi?id=1908761#c5

That did indeed stop the symptoms for me. I am not going to try to explain this to a bunch of users, though. It still needs a real fix.

Comment 27

[Alessandro Castellani [:aleca]]

I'm going to review the proposed patches and do some profile testing.
If everything is good we will uplift this to 128 for the next point release.

Comment 28

[alex89]

We also encountered this issue. It is relevant even after a clean installation of Thunderbird, with the number of emails ranging from 20 to 60 thousand in a folder. This applies to both maildir and mbox email storage options, and it does not depend on the performance of the computer.

Comment 29

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/cb5dead87b07
Simplify and improve handling of secondary sorts in DBViewWrapper. r=aleca,babolivier

Comment 30

[Alessandro Castellani [:aleca]]

We got a new failure that it might be related to this patch, but I'm not sure, and it seems to happen only on Windows debug builds.
Anyone able to reproduce it locally?
Bug 1911891

Comment 33

[Wayne Mery (:wsmwk)]

Is bug 1911891 a bad regression? If not, we should proceed to uplift this P1 to beta and 128.

Comment 34

[Hartmut Welpmann [:welpy-cw]]

(In reply to Wayne Mery (:wsmwk) from comment #33)

Is bug 1911891 a bad regression?

Not at all, in my opinion. While the automatic test fails (only) on the Win32 Debug build, the feature itself appears to work using the exact same build.

If not, we should proceed to uplift this P1 to beta and 128.

D'accord.

Comment 35

[Wayne Mery (:wsmwk)]

Comment on attachment 9416611 [details]
Bug 1908761 - Simplify and improve handling of secondary sorts in DBViewWrapper. r=#thunderbird-reviewers

[Triage Comment]

[Approval Request Comment]
Regression caused by (bug #): -
User impact if declined: TB is very slow
Testing completed (on c-c, etc.): 2 weeks on nightly
Risk to taking this patch (and alternatives if risky): low, has tests

Comment 36

[Corey Bryant]

Comment on attachment 9416611 [details]
Bug 1908761 - Simplify and improve handling of secondary sorts in DBViewWrapper. r=#thunderbird-reviewers

[Triage Comment]
Approved for beta

Comment 37

[Rob Lemley [:rjl]]

Thunderbird 130.0b3:
https://hg.mozilla.org/releases/comm-beta/rev/fce31770de0e

Comment 40

[Wayne Mery (:wsmwk)]

128 uplift is needed given the widely reported impact to users

Comment 41

[Wayne Mery (:wsmwk)]

Comment on attachment 9416611 [details]
Bug 1908761 - Simplify and improve handling of secondary sorts in DBViewWrapper. r=#thunderbird-reviewers

[Triage Comment]
Approved for esr128 for severe performance issue

Comment 42

[Rob Lemley [:rjl]]

Thunderbird 128.2.0esr:
https://hg.mozilla.org/releases/comm-esr128/rev/835732357595