1851871 - Message header display delayed by images that are slow to load

Status: RESOLVED FIXED

(Thunderbird :: Message Reader UI, defect, P1)

Description

[Jonathan Kamens]

Attachment: sample-msg.eml

1. Drop the attached message into your inbox.
2. View the message and tell Thunderbird that you want to allow remote images in it.
3. Quit and restart Thunderbird.
4. View the message again.
5. Note that it takes 5 seconds for the message header in the preview pane to be populated.

The embedded image in the body that takes 5 seconds to load is blocking the rendering of the message header.

Possibly related to bug 1836180, but I'm filing a new bug because the root cause there may be different. It's not clear from reading that bug that a root cause was ever identified, so it could actually be the same, but given that we know the root cause for this bug, maybe we should fix that first.

Note: the example message depends on there being a CGI script on my server that serves up an image after a 5-second delay. If you want to long to work on this bug I may have deleted that script. But you should be able to pretty easily reconstruct an equivalent test case.

Comment 1

[Jonathan Kamens]

Worth noting: this also delays the message being marked read. That is, even if I have Thunderbird configured to mark a message as read as soon as I click on it in the thread pane, it doesn't get marked read until the images finish loading.

Comment 4

[gene smith]

I've been seeing the problem you point out for some time. With 110 or earlier (pre-SN), the headers displayed first before the message body. So almost as soon as you click on a message you see the new subject etc. Now when you click a message, the header remains blank (if the first message selected) or it shows the last message's header while the selected message's body is displaying.
Since your example .eml has an artificial 5 second delay for fetching the remote image, it's a good illustration of the problem.

The display of headers (and any attachments listed at the bottom) occurs here:
https://searchfox.org/comm-central/rev/c58bfdfa19b54597373ef699d595e857111545ab/mail/base/content/msgHdrView.js#552
and this only occurs after the body display is completely finished. Somehow it needs to happen before or in parallel with the display of body which occurs here:
https://searchfox.org/comm-central/rev/bef705d06ea76e7947145009d288eb9b093c2264/mail/base/content/aboutMessage.js#124

Comment 5

[gene smith]

A similar example of the problem is attached at bug 1840943 comment 22.
In this case something in the https links in the sample email is taking a while to finish so the signal to show the headers
https://searchfox.org/comm-central/rev/c58bfdfa19b54597373ef699d595e857111545ab/mail/base/content/msgHdrView.js#552 is getting delayed for about 35 seconds so the previous message's header remains visible until the message body loading finishes. I checked with wireshark that https activity continued until the STATE_STOP event occurs at the above searchfox link. I didn't try to figure out specially what in the email body is causing the delay (there are a lot of links).

Comment 6

[gene smith]

Re: regressionwindow-wanted
I'm pretty sure this is caused by code changes during SN transition. The sample emails provided by reporters work fine at the pre-SN 110 daily version I still have. In both cases the header appears first and then the body and in both cases, since the complete body load is delayed (5 second and 35 seconds) the bar graph remains active during this time with v110. (With SN version there is no bar graph but that's another bug.)

Comment 7

[gene smith]

If this is a regression that can be pinned on particular change set, I would guess this: https://phabricator.services.mozilla.com/D161656 in bug Bug 1799764.
This moved the call to processHeaders() out of the mime emitter code and into onStateChange(). Maybe when it was in mime emitter as soon as the headers are streamed they are displayed. (Headers are first in the stream.) With processHeader in onStateChange() in msgHdrView.js the whole message has to be streamed (from imap, pop3, webserver, disk cache or offline store) before the headers appear so there can be a significant delay as is being reported and observed.

BenC wrote this in his review of the patch. Maybe he is referring to current issue?:

I don't like how the population of the header/attachment data occurs at the moment, but thats more a factor of how the Mime stuff happens right now.
I think the population aspect could be changed in future without affecting users of the nsIMailChannel, so it's not too big a deal.

The change
https://hg.mozilla.org/comm-central/rev/8689605fa4c8901006b16e50aaa5e4d9bf6ce700
was pushed on 1-19-2023 along with many other changesets. After this the problem occurs. Before this the problem doesn't occur.

Comment 9

[Wayne Mery (:wsmwk)]

Geoff, should this all just move to bug 1840943? Or do we have two issues? And, do we now know enough to fix this?

Comment 11

[Geoff Lankow (:darktrojan)]

I think that, as of about 10 minutes ago, I may have a solution to both problems. At least I think I've found the problem, I'm not sure if my solution is good enough yet. Let's leave both bugs open for now since they may be more discoverable that way.

Comment 12

[Geoff Lankow (:darktrojan)]

In fact, they're two different but related problems. Bug 1840943 needs to be solved before this one can be.

Comment 13

[Wayne Mery (:wsmwk)]

(In reply to Geoff Lankow (:darktrojan) from comment #12)

In fact, they're two different but related problems. Bug 1840943 needs to be solved before this one can be.

Feedback on Bug 1840943 is looking pretty good.

Comment 14

[Geoff Lankow (:darktrojan)]

Attachment: Bug 1851871 - Notify the front end when headers are ready, and try to update the display then. r=BenC,aleca,freaktechnik

This is a little bit of a hack, but it should appear to perform much better.

Once nsMimeHtmlEmitter has set the message header data on the nsIMailChannel, it causes an
onStatusChange event to fire, and the front end responds by displaying the headers.

To avoid confusion when the headers update and the message doesn't (which annoyingly won't happen
until the whole message is ready), on STATE_START the previous message is cleared from the display.

Comment 15

[gene smith]

(In reply to Geoff Lankow (:darktrojan) from comment #14)

Created attachment 9353803 [details]
Bug 1851871 - Notify the front end when headers are ready, and try to update the display then. r=BenC,aleca,freaktechnik

Tested the diff 2 patch on the test messages. Looks good! The header appears immediately on select of the message and then the message body appears and attachment list appears after the message is completely obtained. Seems to be working the way it did with <=110 again !!

Comment 16

[Martin Maurer]

Just to make sure/or question:
I am the author of the duplicate entry bug 1852888
Are the bugfixes only related to IMAP? I am using only POP3, not using IMAP at all.

Comment 17

[gene smith]

(In reply to Martin Maurer from comment #16)

Just to make sure/or question:
I am the author of the duplicate entry https://bugzilla.mozilla.org/show_bug.cgi?id=1852888
Are the bugfixes only related to IMAP? I am using only POP3, not using IMAP at all.

No, the fix is not imap specific. I tried your test message on a POP3 folder and see the problem (the previously displayed message's header remains for about 30 seconds then the correct header appears). With the fix in place, the correct header appears immediately when message is opened in the POP3 folder. This should also be the same for Local Folders.

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/14a2afecab58
Notify the front end when headers are ready, and try to update the display then. r=BenC,aleca,freaktechnik

Comment 19

[Daniel Darnell [:dandarnell]]

Thunderbird 118.0b6:
https://hg.mozilla.org/releases/comm-beta/rev/fc317d2544f6

Comment 23

[Wayne Mery (:wsmwk)]

Comment on attachment 9353803 [details]
Bug 1851871 - Notify the front end when headers are ready, and try to update the display then. r=BenC,aleca,freaktechnik

(I want to be certain we don't miss this for 115.3.0)

Comment 28

[Wayne Mery (:wsmwk)]

Comment on attachment 9353803 [details]
Bug 1851871 - Notify the front end when headers are ready, and try to update the display then. r=BenC,aleca,freaktechnik

[Triage Comment]
Approved for esr115

Comment 29

[Daniel Darnell [:dandarnell]]

Thunderbird 115.3.0:
https://hg.mozilla.org/releases/comm-esr115/rev/b1bb2165c6ba