540288 - News article is empty if selected during download from news server

Status: RESOLVED FIXED

(MailNews Core :: Networking: NNTP, defect)

Description

[Christian Franke]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100104 SeaMonkey/2.0.2
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100104 SeaMonkey/2.0.2

If a new news article are selected for display when "Get Messages For Account" is still in progress, the article is shown as empty. A restart of SM fixes this.


Reproducible: Always

Steps to Reproduce:
1. Select a newsgroup account with many subscribed groups
2. RightClick|Get Messages for Account
3. Select one of newly added articles
4. Wait until download finishes

Actual Results:  
The selected article is not shown. Reselecting the article does not help. 

Expected Results:  
The selected article should be shown.


If several new articles are selected during download, all are not visible.

The newsgroups used for testing are not selected for offline use.

Bug was also present in SM 2.0, but not in any SM 1.1.*.

Comment 1

[Tobias Fischer]

Build identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.8pre) Gecko/20100117 Mnenhy/0.8.0pre6 SeaMonkey/2.0.3pre

I have seen this one time on 2010-01-15 too, while try to read an Posting in d.c.s.m.browser via news.individual.de. Because I am a little surprised to get an empty Posting, I have tried to read the Source-Code (Ctrl-U), but there was nothing been displayed too until I restart SeaMonkey. 

I have tried to reproduce this in Thunderbird: 
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100117 Shredder/3.0.2pre
but was unable to do so. 
I can not reproduce this issue properly again, so I left this as unconfirmed for the moment. 

Christian, please add your News-Server were you run into this, try to reproduce this again, and take a look into the Error-Console when you run into this issue again.

Comment 2

[u331436]

I also have this behavior, on an internal (my company) news server as well as on the servers (through a proxy) for seamonkey, thunderbird and calendar (mozilla.dev.apps and mozilla.support.apps). SM restart allows to read the 1st news (the next ones on a considered servers are read OK) and reading the ones already read are fine. I also saw that Ctrl U brings an empty page.

(SM 2.1a1pre on Win XP SP3 and Linux, with lightning nightly)

Comment 3

[Christian Franke]

(In reply to comment #1)
> Christian, please add your News-Server were you run into this, try to reproduce
> this again, and take a look into the Error-Console when you run into this issue
> again.

News-Servers: various, including news.mozilla.org

Error Console:
- After first open of "Mail & Newsgroups" window (likely unrelated):
 Error: this.mPanelContainer is null
 Source File: chrome://navigator/content/tabbrowser.xml
 Line: 2294
- After steps from comment 0:
 (No new messages)

Comment 4

[Alfred Peters [:infofrommozilla]]

I can confirm this for TB3.* (atm.:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1pre)
Gecko/20100123 Lightning/1.1a1pre Shredder/3.2a1pre)

Symptoms:
- Immediately after starting header download the status line displays "there are no new messages on the server" – what is obvious wrong.
- The message window is redrawn immediately - but in my server log I can see that the ARTICLE command is't send till the header download has finished. (Before the bug occurs, ARTICLE is send in between the header download)

Reasons:
Bugfix (attachment id=402483) of Bug 311774.
Proofed by checkout. (changeset: 4006:b951cf498d4f
<http://hg.mozilla.org/comm-central/rev/b951cf498d4f>
changeset: 3966:a8f2369df790
<http://hg.mozilla.org/comm-central/rev/a8f2369df790>
and changeset: 3897:c41b9cf09178
<http://hg.mozilla.org/comm-central/rev/c41b9cf09178>)
(The ARTICLE behavior changes back to normal)

Workaround:
- Force reload of message (e.g. Menu)
- Clear cache ("Network&Diskspace" – "clear now") and revisit message
- set "browser.cache.memory.capacity" to 0 (reloads automatically)

Note: see also Bug 539194

Comment 5

[Jürgen 'JiHa' Harter]

I also see this on Linux x86_64.
Workarounds as stated in comment 4.

Comment 6

[Joshua Cranmer [:jcranmer]]

Well, that the empty article is cached is nothing new. It does seem that the frequency of empty articles is increased due to longer possible time of interference, though.

All in all, though, it's not really a regression caused by bug 311774. It's just a preexisting problem made more evident.

Comment 7

[Mark Banner (:standard8)]

I've definitely seen these at times.

Whether or not its a new regression I don't know, but I hadn't seen it before we released 3.0.

Comment 8

[Man-wai Chang]

I encountered exactly the same problem! I thought it's a problem for slow 56k dial-up ISP only. Can't believe broadband readers had the same problem!

I support the motive of adding a Reload Message to the right-click context menu!

Both newsgroup (NNTP) and IMAP articles/messages are being cached just like web-pages! So a reload/refresh function is needed!

Also, Thunderbird 3 should report an error rather than displaying a blank artcile if it's a timeout!

Comment 9

[stefan.blumenrath]

What about a workaround like:
if ((message==empty) and (online==true) and (not_checking_news_right_now))
{reload message once}

I know it's a little dirty but better than displaying nothing

Comment 10

[Joshua Cranmer [:jcranmer]]

(In reply to comment #9)
> What about a workaround like:
> if ((message==empty) and (online==true) and (not_checking_news_right_now))
> {reload message once}
> 
> I know it's a little dirty but better than displaying nothing

The if statement would be more along the lines of "ignore the cached version if the message is empty" or perhaps "invalidate the cache if the message is empty."

The problems is that "message is empty" may be a bit hard to test for--I need to have this error crop up in one of my debugging sessions to be certain.

Comment 11

[Alfred Peters [:infofrommozilla]]

(In reply to comment #10)
> The problems is that "message is empty" may be a bit hard to test for--I need

Is there really no way to tell TB to wait for the ARTICLE response? Or get notified on arrival to clear the cache and redraw the message then.

If not, what about rising its priority? Or lowering the priority of Bug 311774 processing, like: fetch the headers group by group so that other commands can get through?

If that isn't possible either, it would (IMO) be the best to block any user action during Bug 311774 processing. E.g. show a modal dialog: "Loading group <n> of <m>".

Comment 12

[stefan.blumenrath]

(In reply to comment #10)

> The if statement would be more along the lines of "ignore the cached version if
> the message is empty" or perhaps "invalidate the cache if the message is
> empty."

Does the cached versions of messages really improve the performance, that it's worth to reload one or two times in an hour? (That is /my/ personally rate, when reading news, subscribed about thirty groups, downloading all ten minutes from localhost-server. My actual workaround is reducing the number of subscriptions to ten, the problem still exist, but is maybe one in a day or two)
 
> The problems is that "message is empty" may be a bit hard to test for--I need
> to have this error crop up in one of my debugging sessions to be certain.

subscribe to a few more groups, try some high-traffic-groups, jump from one group to another while downloading should do...

Comment 13

[Dan Mosedale (:dmosedale, :dmose)]

If this were the last bug standing, we wouldn't hold for this, but it would nonetheless be great to get a fix.  Blocking- and wanted+ set.

Comment 14

[stefan.blumenrath]

I just tried to save such an empty posting and it didn't get saved. The save-as-window appeared, i confirmed but nothing was saved. IF there is a check
message is empty -> don't save, this could be the condition jcranmer in comment #10 meant to be 'hard to test for' - Else: another Bug...

Comment 17

[Tikras Vardas]

Still for me this bug is critical as it makes Thunderbird not usable as a news reader application. It is is very annoying when it is impossible to read a reply in a conversation that you see exists.

It should be fairly easy to test such condition because cached empty message also has no headers that can be compared to to the ones from message list.

Comment 18

[David :Bienvenu]

The way this is supposed to work is that mem cache entries are only supposed to be marked "valid" if we successfully completed the fetch of the message. And we're only supposed to use mem cache entries that are marked valid. My guess is that mem cache entries are getting marked valid even when there's an error fetching the message.

Comment 19

[Joshua Cranmer [:jcranmer]]

NNTP cache entry:
MarkValid called:
1. FinishMemCacheEntry(true)
   a. OnStopRequest if aStatus succeeded and last response was a 2xx
2. OnCacheEntryAvailable, w/ read and write access
Doom called:
1. FinishMemCacheEntry(false)
   a. OnStopRequest if aStatus failed or last response was not 2xx
   b. SendFirstNNTPCommandResponse if failed
   c. in the NNTP_ERROR state
   d. Autosubscribe dialog canceled

For comparison, the IMAP cache entry code (which may be more correct):
MarkValid called:
1. Mock channel is closed
2. OnCacheEntryAvailable, w/ read and write access and success of read
Doom called:
1. Cache async notifier comes after channel closure
2. Reading an IMAP part fails
3. Reading from cache fails
4. DeathSignalReceived
5. Mock channel is canceled

I suspect, from comparing with IMAP code, that the OnStopRequest call to FinishMemCacheEntry may not be quite correct. If we haven't gotten a call back from the server yet, the OnStopRequest will be checking the response of the previous command set whose success is independent of the current command. CVS blame indicates that check was made as part of bug 107776... gee, that sounds familiar. :-)

The fix, then, would be to make sure we don't call FinishMemCacheEntry based on the response of the previous received code. Seeing as how Initialize is setting the m_responseCode to 0, it may be worth doing a try build that sets m_responseCode to 0 in SendArticleNumber to see if this is the cause of the incorrect caching.

Either that, or we could move the cache entry dooming/marking available out of OnStopRequest.

Comment 20

[David :Bienvenu]

that sounds right, jcranmer, with the caveat that the IMAP code has similar if not as common issues.

Comment 21

[Joshua Cranmer [:jcranmer]]

In doing the testing, the case I came up with was OnStopRequest being called after m_runningUrl had been set to null (either that, or the cache entry was disassociated from it). So it never was getting to the check to the mark valid or doomed entry.

Right now, I have an aesthetically unpleasing patch that basically stops reading from the cache on an empty article. The aesthetic issue is that this won't stop not-fully-cached articles. Then again, empty articles are what people are seeing, not half-way-displayed articles, so this may be sufficient.

I'm trying to see if I can get some sort of automated testing on the cache working.

Comment 22

[Joshua Cranmer [:jcranmer]]

Attachment: Patch, version 1

Comment 23

[Joshua Cranmer [:jcranmer]]

Attachment: Patch, version 1 (really this time)

Oops, forgot the test.

Comment 24

[David :Bienvenu]

Comment on attachment 434001 [details] [diff] [review]
Patch, version 1 (really this time)

on my windows build, the test still passes without the core code change, so it must not be quite right.

Comment 26

[Man-wai Chang]

> Workaround
> set "browser.cache.memory.capacity" to 0 (reloads automatically)

I found that setting the cache size to 0 might disable Thunderbird-3.x.x's abiliyt to display attachment properly....

Comment 28

[Daniel]

Build identifier: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.9) Gecko/20100317 SeaMonkey/2.0.4

I'm still seeing this, on my Linux build and/or my Windows 7 build of SM, usually with Usenet posts. Hitting Reply brings up a basically empty screen, just a "Fred posted......" in the message window.

Daniel

Comment 29

[Alfred Peters [:infofrommozilla]]

(In reply to comment #23)
> Created an attachment (id=434001) [details]
> Patch, version 1 (really this time)

I've compiled TB with that patch and still get an empty message view during download. What has changed is: Revisiting the article shows the correct view now.
If this behavior is intended, it works for me.

Just a thought: An automatic "ReloadMessage()" after finishing download would do the trick.

Just another thought: If viewing a Message isn't possible it brings me back to my suggestion of #11: Block the viewing of messages during download.

Comment 30

[Phillip M. Jones, C.E.T.]

This also applies to SeaMonkey as well and I am on Mac with OSX.4.11. I am using SM 2.0.4.

In my case I have set to automatically download headers and when I go into them in turn its almost always the first message in list, but not always. And I notice also if instead of a full highlight of the header (subject) if it shows as  a hollow outline is guaranteed not to show. The only way to see the defect messages is to mark as unread. Shut down SeaMonkey, wait a few Seconds, Restart SeaMonkey  go to new item in question and will show this time.

So this bug is pervasive in Thunderbird and SeaMonkey as well.

In my case there are not many days that his doesn't happen to at  least one message in any group. It's almost a daily occurrence  for me. So this bug need to apply to SeaMonkey as well as Thunderbird. And affects all platforms not just windows.

Comment 31

[u331436]

a workaround that works for me:
Select the news article, right-click "Get selected message", move focus out of the news article then back onto it, and the article shows.

(Using SM2.1a1pre on WinXP)

Comment 32

[Phillip M. Jones, C.E.T.]

I've tried similar on Mac. Doesn't help. Must mark as un read the shut down SM then open a nd go to it.

Comment 33

[Robert Kaiser]

It's already marked wanted bug we won't block releases on it.

Comment 34

[Arthur Marsh]

I've been reading the gmane.linux.debian.user.kde
 newsgroup via news.gmane.org and a locally running news server using leafnode.

If I attempt to read the newsgroup after just starting thunderbird/icedove (Mozilla/5.0 (X11; U; Linux i686; en-AU; rv:1.9.1.9) Gecko/20100515 Icedove/3.0.4 ) starts up, I can read between 5 and 10 messages before hitting this problem.

By leaving thunderbird/icedove several minutes after start-up before reading I was then able to read a large number of messages without problem (e.g. 50-100).

However, if I restart thunderbird/icedove and start reading again straight away I will run into this problem again a few minutes later.

Also, with 50-60 newsgroups defined, reading of the next message can block for half a minute or so while the newsgroups are scanned for new messages.

For me, this is annoying enough to want to downgrade icedove/thunderbird.

Do the thunderbird/icedove developers read mailing lists as newsgroups on gmane.org using thunderbird/icedove?

This feels like a functionality (read newsgroup messages reliably) is being lost as "we won't block releases on it". )-:

Comment 35

[Phillip M. Jones, C.E.T.]

Added comment: this applies to any news server including Mozilla.org, Annexcafe.com, secnewsnetescape.com,  Msnews.Microsoft.com, and Eternal_September.com (USENET).  doesn't settle down to any one news server.

Comment 36

[Joshua Cranmer [:jcranmer]]

(In reply to comment #34)
> This feels like a functionality (read newsgroup messages reliably) is being
> lost as "we won't block releases on it". )-:

The definition of blocking a release means that we would rather not release (i.e., delay the release indefinitely) than release a version with this bug unfixed. That is a pretty high bar, and one that this bug does not fulfill. On the other hand, it is a wanted bug.

In any case, the fix is probably known, but the test I added did not check the condition I thought it did. I have not had time to debug the test to figure out why, though.

Comment 37

[Alfred Peters [:infofrommozilla]]

(In reply to comment #36)

> In any case, the fix is probably known, but the test I added did not check the
> condition I thought it did.

Sure, your Patch can't display any data that is not available, but it does prevent TB from thinking this empty cache-entry is the article.
So far it works pretty fine and IMHO it should be checked in.

>  I have not had time to debug the test to figure out
> why, though.

Or do I misunderstand something?

Comment 38

[Arthur Marsh]

By going to Account Settings, Server Settings for my NNTP host and un-checking "Check for new messages every ... minutes" I have be able to avoid this problem of being unable to read some newsgroup articles completely and also avoid the problem of memory usage increasing over time.

When I want to check for new newsgroup articles I collapse and re-expand the listing for my NNTP host. This may actually consume some more RAM but it happens so rarely in comparison to automatic checking few minutes that it is not noticable.

Comment 39

[Phillip M. Jones, C.E.T.]

I have had that turned off and still have the problem

Comment 40

[Christoph Lipka]

Any update on this? This bug keeps bugging me literally every day, and anything that makes life easier in this respect would be greatly appreciated; having to re-select the message, or even clicking on a context menu entry to force a re-load of the message body - anything along these lines would do the job for me.

Comment 41

[Mike Lykov]

I try in with Mozilla/5.0 (X11; U; Linux i686; ru; rv:1.9.2.9) Gecko/20100825 Thunderbird/3.1.3 release (this bug not in changelog)

when browser.cache.memory.capacity is set to 0, and i try to see article with joined picture - text is shown, picture not shown.
When i restore browser.cache.memory.capacity = 4096 (as by default) - pictures are showing in the articles below text.

So i hope for fix in next release, without need to disable memory cache.

Comment 44

[Joshua Cranmer [:jcranmer]]

Attachment: Patch, version 1.1

This time, the tests should be correct!

Comment 45

[Arthur Marsh]

Hi, I tried the patch and it did enable me to successfully re-attempt to read a newsgroup message that was somehow unsuccessfully read the first time.

However, virtual memory went from around 282 MiB shortly after start-up to over 613 MiB after about 12 hours with checking for new newsgroup messages every 5 minutes. I have since disabled checking for new newsgroup messages and will only do so manually like I have been doing for the last several months.

Comment 46

[David :Bienvenu]

Comment on attachment 490309 [details] [diff] [review]
Patch, version 1.1

looks good, thx.

Comment 47

[Joshua Cranmer [:jcranmer]]

Pushed as changeset 1dcc58a75dbb.

Comment 48

[Alfred Peters [:infofrommozilla]]

"RESOLVED FIXED"? I don't think so.
Attachment 490309 [details] [diff] just fixes a symptom of this Bug. But I stil get empty Artikles while TB downloads the Headers. :-(

Comment 49

[foxi]

I just had this occur to me after installing TB V5.0 beta 1

Windows XP 32 bit SP3

All newsgroup posts are blank, they remain blank, restarting TB does not help.

Restarting in TB safe mode does not help, disabling addons.

It seems that no new newsgroup posts are being downloaded - TB displays a 'connecting' message and nothing else occurs.

Will this bug be reopened by people posting or do I need to post a new bug?

Thanks

Comment 50

[foxi]

(In reply to comment #49)
> I just had this occur to me after installing TB V5.0 beta 1
> 
> Windows XP 32 bit SP3
> 
> All newsgroup posts are blank, they remain blank, restarting TB does not
> help.

I should add that all existing newsgroup posts became blank - those that are set to remain on my computer (and set to be purged in 30 days).

Comment 51

[foxi]

Excuse me if I don't fully understand how bugzilla works: Somebody asked by email "Did you try rebuilding the indexes on your newsgroups ?"

No I didn't.  I deleted the Thunderbird program folder and reinstalled V3.xx (lastest stable build) with no change.  I then reinstalled V5.0 beta 1 with no change.

I then rebooted XP and it was back to normal, and has remained normal since then.

ta.