Closed Bug 1656090 Opened 3 years ago Closed 3 years ago

[TB 78 regression] Feed reader doesn't show similar but different entries


(MailNews Core :: Feed Reader, defect)



(thunderbird_esr78+ fixed, thunderbird80 fixed)

81 Branch
Tracking Status
thunderbird_esr78 + fixed
thunderbird80 --- fixed


(Reporter: agentmc, Assigned: benc)



(Keywords: regression)


(1 file, 1 obsolete file)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.108 Safari/537.36

Steps to reproduce:

I have this RSS:
Import it and compare the Thunderbird view to readout of any public RSS reader, e.g.

Actual results:

In version 69 and below it was shown as expected. In version 78, roughly half of the messages are lost.
What is going on is that is a torrent feed. Torrents on this feed are created in 2 forms: [1080p] and [1080p HEVC]. The majority of the content for both forms is identical, but see Expected Results for what is different.
It looks like TB 78 only accepts ONE of two messages, apparently thinking "they are similar - so they are the same".

Expected results:

All the messages must be shown.
The behavior above shouldn't be the case as:

  1. all the messages on my feed have a <guid> entry, and the guids for these two items ([1080p] and [1080p HEVC]) will be different because ultimately on the 3rd-party resource it's different torrent ids and the guid is directly embedding those torrent ids. Because of this (and because it's a Primary Key on the table) there is 0 possibility that a duplicate guid will exist.
  2. both the title and the body of these messages is different (though the difference is minor).
Severity: -- → S2
OS: Unspecified → Windows 10
Hardware: Unspecified → x86_64
Summary: Feed reader doesn't show similar but different entries → [TB 78 regression] Feed reader doesn't show similar but different entries
Component: Untriaged → Feed Reader
Keywords: regression
Product: Thunderbird → MailNews Core

Ben, please have a look. From the de-RDF?

Flags: needinfo?(benc)
Blocks: tb78found

Yes, my fault, from the RDF-removal. The item URL is always used as the key in the feeditems.json, but some feeds should use a UUID instead. Doh.

This patch should fix it, and adds a test case.

There is the issue that this will cause duplicate items to show up for people who already have a feeditems.json using the url as key (and with feeds that use non-url guids)... I'm not sure how much of an issue this out in the wild. Maybe an uplift into 78 would head off most of the damage?

Assignee: nobody → benc
Flags: needinfo?(benc)
Attachment #9170265 - Flags: review?(mkmelin+mozilla)
Comment on attachment 9170265 [details] [diff] [review]

Review of attachment 9170265 [details] [diff] [review]:

LGTM. Agreed uplifting will cause duplicates for some cases, but should cause least amount of damage in total

::: mailnews/extensions/newsblog/test/unit/resources/
@@ +15,5 @@
>     whitespace.
> +- `rss2_guid.xml`
> +   RSS2.0 feed where two items have the same link but different guid.
> +   (they should be both appear in the feed).

remove "be" I guess
Attachment #9170265 - Flags: review?(mkmelin+mozilla) → review+
Ever confirmed: true
OS: Windows 10 → All
Hardware: x86_64 → All
Attachment #9170265 - Attachment is obsolete: true
Attachment #9170561 - Flags: review+

Comment on attachment 9170561 [details] [diff] [review]

[Approval Request Comment]
Regression caused by:
Occurred during the RDF removal (Bug 1534163).
User impact if declined:
Some RSS feeds will be missing items, as described in this bug (Bug 1656090).
Testing completed (on c-c, etc.):
Minor change, minimal testing (new unit test, plus superficial "it works for me" testing :- ).
Risk to taking this patch (and alternatives if risky):
Low risk, fixes an obvious regression.

Attachment #9170561 - Flags: approval-comm-beta?
Target Milestone: --- → 81 Branch
Attachment #9170561 - Flags: approval-comm-esr78?

Comment on attachment 9170561 [details] [diff] [review]

[Triage Comment]
Approved for beta
Approved for er78

Attachment #9170561 - Flags: approval-comm-esr78?
Attachment #9170561 - Flags: approval-comm-esr78+
Attachment #9170561 - Flags: approval-comm-beta?
Attachment #9170561 - Flags: approval-comm-beta+

Pushed by
Use instead of item.url as key in feeditems.json. r=mkmelin

Closed: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.