1908802 - Suggestions in deleted records aren't dropped from the store

Status: RESOLVED FIXED

(Application Services :: Suggest, defect, P1)

Description

[Drew Willcoxon :adw]

It appears that suggestions in deleted records aren't dropped from the store. This is only a problem for deleted records whose IDs aren't reused, i.e., records that aren't replaced/updated with a new attachment.

The suggest RS client always specifies a type filter when fetching records, and that seems to be the problem. type is stripped out of deleted records, so deleted records never match.

Compare leaving out the type filter for the Fakespot collection, which contains several records that were deleted today...

curl 'https://remote-settings.mozilla.org/v1/buckets/main/collections/fakespot-suggest-products/records?gt_last_modified=0'

... to including it:

curl 'https://remote-settings.mozilla.org/v1/buckets/main/collections/fakespot-suggest-products/records?gt_last_modified=0&type=fakespot-suggestions'

The first request returns records that look like this:

{"deleted":true,"id":"fakespot-suggestions-01000-02000","last_modified":1721323151304}

Comment 1

[Drew Willcoxon :adw]

Hi Cosmin, I just wanted to make you aware of this bug that affects Fakespot. When we update Fakespot suggestions in remote settings and your client ingests them, old suggestions can remain on your client (in suggest.sqlite in your Firefox profile) and you'll still be able to trigger them even though you shouldn't be able to. Using a new profile is a workaround. Comment 0 has details.

Comment 2

[Cosmin Muntean [:cmuntean], Ecosystem QA]

Thanks, Drew for the heads up!

One question regarding this, not sure if it's possible but are these suggestions synced if we are using an FXA account? Should we try to verify this?

Comment 3

[Ben Dean-Kawamura [:bdk]]

Suggestions are not synced using FxA. Each client downloads and maintains the list of Suggestions separately.

Comment 4

[BMO Github Automation]

Attachment: [mozilla/application-services] Bug 1908802 - Handle deleted suggest records (#6334)

Comment 5

[Drew Willcoxon :adw]

This was vendored in in bug 1913183.

Comment 6

[Drew Willcoxon :adw]

This was uplifted to 130 in bug 1913488.

Comment 7

[Drew Willcoxon :adw]

Oops sorry, bug 1913488 specifically excluded this bug as part of the uplift.

Comment 8

[Ben Dean-Kawamura [:bdk]]

This shipped in Firefox 131. Users on previous versions will see this bug if records are deleted. Avoid deleting records until v131 is in widespread use.

Comment 9

[Ben Dean-Kawamura [:bdk]]

I also just realized that this is worse than we thought because of the name of the last record. For Fakespot right now, it's fakespot-suggestions-9000-9996. If that changes then we'll get this bug for the items in the last record. This means we need to always upload exactly 9966 records until 131 is out and enough users have upgraded.