1861540 - Implement a remote settings server for tests and use it to test the Suggest Rust component and desktop remote settings client

Status: RESOLVED FIXED

(Firefox :: Address Bar, task, P1)

Description

[Drew Willcoxon :adw]

Desktop currently mocks RustSuggest during tests, so we're not testing the real Rust component at all. That also means we have to reimplement essentially all functionality in our mock implementation so that it behaves like the real thing, which is getting more annoying to do the more the Rust component is fleshed out.

We also mock RemoteSettings for the JS backend, which has similar but less extreme drawbacks.

Instead, I'd like to use a local remote settings server during tests so we can transparently test both Rust and the desktop remote settings client.

Comment 1

[Drew Willcoxon :adw]

Attachment: Bug 1861540 - Part 1: Implement a remote settings server for tests.

This implements a remote settings server for tests. Unfortunately there doesn't
seem to be a general-purpose implementation already in the tree.

In part 2, I'll update tests so they use it.

Comment 2

[Drew Willcoxon :adw]

Attachment: Bug 1861540 - Part 2: Update tests to use the local remote settings server.

This modifies tests to use the server. There are a few important points to call
out:

This means tests are now using the real Rust component, and we need to make sure
the test RS data is valid and matches what Rust expects. For example, I had to
add icon properties to suggestions and set the advertiser to "Wikipedia" for
non-sponsored suggestions. Otherwise Rust hits an error on ingest. I also
removed some test cases because they tested behaviors that are impossible with
Rust, for example Pocket keywords that are duplicated in the high- and
low-confidence arrays.

We need to be careful to wait until Suggest is done syncing from remote settings
regardless of whether it's using the JS or Rust backend. I added a way to force
the backends to sync. That way, tests can force a sync, wait for it to finish,
and be sure that all sync activity is done.

A common pattern in tests is to call ensureQuickSuggestInit() and then set
Suggest-related prefs (or vice versa). This is a little problematic because both
ensureQuickSuggestInit() and setting prefs can cause Suggest to start a sync.
It's more problematic now that we're not mocking remote settings or Rust. So I
combined the two by adding a prefs param to ensureQuickSuggestInit(). That
way, tests can be sure that all syncing is done once that function returns.

Depends on D192037

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b67e855993cb
Part 1: Implement a remote settings server for tests. r=daisuke
https://hg.mozilla.org/integration/autoland/rev/2a0da4c5f326
Part 2: Update tests to use the local remote settings server. r=daisuke

Comment 4

[Cristian Tuns]

Backed out for causing xpcshell failures in test_quicksuggest_addons.js

• Backout link
• Push with failures
• Failure Log
• Failure line: TEST-UNEXPECTED-FAIL | browser/components/urlbar/tests/quicksuggest/unit/test_quicksuggest_addons.js | remoteSettings_rustEnabled - [remoteSettings_rustEnabled : 1] Found the expected number of results. - 1 == 0

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e61e99ad721f
Part 1: Implement a remote settings server for tests. r=daisuke
https://hg.mozilla.org/integration/autoland/rev/a14048d57f78
Part 2: Update tests to use the local remote settings server. r=daisuke

Comment 6

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/e61e99ad721f
https://hg.mozilla.org/mozilla-central/rev/a14048d57f78