1944609 - Add missing transforms and supported fields in JEXL filters

Status: RESOLVED FIXED

(Application Services :: Remote Settings, enhancement)

Description

[Mathieu Leplatre [:leplatrem]]

Now that we have two official implementations of JEXL filters, we need to clarify exactly what is supported across both implementations.

Editors need to be confident that targeting fields are supported transparently on each platform.

Currently, for example, only the versionCompare transform is supported on the Rust client
https://github.com/mozilla/application-services/blob/39404c8d2a284aadde38a0e638a4d7b557aa55a8/components/remote_settings/src/jexl_filter.rs#L38

And in terms of JEXL app context, the fields in the Rust RemoteSettingsContext don't match the ones in desktop ClientEnvironment
https://github.com/mozilla/application-services/blob/39404c8d2a284aadde38a0e638a4d7b557aa55a8/components/remote_settings/src/lib.rs#L44-L78
https://searchfox.org/mozilla-central/rev/a965e3c683ecc035dee1de72bd33a8d91b1203ed/toolkit/components/utils/ClientEnvironment.sys.mjs#31-264

Missing in RemoteSettingsContext:

• distribution – Retrieves the application distribution ID.
• telemetry – Retrieves the most recent telemetry pings.
• liveTelemetry – Proxy for accessing only the main telemetry ping.
• randomizationId – A unique user identifier (intended to be replaced by client_id in the future).
• isDefaultBrowser – Checks whether the application is the default browser.
• searchEngine – Retrieves the default search engine’s telemetry ID.
• syncSetup – Checks if the sync feature is set up.
• syncDesktopDevices – Gets the number of synced desktop devices.
• syncMobileDevices – Gets the number of synced mobile devices.
• syncTotalDevices – Computes the total number of synced devices.
• addons – Retrieves a list of installed add-ons.
• plugins – Retrieves a list of installed plugins.
• doNotTrack – Checks whether the Do Not Track (DNT) setting is enabled.
• os.windowsVersion – Retrieves the Windows version (only on Windows).
• os.windowsBuildNumber – Retrieves the Windows build number.
• os.macVersion – Computes the macOS version based on Darwin versioning.
• os.darwinVersion – Retrieves the Darwin version (only on macOS).
• attribution – Retrieves attribution data.
• appinfo – Retrieves application and platform information.

This prevents editor to safely target clients across implementations.

In practice, they may not be all relevant, and maybe we could clean this up a bit (and thus remove them from the documentation). Mark, do you have some inputs on this?

I wrote a little script to list what is currently being used, and it's very limited:

Fields used:
['appinfo.ID', 'channel', 'version']

Transforms used:
['preferenceValue', 'versionCompare']

Comment 1

[Mark Banner (:standard8)]

Sorry for the delay in responding.

(In reply to Mathieu Leplatre [:leplatrem] from comment #0)

• appinfo – Retrieves application and platform information.

Note that this one has a bunch more possible information as well - all of nsIXULRuntime, nsIXULAppInfo, nsIPlatformInfo and nsICrashReporter

This prevents editor to safely target clients across implementations.

In practice, they may not be all relevant, and maybe we could clean this up a bit (and thus remove them from the documentation). Mark, do you have some inputs on this?

I think we should definitely have a consistent list between implementations. We should be able to target collections across multiple implementations and have them produce the same records.

I wrote a little script to list what is currently being used, and it's very limited:

Fields used:
['appinfo.ID', 'channel', 'version']

Transforms used:
['preferenceValue', 'versionCompare']

From a search perspective, I think the most important items for us so far has been the version (alongside versionCompare).

The appinfo.ID use (which I think is only search-config-icons), we are going to be removing in the next week or so. However we'd still like to keep the requirement for a platform specific filter.

It looks like translations uses channel, and appinfo.OS (xref).

I'm not sure who uses preferenceValue maybe Normandy? That might be a harder one to support.

I think from a Rust remote settings perspective, based on the list above I think we should support as a minimum:

• appInfo.OS (probably not appInfo.ID)
• version - hopefully we can use the gecko equivalent versions on all platforms.
• channel

With versionCompare and maybe preferenceValue.

We might also want to ask other consumers what they might need, but I think in the first instance it'd be good to get what is currently required implemented, and then work out other parts later.

For the channel, it might also be worth using an enumeration for that, rather than strings. For instance, we already have one in the search code.

I'm not sure if we'd want an enumeration for appInfo.OS as well?

Hopefully this answers the question. As I mentioned in bug 1951084, it'd be useful if we could have the filtering enabled for when this goes out on Mobile (probably sometime in the next sprint or two), if that's possible, since then we don't have to worry about backwards compatibility.

Comment 2

[Mathieu Leplatre [:leplatrem]]

I opened a PR on the docs page, so that users know what to expect: https://github.com/mozilla/remote-settings/pull/811

In the short term:

• env.version
• env.channel
• env.locale
• env.appinfo.OS (missing)
• versionCompare()

And then there will be a mid/long term work of normalization across gecko and A-S. But first, please let stop the bleeding, please do not add new custom fields that would be A-S only.

Adding a new fields should be prettier straightforward, and there should be very few reasons to go platform-specific:

1. Put Remote Settings stakeholders in the loop (would allow to avoid disparities like casing, etc.)
2. Add the field on Gecko in this file (even as constant or empty value)
3. Mention the field in the official JEXL documentation

Comment 3

[BMO Github Automation]

Attachment: [mozilla/application-services] Bug 1944609 - Normalize JEXL context (#6663)

Comment 4

[BMO Github Automation]

Authored by https://github.com/bendk
https://github.com/mozilla/application-services/commit/2f617f50be1addaa0df47ebcdb36a0b74d2ed5af
[main] Bug 1944609 - Normalize JEXL context