1804480 - Add support for multi-message treatment branches in desktop experiments

Status: VERIFIED FIXED

(Firefox :: Messaging System, enhancement, P2)

Description

[Meg Viar [:mviar]]

Two recent use cases arose for multi-message treatment branches in desktop experiments.

One was the PDF.js feature tour experiment (see bug 1793636). For this experiment we want to deliver multiple message per treatment branch, with the message targeting controlling the order in which the messages are seen by users. We have a workaround that involves landing the messages in MC ahead of the experiment, but may want more flexibility to experiment without on-train changes in the future.

The second is for a localized experiment (see bug 1804084). Our current approach is to create a different version of the experiment for each locale, adding the localized strings for each manually. Ideally, we could have multiple message per branch with message targeting controlling which of those messages the user sees based on their locale. That capability is already implemented on mobile (see docs here). This documentation outlines the current approach on desktop, which is under active development. This spec outlines the Nimbus localization strategy. This is a more robust solution that would allow experiments to use string ids for translations in all locales. The multi-message approach may still have value if/when the capabilities in the spec are realized. For example, we may have experiments where we want to quickly turn around translations for a smaller subset of locales and target these locales on a per-message basis.

This bug covers investigation of what would be required to support multiple messages in desktop experiments and implementation if applicable. This will likely include a modified version of MessagingExperiments.schema.json that takes an array or an object with messages ids as keys.

Comment 1

[Ed Lee :Mardak]

This would likely help for multiple rollouts currently needing dedicated feature ids like bug 1795349 and bug 1809331. Potentially with experimenter nimbus UI allowing editing of live rollouts and publishing "dirty" changes with https://github.com/mozilla/experimenter/issues/7823, we could edit/add/remove rollout messages with a single feature id as each message in the array will have its own message id, targeting, trigger, etc.

Comment 2

[Shane Hughes [:aminomancer]]

I think we don't want to use a plain array in place of a message definition object, because the local message corpus validation behavior should mirror the experiment message validation behavior as closely as possible, and the local message providers are top-level arrays, which apparently causes next-level arrays to be decoded as dictionaries (keyed by insertion index).

Edit: Ignore the above, this was actually just caused by translation behavior that I missed, which I will fix. So I will go with plain arrays after all.

Comment 3

[Shane Hughes [:aminomancer]]

Some off-train changes are needed to support passing arrays for branch configs: mozilla/nimbus-shared/pull/228

I changed the format to this:

{
  "template": "multi",
  "messages": [{ "id": "msg_1" }, { "id": "msg_2" }]
}

which means these configs will now validate against the existing NimbusExperiment schema (it accepts any contents, just needs to be an object). So we don't need any off-train changes anymore.

Comment 4

[Shane Hughes [:aminomancer]]

Attachment: Bug 1804480 - Support multi-message treatment branches in desktop experiments. r=#omc-reviewers

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ebff0ca66479
Support multi-message treatment branches in desktop experiments. r=omc-reviewers,barret

Comment 6

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/ebff0ca66479

Comment 7

[Simona Rosu [:srosu], Ecosystem QA]

I‘ve verified this enhancement using the latest Firefox Nightly 112.0a1 (Build ID: 20230305214646) on Windows 10 x64, macOS 11.7.1, and Ubuntu 22.04 x64.

• After following the testing steps from this Test Plan, I can confirm the following:
  • A spotlight with the first message appears after enrolling in the experiment, and opening and closing a new tab.
  • A spotlight with the second message appears after dismissing the first one, and reopening and closing another new tab.

Comment 8

[Shane Hughes [:aminomancer]]

Thanks for verifying! Good to know the test plan works.