Open Bug 1780096 (fxms-mega-schema) Opened 4 months ago Updated 2 days ago

[meta] FxMS Schema Generation

Categories

(Firefox :: Messaging System, task, P1)

task

Tracking

()

People

(Reporter: barret, Assigned: barret)

References

(Depends on 3 open bugs, Blocks 1 open bug)

Details

(Keywords: meta)

The Firefox Messaging System supports several different message templates:

  • CFRs
  • Doorhangers
  • Infobars
  • Spotlights
  • Toolbar Badge Icons
  • Update Actions
  • Whats New Page
  • Protections Panel Messages
  • Private Browsing Promos

Each of these templates corresponds to a JSON schema, which defines what is and is not acceptable for the message template. Each of these templates share a common structure, with field definitions re-used across each template. Because these are separate files, this amounts to copy-pasting field definitions between them. That means they must manually be kept up to date.

Additionally, the Firefox Messaging System is connected to Nimbus, which allows for experiments containing FxMS messages to be run. The Firefox Nimbus desktop client performs schema validation on schemas provided for each feature. There are several features supported by the Firefox Messaging System, each of which can accept any valid template. Nimbus only supports a single schema per feature, so that means we need to combine the schemas for each template into a single schema (henceforth known as the FxMS Mega Schema).

As of bug 1771073, this is the status quo: we have a script in tree that generates the FxMS Mega Schema and a test that guarantees that the generated schema is up to date. Whenever a commit changes the template schemas for any of the above messages, the test will re-generate mega schema and sure that it matches the committed version.

Each FxMS message looks something like this:

{
  "id": "A-UNIQUE-ID",
  "template": "spotlight",
  "targeting": "firefoxVersion >= 104 && firefoxVersion <= 106",
  "groups": ["cfr"],
  "trigger": {
    "id": "defaultBrowserCheck"
  },
  "frequency": "lifetime",
  "priority": 1,
  "content": {
    /* Actual message content goes here. */
  }
}

The majority of these fields are shared between multiple message templates. Some fields (id, template, groups, frequency, and priority) are shared between all templates. This leads to a lot of duplication between schemas. (Spoiler: the result of applying all the refactorings described in this patch series result in a ~500 line decrease in MessagingExperiments.schema.json.)

If these shared field definitions are separated from the messages themselves, then it will be easier to:

  • keep the shared definitions (especially complicated ones like triggers) up to date across messages -- it will happen automatically and be ensured by tests;
  • add new message template that re-use these definitions (see-also bug 1775128, which implements a new message template); and
  • maintain and evolve existing messages, as we now only need to add the fields we care about.

Additionally, there are capabilities shared between several message templates, such as support for L10n and triggering actions. Currently any message that supports L10n must re-define the structure, which looks like the following

{
  "oneOf": [
    { 
      "type": "string",
      "description": "A literal string to show to the user"
    },
    {
      "type": "object",
      "properties": {
        "string_id": {
          "type": "string",
          "description": "The name of a Fluent string ID to show to the user"
        }
      }
    }
  ]
}

at each instance where it is used (most common) or as a top-level definition in the file (less common). If we were to refactor this out into a common schema shared between all templates, we could then use something like:

{ "$ref": "file:///FxMSCommon.schema.json#/$defs/localizableText" }

This will allow us to re-use concepts across all our message templates, making them simpler and more modular.

Finally, due to limitations and bugs in the Experimenter, we need to patch our generated schema files because Experimenter does not support self-references (e.g., {"$ref": "#/$defs/Foo"} which refers to aa definition in the same schema file) and instead only supports references that use full URIs. As of bug 1778368, our schema bundling script handles this and there are integration tests to ensure that Experimenter can load our generated schemas. (More integration tests are coming as part of bug 1779950, which ensures we run our integration tests against all in-tree messages, and bug 1775849, which integrates the messages from messaging-system-inflight-assets (i.e., the non-experiment messages deployed via Remote Settings).)

In the case of generating invalid schemas, we may want to investigate the case of remotely disabling schema validation in Nimbus -- possibly through a pref. (See-also bug 1780088.)

Depends on: 1802351
You need to log in before you can comment on or make changes to this bug.