1766400 - [Prod] The Nimbus-Preview collection does not work anymore as we are unable to enroll in Preview experiments

Status: VERIFIED FIXED

(Firefox :: Remote Settings Client, defect, P1)

Description

[Carmen Fat [:cfat] - Ecosystem QA]

[Notes]:

• This is reproducible with both the PRODUCTION and STAGE PREVIEW.
• Besides force enrollment, we have also tried enrolling in the experiment by generating user IDs for each experiment branch. However, we did not manage to get enrolled in the experiment this way either.
• The following errors are thrown in the Browser Console:
  [Exception... "Component returned failure code: 0x80004001 (NS_ERROR_NOT_IMPLEMENTED) [nsIAppStartup.secondsSinceLastOSRestart]" nsresult: "0x80004001 (NS_ERROR_NOT_IMPLEMENTED)" location: "JS frame :: resource:///modules/BrowserGlue.jsm :: _collectStartupConditionsTelemetry :: line 1570" data: no] BrowserGlue.jsm:1570:9
services.settings: main-preview/nimbus-preview Signature failed InvalidSignatureError: Invalid content signature (main-preview/nimbus-preview) RemoteSettingsClient.jsm:965
services.settings: main-preview/nimbus-preview local data was corrupted RemoteSettingsClient.jsm:1000
services.settings: main-preview/nimbus-preview Signature verified failed. Retry from scratch RemoteSettingsClient.jsm:674
services.settings: main-preview/nimbus-preview Signature failed again InvalidSignatureError: Invalid content signature (main-preview/nimbus-preview) RemoteSettingsClient.jsm:965
Invalid content signature (main-preview/nimbus-preview) RemoteSettingsClient.jsm:135

[Affected versions]:

• Firefox RC 100, Build ID: 20220425210429
• Firefox Beta 100.0b, Build ID: 20220421201645

[Affected platforms]:

• Windows 10 x64
• macOS 11.6.5
• Ubuntu 20.04 x64

[Prerequisites]:

• Have a new Nimbus experiment created and launched to preview.
• Have the nimbus.debug pref set to true.
• Change messaging-system.rsexperimentloader.collection_id to nimbus-preview value.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Paste the force enrollment link from the recipe in a new tab.
3. Press the “Enter” key and observe what happens next.

[Expected result]:

• You are enrolled in the experiment.

[Actual result]:

• The Could not find experiment slug <slug name> in collection nimbus-preview error is displayed in the “about:studies” page.

[Additional notes]:

• Here is a screen recording with the issue.

Comment 1

[Carmen Fat [:cfat] - Ecosystem QA]

@Mathieu, can you please take a look at this issue? This is currently blocking QA from testing requests that are in Preview mode. Thank you.

Comment 2

[Mathieu Leplatre [:leplatrem]]

I can indeed reproduce your client errors.

The signature are valid when verified using Python (server) code.

Let me investigate 🙏

Comment 3

[Mathieu Leplatre [:leplatrem]]

I found the issue.

In your data, you have float values. The server should not have let you store them, we will look at that.

In the mean time, make sure you change the lineHeight values to String instead of float.

For example in https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/nimbus-preview/records/covered-regular-eco2204 you have lineHeight: 1.25

Comment 4

[Ciprian Muresan [:cmuresan], Formerly of Ecosystem QA - please needinfo @mcoman for things regarding the team]

@leplatrem thanks so much for the quick check!

I've edited the Staged experiment that we have to use strings and the preview collection works now. I've also removed the Prod experiment from the preview collection so that will sort itself out now. I'll also bring this up with Jared to see if Nimbus can do something to not reach this state in the future.

Marking this as Resoved - Fixed based on comment 3 and this comment.

@ed, if we use string for the lineHeight property will that affect the message that we're trying to send with the Eco experiment?

Comment 5

[Ed Lee :Mardak]

lineHeight value as a string happens to behave the same. I've updated the prod experiment branch values to quote the float values.

Comment 6

[Carmen Fat [:cfat] - Ecosystem QA]

Marking this issue as Verified since it was all cleared up in comment 3 and we did not encounter any other issues using the Nimbus-Preview collection since then.