1861549 - `nimbus-cli enroll` must be run twice to enroll in an experiment

Status: NEW

(Application Services :: Nimbus SDK, defect)

Description

[Lina Butler [:lina]]

Steps to reproduce

1. Install a simulated Release build of Fenix that includes the fix for bug 1856435 on a device. Please see the additional information below for why I used a simulated Release build.
2. Follow these instructions to enable the "secret settings" menus.
3. Navigate to Settings > Nimbus Experiments.
4. Observe that "Enhanced Cross-Platform Suggest [Android]" is not listed, as expected.
5. Run the following command from the Terminal on the host computer: nimbus-cli -a fenix -c release enroll preview/cross-platform-enhanced-suggest-android --branch treatment
6. Observe that the command runs successfully, and that Fenix is now closed on the device.
7. Relaunch Fenix.
8. Open Settings > Nimbus Experiments.

Expected behavior

"Enhanced Cross-Platform Suggest [Android]" is listed in the experiments list.

Actual behavior

"Enhanced Cross-Platform Suggest [Android]" is not listed. Instead, the list of experiments is the same as step 4.

Workaround:

9. Run nimbus-cli -a fenix -c release enroll preview/cross-platform-enhanced-suggest-android --branch treatment from the Terminal again.
10. Observe that the command runs successfully, and that Fenix is closed and now re-launched automatically on the device.
11. Open Settings > Nimbus Experiments.
12. Observe that the experiments from step 4 have been removed, and that "Enhanced Cross-Platform Suggest [Android]" is now listed.

Here's a screen recording of the issue. I ran step 5 at 0:20, and step 9 at 0:48.

Device information

• Firefox version: Simulated Release build from this commit.
• Android device model: Pixel 5
• Android OS version: 14

Any additional information?

Fenix Beta 120.0b3, which has a fix for bug 1860908, wasn't available when I filed this bug, so I used a simulated Release build of Fenix instead, following these instructions. I used the Build Variant Matrix Selector plug-in for Android Studio to select product: fenix and buildType: release.

My local.properties are:

autoPublish.application-services.dir=../../application-services
rust.targets=arm64,darwin-aarch64
autosignReleaseWithDebugKey
debuggable

I'm using the auto-publish flow to build Fenix with a local Application Services, at this commit.

Comment 1

[James Hugman [:jhugman] [@jhugman]]

5. Run the following command from the Terminal on the host computer: nimbus-cli -a fenix -c release enroll preview/cross-platform-enhanced-suggest-android --branch treatment
6. Observe that the command runs successfully, and that Fenix is now closed on the device.
7. Relaunch Fenix.

Hmm, this is strange: I'd expect the nimbus-cli to kill the app then immediately restart it, then enrolling it in the experiment. Are you able to reproduce this bug now? (I'm sorry it's taken so long to get to this!)

Comment 2

[Lina Butler [:lina]]

Oops, I haven't confirmed that this is still an issue yet—restoring my ni? to do that as part of the remote tab management work I'm doing now.