Improve Pref Experiment Workflow
Categories
(Firefox for Android :: Experimentation and Telemetry, task)
Tracking
()
People
(Reporter: beth, Unassigned)
References
Details
Right now to run a pref setting experiment on Fenix, one needs to do the following:
- Add the feature to the Fenix Nimbus manifest
- Add a pref instance to GeckoRuntimeSettings
- Add a getter and setter for that member
- Add a member to the GeckoEngine settings object in android-components and assign it in the
.applyblock - Update the DefaultSettings ctor to set this new member based on a value from FxNimbus
On Desktop, we annotate the feature manifest with what prefs each variable will set, which is much simpler to (a) implement and (b) remove when your experimentation is complete.
I don't think this doesn't belong in the Nimbus SDK bug tracker. Any solution here will likely have to be in client code
|
||
Comment 1•1 year ago
•
|
||
Sorry, I completely forgot that there is an alternative route for engine experiments. There is an ExperimentDelegate on the GeckoRuntime, that can fetch the experiment data from the embedding application.
This might reduce the steps needed for running the kinds of experiments that you want, but this issue is probably still valid to see if this option can also be improved.
|
Reporter | |
Comment 2•1 year ago
|
||
Yeah that could likely be used to set prefs in a data driven manner like how its done for Desktop
|
||
Comment 4•1 year ago
|
||
The Bugbug bot thinks this bug should belong to the 'Fenix::Experimentation and Telemetry' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.
|
||
Comment 5•1 year ago
|
||
I'm going to make some gecko preferences, currently exposed as Nimbus features, accessible for pref flip experiments in Fenix.
There are about a half dozen preferences in the networking and security components.
I don't need exposure counting, just the ability to have the pref's set via experiment enrollment.
I'm wondering what's a better route -- the steps Beth described in comment 0, which seem straightforward.
Or via the ExperimentDelegate which Olivia added in bug 1845824. I don't fully understand how it binds prefs but it seems interesting.
Olivia -- do you have any thoughts on this?
|
|
Reporter | |
Comment 6•1 year ago
|
||
(In reply to Andrew Creskey [:acreskey] from comment #5)
I'm going to make some gecko preferences, currently exposed as Nimbus features, accessible for pref flip experiments in Fenix.
There are about a half dozen preferences in the networking and security components.
I don't need exposure counting, just the ability to have the pref's set via experiment enrollment.
I'm wondering what's a better route -- the steps Beth described in comment 0, which seem straightforward.
Or via the ExperimentDelegate which Olivia added in bug 1845824. I don't fully understand how it binds prefs but it seems interesting.
Olivia -- do you have any thoughts on this?
The ExperimentDelegate does not do that for you automatically. It lets you query feature values at runtime, so you could set prefs yourself with whatever API calls are available.
|
|
||
Comment 7•1 year ago
|
||
(In reply to Beth Rennie [:beth] (she/her) from comment #6)
(In reply to Andrew Creskey [:acreskey] from comment #5)
I'm going to make some gecko preferences, currently exposed as Nimbus features, accessible for pref flip experiments in Fenix.
There are about a half dozen preferences in the networking and security components.
I don't need exposure counting, just the ability to have the pref's set via experiment enrollment.
I'm wondering what's a better route -- the steps Beth described in comment 0, which seem straightforward.
Or via the ExperimentDelegate which Olivia added in bug 1845824. I don't fully understand how it binds prefs but it seems interesting.
Olivia -- do you have any thoughts on this?
The ExperimentDelegate does not do that for you automatically. It lets you query feature values at runtime, so you could set prefs yourself with whatever API calls are available.
Ah, I see.
Let me go through the Fenix Feature Manifest to improve my understanding of this workflow.
|
|
||
Comment 8•1 year ago
|
||
FYI, my first attempt at getting a pref-setting Nimbus experiment setup on Fenix, bug 1925390.
|
||
Comment 9•1 year ago
|
||
The ExperimentDelegate does not do that for you automatically. It lets you query feature values at runtime, so you could set prefs yourself with whatever API calls are available.
Yes, exactly this! The experiment delegate is a piece of the puzzle of improving this flow, but nothing automatic.
I have a WIP proposal trying to work out what Android should be doing with Gecko preferences, especially with respect to Nimbus here, please feel free to leave comments / concerns / ideas. The meta ticket is bug 1908182.
|
|
||
Comment 10•1 year ago
|
||
(In reply to Olivia Hall [:olivia] from comment #9)
The ExperimentDelegate does not do that for you automatically. It lets you query feature values at runtime, so you could set prefs yourself with whatever API calls are available.
Yes, exactly this! The experiment delegate is a piece of the puzzle of improving this flow, but nothing automatic.
I have a WIP proposal trying to work out what Android should be doing with Gecko preferences, especially with respect to Nimbus here, please feel free to leave comments / concerns / ideas. The meta ticket is bug 1908182.
Thank you, Olivia. I don't have anything to add to the proposal itself other than to say that this work would be deeply appreciated by the performance, networking, and many other gecko component teams.
Description
•