Closed Bug 1624309 Opened 1 year ago Closed 1 year ago

Add persistent storage for ExperimentStore

Categories

(Firefox :: Messaging System, enhancement, P1)

enhancement

Tracking

()

RESOLVED FIXED
Firefox 77
Tracking Status
firefox76 + fixed
firefox77 + fixed

People

(Reporter: k88hudson, Assigned: andreio)

References

Details

Attachments

(1 file)

We need a persistent storage mechanism for ExperimentStore (used by ExperimentManager) that will serve the needs of messaging experiments.

  • Should store Experiments by identifier (slug) as well as some general metadata
  • Choose an appropriate persistent storage mechanism (focus on reliability): JSONFile/indexedDB/RKV/prefs?
  • Figure out a solution for startup-critical experiments, which are a small subset but may need to know their experiment state synchronously
Assignee: nobody → andrei.br92
Priority: -- → P1
Depends on: 1624320
Pushed by aoprea@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d64c9eb2f326
Add persistent storage for ExperimentStore r=k88hudson

Backed out changeset d64c9eb2f326 (Bug 1624309) for causing brwoser-chrome failures at toolkit/components/normandy/test/browser/browser_actions_MessagingExperimentAction.js

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&revision=d64c9eb2f326524738c7889c1bc953455b6921a5

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=298067062&repo=autoland&lineNumber=6983

Backout link: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&revision=54789c4c8fcbfd4f1fc4827ae2c10791815ad39a

[task 2020-04-17T09:54:11.487Z] 09:54:11     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_MessagingExperimentAction.js | should validate arguments - 
[task 2020-04-17T09:54:11.488Z] 09:54:11     INFO - Buffered messages finished
[task 2020-04-17T09:54:11.488Z] 09:54:11     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/normandy/test/browser/browser_actions_MessagingExperimentAction.js | [[{"id":1,"arguments":{"slug":"foo","branches":[{"slug":"control","ratio":1,"groups":["green"],"value":{}},{"slug":"variant","ratio":1,"groups":["green"],"value":{}}]}},"apply_error"]] deepEqual [[{"id":1,"arguments":{"slug":"foo","branches":[{"slug":"control","ratio":1,"groups":["green"],"value":{}},{"slug":"variant","ratio":1,"groups":["green"],"value":{}}]}},"success"]] - JS frame :: chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/browser_actions_MessagingExperimentAction.js :: arguments_are_validated :: line 56
[task 2020-04-17T09:54:11.488Z] 09:54:11     INFO - Stack trace:
[task 2020-04-17T09:54:11.488Z] 09:54:11     INFO - chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/browser_actions_MessagingExperimentAction.js:arguments_are_validated:56
[task 2020-04-17T09:54:11.488Z] 09:54:11     INFO - chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/head.js:wrappedTestFunction:300
[task 2020-04-17T09:54:11.488Z] 09:54:11     INFO - chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/head.js:inner:243
[task 2020-04-17T09:54:11.488Z] 09:54:11     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1039
[task 2020-04-17T09:54:11.488Z] 09:54:11     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1074
[task 2020-04-17T09:54:11.488Z] 09:54:11     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:904
[task 2020-04-17T09:54:11.489Z] 09:54:11     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:918
Flags: needinfo?(andrei.br92)

Fixed and relanding. Thanks.

Flags: needinfo?(andrei.br92)
Pushed by aoprea@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/dea369153623
Add persistent storage for ExperimentStore r=k88hudson
Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 77

[Tracking Requested - why for this release]: This is a dependency for the Pull Factor Experiment in Firefox 76 https://bugzilla.mozilla.org/show_bug.cgi?id=1628146; without it, Experiment state will not be persistent and we will not be able to measure retention over time.

Comment on attachment 9135712 [details]
Bug 1624309 - Add persistent storage for ExperimentStore

Beta/Release Uplift Approval Request

  • User impact if declined: We will not be able to launch the onboarding Pull Factor Experiment in Firefox 76. This is critical for us to be able to learn about how to make onboarding better and do more experimentation in Q2.
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: No
  • If yes, steps to reproduce: Note that while this is covered by automated tests, it is difficult to QA separately and be covered by an end-to-end functional test plan in the related uplifts: Bug 1620021
  • List of other uplifts needed: Bug 1620021
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): This patch only used by the code in Bug 1620021, so is not risky in itself; no user-facing effects if it fails.
  • String changes made/needed:
Attachment #9135712 - Flags: approval-mozilla-beta?

Comment on attachment 9135712 [details]
Bug 1624309 - Add persistent storage for ExperimentStore

Needed for experiments targeting 76. Approved for 76.0b7.

Attachment #9135712 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

Backed out 4 changesets (bug 1620021, bug 1624309) for causing browser-chrome faliures on browser_aboutwelcome_simplified.js and browser_ext_themes_ntp_colors.js a=Backout

Backout revision https://hg.mozilla.org/releases/mozilla-beta/rev/19a84d3cda6d15e0bf9f52fa67ed0fd2877b1704

Failed push https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&resultStatus=testfailed%2Cbusted%2Cexception&revision=22851affc3b941cff06a869f2b322ccc60c789c0&searchStr=bc

Failure logs https://treeherder.mozilla.org/logviewer.html#?job_id=298703999&repo=mozilla-beta for browser_ext_themes_ntp_colors.js failure
https://treeherder.mozilla.org/logviewer.html#?job_id=298704015&repo=mozilla-beta for browser_aboutwelcome_simplified.js failure

Andrei can you please take a look?

Flags: needinfo?(andrei.br92)
Flags: needinfo?(andrei.br92)

Re-landed for 76.0b8 with the follow-up fix from bug 1620021.
https://hg.mozilla.org/releases/mozilla-beta/rev/a12f4c96ce6f

You need to log in before you can comment on or make changes to this bug.