Closed Bug 1584961 Opened 3 years ago Closed 3 years ago

Intermittent (TV) toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | a telemetry experiment should be activated - [["activity-stream-extended-triplets","control"]

Categories

(Firefox :: Normandy Client, defect, P5)

defect

Tracking

()

RESOLVED FIXED
Firefox 71
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox67 --- unaffected
firefox68 --- unaffected
firefox69 --- unaffected
firefox70 --- unaffected
firefox71 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: mythmon)

References

(Regression)

Details

(Keywords: intermittent-failure, regression)

Attachments

(1 file)

Filed by: dluca [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=269021503&repo=mozilla-inbound
Full log: https://queue.taskcluster.net/v1/task/JB6gENfJR7yBfL3hJz_4vw/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/JB6gENfJR7yBfL3hJz_4vw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1


INFO - TEST-START | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js
[task 2019-09-30T11:39:48.838Z] 11:39:48     INFO - TEST-INFO | started process screenshot
[task 2019-09-30T11:39:48.920Z] 11:39:48     INFO - TEST-INFO | screenshot: exit 0
[task 2019-09-30T11:39:48.920Z] 11:39:48     INFO - Buffered messages logged at 11:39:48
[task 2019-09-30T11:39:48.922Z] 11:39:48     INFO - Entering test bound simple_recipe_enrollment
[task 2019-09-30T11:39:48.925Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | lastError should be null - 
[task 2019-09-30T11:39:48.925Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | integer pref should be set - 
[task 2019-09-30T11:39:48.927Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | boolean pref should be set - 
[task 2019-09-30T11:39:48.930Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | string pref should be set - 
[task 2019-09-30T11:39:48.934Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | integer startup pref should be set - 
[task 2019-09-30T11:39:48.934Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | boolean startup pref should be set - 
[task 2019-09-30T11:39:48.934Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | string startup pref should be set - 
[task 2019-09-30T11:39:48.935Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | Rollout should be stored in db - [{"slug":"test-rollout","state":"active","preferences":[{"preferenceName":"test.pref1","value":1,"previousValue":null},{"preferenceName":"test.pref2","value":true,"previousValue":null},{"preferenceName":"test.pref3","value":"it works","previousValue":null}],"enrollmentId":"ee5046f3-b655-40c5-882d-3ff42be59342"}] deepEqual [{"slug":"test-rollout","state":"active","preferences":[{"preferenceName":"test.pref1","value":1,"previousValue":null},{"preferenceName":"test.pref2","value":true,"previousValue":null},{"preferenceName":"test.pref3","value":"it works","previousValue":null}],"enrollmentId":"ee5046f3-b655-40c5-882d-3ff42be59342"}] - 
[task 2019-09-30T11:39:48.935Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | Rollout should have a UUID enrollmentId - 
[task 2019-09-30T11:39:48.936Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | parent must be in snapshot. Has [parent]. - true == true - 
[task 2019-09-30T11:39:48.937Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | After filtering we must have the expected number of events. - 1 == 1 - 
[task 2019-09-30T11:39:48.938Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | category in event normandy#enroll#preference_rollout must match. - "normandy" matches "normandy" - 
[task 2019-09-30T11:39:48.941Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | method in event normandy#enroll#preference_rollout must match. - "enroll" matches "enroll" - 
[task 2019-09-30T11:39:48.941Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | object in event normandy#enroll#preference_rollout must match. - "preference_rollout" matches "preference_rollout" - 
[task 2019-09-30T11:39:48.941Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | value in event normandy#enroll#preference_rollout must match. - "test-rollout" matches "test-rollout" - 
[task 2019-09-30T11:39:48.942Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | Actual event normandy#enroll#preference_rollout expected to have extra. - true == true - 
[task 2019-09-30T11:39:48.942Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | Expected key enrollmentId must be in actual extra. Actual keys: [enrollmentId]. - true == true - 
[task 2019-09-30T11:39:48.942Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | extra[enrollmentId] must match in event normandy#enroll#preference_rollout. - "ee5046f3-b655-40c5-882d-3ff42be59342" matches "ee5046f3-b655-40c5-882d-3ff42be59342" - 
[task 2019-09-30T11:39:48.943Z] 11:39:48     INFO - Buffered messages finished
[task 2019-09-30T11:39:48.943Z] 11:39:48     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | a telemetry experiment should be activated - [["activity-stream-extended-triplets","control"],["test-rollout","active",{"type":"normandy-prefrollout","enrollmentId":"ee5046f3-b655-40c5-882d-3ff42be59342"}]] deepEqual [["test-rollout","active",{"type":"normandy-prefrollout","enrollmentId":"ee5046f3-b655-40c5-882d-3ff42be59342"}]] - JS frame :: chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js :: simple_recipe_enrollment :: line 109
[task 2019-09-30T11:39:48.943Z] 11:39:48     INFO - Stack trace:
[task 2019-09-30T11:39:48.943Z] 11:39:48     INFO - chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js:simple_recipe_enrollment:109
[task 2019-09-30T11:39:48.944Z] 11:39:48     INFO - chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/head.js:wrappedTestFunction:345
[task 2019-09-30T11:39:48.944Z] 11:39:48     INFO - chrome://mochitests/content/browser/toolkit/components/normandy/test/browser/head.js:wrappedTestFunction:304
[task 2019-09-30T11:39:48.944Z] 11:39:48     INFO - resource://normandy/lib/PreferenceRollouts.jsm:inner:194
[task 2019-09-30T11:39:48.944Z] 11:39:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1350
[task 2019-09-30T11:39:48.945Z] 11:39:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1385
[task 2019-09-30T11:39:48.945Z] 11:39:48     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:1213
[task 2019-09-30T11:39:48.945Z] 11:39:48     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:805
[task 2019-09-30T11:39:48.945Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | Telemetry events should not fail - true == true - 
[task 2019-09-30T11:39:48.946Z] 11:39:48     INFO - Console message: [JavaScript Warning: "normandy.recipe_freshness - Truncating float/double number."]
[task 2019-09-30T11:39:48.946Z] 11:39:48     INFO - Leaving test bound simple_recipe_enrollment
[task 2019-09-30T11:39:48.947Z] 11:39:48     INFO - Entering test bound update_enrollment
[task 2019-09-30T11:39:48.947Z] 11:39:48     INFO - Console message: [JavaScript Warning: "normandy.recipe_freshness - Truncating float/double number."]
[task 2019-09-30T11:39:48.947Z] 11:39:48     INFO - TEST-PASS | toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | lastError should be null - 
Summary: Intermittent toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | a telemetry experiment should be activated - [["activity-stream-extended-triplets","control"] → Intermittent (TV) toolkit/components/normandy/test/browser/browser_actions_PreferenceRolloutAction.js | a telemetry experiment should be activated - [["activity-stream-extended-triplets","control"]

This bug is most likely a regression caused by Bug 1555176

Regressed by: 1555176
Assignee: nobody → mcooper
Status: NEW → ASSIGNED

Dorel, how did you determine that bug 1555176 was the regressing bug? From my reading, it looks like bug 1572640 is the regressing bug, since it is the bug that added the activity-stream-extended-triplets experiment. That bug introduced an experiment that runs at Firefox startup, and has a small chance to call setExperimentActive.

This avoids an intermittent test failure when other parts of Firefox set experiments active while this test is running

Pushed by mcooper@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/97f48d8b40a6
Use sinon matcher instead of exact call list in Normandy preference rollout test r=Gijs
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 71
Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.