1693301 - Test utility for automatic enrollment and cleanup

Status: RESOLVED FIXED

(Firefox :: Nimbus Desktop Client, enhancement, P1)

Description

[Andrei Oprea [:andreio]]

We should write some utils to make it easier to enroll and cleanup.

Comment 1

[Andrei Oprea [:andreio]]

Attachment: Bug 1693301 - Nimbus test utility for automatic enrollment and cleanup

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2ee3856bd2ae
Nimbus test utility for automatic enrollment and cleanup r=k88hudson

Comment 3

[Natalia Csoregi [:nataliaCs]]

Backed out for failures on browser_remotesettingsexperimentloader_init.js

backout: https://hg.mozilla.org/integration/autoland/rev/0caed8e54b79bd4f9bdd8972d429ad2df7ad890a

push: https://treeherder.mozilla.org/jobs?repo=autoland&revision=2ee3856bd2aebb4ae25455088809895fc408a4ac&group_state=expanded&selectedTaskRun=d2p_TCvJQAyDPdFmttKqfw.0

failure log: https://treeherder.mozilla.org/logviewer?job_id=331875760&repo=autoland&lineNumber=2808

[task 2021-03-03T11:18:27.004Z] 11:18:27 INFO - TEST-PASS | toolkit/components/nimbus/test/browser/browser_remotesettingsexperimentloader_init.js | Clean state - true == true -
[task 2021-03-03T11:18:27.004Z] 11:18:27 INFO - Console message: [JavaScript Error: "Unknown Collection "main/nimbus-desktop-experiments"" {file: "resource://services-settings/RemoteSettingsClient.jsm" line: 160}]
[task 2021-03-03T11:18:27.004Z] 11:18:27 INFO - UnknownCollectionError@resource://services-settings/RemoteSettingsClient.jsm:160:5
[task 2021-03-03T11:18:27.004Z] 11:18:27 INFO - sync@resource://services-settings/RemoteSettingsClient.jsm:470:13
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO -
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - Buffered messages finished
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/nimbus/test/browser/browser_remotesettingsexperimentloader_init.js | Uncaught exception - at resource://nimbus/lib/ExperimentManager.jsm:200 - Error: An experiment with the slug "foo0.5839170966619033" already exists.
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - Stack trace:
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - enroll@resource://nimbus/lib/ExperimentManager.jsm:200:13
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - onRecipe@resource://nimbus/lib/ExperimentManager.jsm:118:18
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - Leaving test bound test_double_feature_enrollment
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - Console message: 1614770306363 Toolkit.Telemetry WARN TelemetryEnvironment::setExperimentActive - the experiment type was truncated.
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - GECKO(2155) | console.debug: ExperimentManager:
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - GECKO(2155) | Skipping enrollment for "bar0.9567669995147388" because there is an existing experiment for its feature.
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - GECKO(2155) | console.debug: RSLoader:
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - GECKO(2155) | 2 recipes matched. Finalizing ExperimentManager.
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - GECKO(2155) | MEMORY STAT | vsize 20975062MB | residentFast 1120MB
[task 2021-03-03T11:18:27.005Z] 11:18:27 INFO - TEST-OK | toolkit/components/nimbus/test/browser/browser_remotesettingsexperimentloader_init.js | took 773ms

Comment 4

[Natalia Csoregi [:nataliaCs]]

Also failing: https://treeherder.mozilla.org/logviewer?job_id=331877438&repo=autoland&lineNumber=3854

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3d03ee4e4606
Nimbus test utility for automatic enrollment and cleanup r=k88hudson

Comment 6

[Atila Butkovits]

Backed out for causing failure at browser_remotesettingsexperimentloader_init.js.

Backout link: https://hg.mozilla.org/integration/autoland/rev/d0b35ab42e034516cd148dc6ebf3ce3321e1f92b

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&selectedTaskRun=PXaUsqKnQTu0LC9cJG8tqQ.0&revision=3d03ee4e4606138194a34fd63b8152c08104d38b&searchStr=os%2Cx%2C10.14%2Cwebrender%2Copt%2Cmochitests%2Ctest-macosx1014-64-qr%2Fopt-mochitest-browser-chrome-e10s%2Cbc2

Failure log: https://treeherder.mozilla.org/logviewer?job_id=331929356&repo=autoland&lineNumber=8630

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/635ba804cdd8
Nimbus test utility for automatic enrollment and cleanup r=k88hudson

Comment 8

[Noemi Erli[:noemi_erli]]

Backed out changeset 635ba804cdd8 (Bug 1693301) for causing failures in browser_remotesettingsexperimentloader_init.js CLOSED TREE

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&collapsedPushes=842382&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=635ba804cdd8f9b05fc132344e0276985b128583&selectedTaskRun=Am2QJ99DTRWuI9zQNXbwkg.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=332028582&repo=autoland&lineNumber=3117

Backout: https://hg.mozilla.org/integration/autoland/rev/ee5c331e1eb67697f8116b193b84f0469ab79b7b

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9551a9d272d7
Nimbus test utility for automatic enrollment and cleanup r=k88hudson

Comment 10

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/9551a9d272d7