1447499 - Show opt-out preference studies in about:studies

Status: RESOLVED FIXED

(Firefox :: Normandy Client, enhancement, P3)

Description

[Michael Cooper [:mythmon]]

Preference studies don't show up in about:studies, because about:studies was designed specifically for add-on studies. However, the usage of studies doesn't draw a strong distinction between preference and add-on studies, and the two should be unified to the user.

We should include preference studies in about:studies, including allowing users to opt-out of them.

Comment 1

[Darkspirit]

(Jan Andre Ikenmeyer [:darkspirit] from bug 1477380 comment 15)
> (In reply to Michael Cooper [:mythmon] from bug 1477380 comment 14)
> > It is expected that this type of study will not be listed in about:studies. Preference studies aren't included in the listing there.
> 
> All studies have to be listed on about:studies. That is user expectation.
> Also, if things suddenly change one could see which study has completed and how to bring it back. Where can I see the following "user facing description"? ;)
> 
> (Thomas Elin [:relaas] from bug 1474484 comment 0)
> > User facing title of the experiment: WebRender
> > User facing description of the experiment: New generation graphics rendering engine

(Gijs (he/him) from bug 1477380 comment 22)
> (In reply to Jan Andre Ikenmeyer [:darkspirit] from bug 1477380 comment 15)
> > (In reply to Michael Cooper [:mythmon] from bug 1477380 comment 14)
> > > It is expected that this type of study will not be listed in about:studies. Preference studies aren't included in the listing there.
> > All studies have to be listed on about:studies. That is user expectation.
> 
> I am sympathetic to this point... is there a separate bug on file discussing this? I recently had a confusing experience where the number of default rows in my location bar had changed, and I checked about:studies to see if that was why, and didn't see anything. Now I'm realizing that it might still have been a preference study... but there's no way for me as a user to find out. That's... not great.

Comment 2

[Darkspirit]

https://mikeconley.ca/blog/2018/07/24/firefox-performance-update-10/
> The Graphics team has enabled WebRender by default for a subset of our Nightly population to test it. If you’re in that group, please file bugs if you see them! Check about:studies to see if you’re in the testing group.

Comment 3

[Michael Cooper [:mythmon]]

Attachment: Bug 1447499 - Simplify about:studies code

Comment 4

[Michael Cooper [:mythmon]]

Attachment: Bug 1447499 - Add preference studies to about:studies

Comment 5

[:Gijs (he/him)]

Comment on attachment 9007930 [details]
Bug 1447499 - Simplify about:studies code

:Gijs (he/him) has approved the revision.

Comment 6

[Michael Cooper [:mythmon]]

Attachment: Bug 1447499 - Refactor test helper PreferenceExperiments.withMockExperiment to work like AddonStudies.withStudies.

Comment 7

[:Gijs (he/him)]

Comment on attachment 9008238 [details]
Bug 1447499 - Refactor test helper PreferenceExperiments.withMockExperiment to work like AddonStudies.withStudies.

:Gijs (he/him) has approved the revision.

Comment 8

[:Gijs (he/him)]

Comment on attachment 9007931 [details]
Bug 1447499 - Add preference studies to about:studies

:Gijs (he/him) has approved the revision.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/675b77e1c236
Simplify about:studies code r=Gijs
https://hg.mozilla.org/integration/autoland/rev/3579f8912b70
Refactor test helper PreferenceExperiments.withMockExperiment to work like AddonStudies.withStudies. r=Gijs
https://hg.mozilla.org/integration/autoland/rev/8eeaf62be0dc
Add preference studies to about:studies r=Gijs

Comment 10

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/dc0f1bb95086
Backed out 3 changesets for browser chrome failures on browser_all_files_referenced. CLOSED TREE

Comment 11

[Cosmin Sabou [:CosminS]]

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed,busted,exception&revision=8eeaf62be0dc07c0710dc8b1f8442ede33adaedb&searchStr=bc&selectedJob=199138891

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=199138891&repo=autoland&lineNumber=1746

[task 2018-09-13T18:07:57.329Z] 18:07:57     INFO - indirectly whitelisted file: resource://gre/modules/accessibility/Traversal.jsm used from resource://gre/modules/accessibility/ContentControl.jsm
[task 2018-09-13T18:07:57.330Z] 18:07:57     INFO - Buffered messages finished
[task 2018-09-13T18:07:57.332Z] 18:07:57     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/static/browser_all_files_referenced.js | there should be no unreferenced files - Got 1, expected 0
[task 2018-09-13T18:07:57.333Z] 18:07:57     INFO - Stack trace:
[task 2018-09-13T18:07:57.337Z] 18:07:57     INFO - chrome://mochikit/content/browser-test.js:test_is:1304
[task 2018-09-13T18:07:57.337Z] 18:07:57     INFO - chrome://mochitests/content/browser/browser/base/content/test/static/browser_all_files_referenced.js:checkAllTheFiles:766
[task 2018-09-13T18:07:57.338Z] 18:07:57     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1102
[task 2018-09-13T18:07:57.338Z] 18:07:57     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1093
[task 2018-09-13T18:07:57.339Z] 18:07:57     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:995
[task 2018-09-13T18:07:57.339Z] 18:07:57     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
[task 2018-09-13T18:07:57.340Z] 18:07:57     INFO - Not taking screenshot here: see the one that was previously logged
[task 2018-09-13T18:07:57.341Z] 18:07:57     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/static/browser_all_files_referenced.js | unreferenced file: resource://normandy-content/about-studies/img/shield-logo.png - 
[task 2018-09-13T18:07:57.341Z] 18:07:57     INFO - Stack trace:
[task 2018-09-13T18:07:57.342Z] 18:07:57     INFO - chrome://mochitests/content/browser/browser/base/content/test/static/browser_all_files_referenced.js:checkAllTheFiles:770
[task 2018-09-13T18:07:57.343Z] 18:07:57     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1102
[task 2018-09-13T18:07:57.344Z] 18:07:57     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1093
[task 2018-09-13T18:07:57.345Z] 18:07:57     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:995
[task 2018-09-13T18:07:57.346Z] 18:07:57     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:795
[task 2018-09-13T18:07:57.349Z] 18:07:57     INFO - ignored unused whitelist entry: resource://app/blocklist.xml
[task 2018-09-13T18:07:57.351Z] 18:07:57     INFO - ignored unused whitelist entry: resource://gre/gmp-clearkey/0.1/manifest.json
[task 2018-09-13T18:07:57.353Z] 18:07:57     INFO - ignored unused whitelist entry: resource://gre/res/test.properties
[task 2018-09-13T18:07:57.353Z] 18:07:57     INFO - GECKO(1629) | [Parent 1629, Main Thread] WARNING: Error loading theme icon 'dialog-warning' for stock: Icon 'dialog-warning' not present in theme ubuntu-mono-dark: 'glib warning', file /builds/worker/workspace/build/src/toolkit/xre/nsSigHandlers.cpp, line 141
[task 2018-09-13T18:07:57.357Z] 18:07:57     INFO - GECKO(1629) | (firefox:1629): Gtk-WARNING **: Error loading theme icon 'dialog-warning' for stock: Icon 'dialog-warning' not present in theme ubuntu-mono-dark
[task 2018-09-13T18:07:57.359Z] 18:07:57     INFO - missing file: resource://gre/modules/commonjs/toolkit/loader.js
[task 2018-09-13T18:07:57.360Z] 18:07:57     INFO - missing file: chrome://help/content/help.js referenced from chrome://pippki/content/protectedAuth.xul
[task 2018-09-13T18:07:57.361Z] 18:07:57     INFO - missing file: resource://gre/modules/GeckoViewTelemetryController.jsm referenced from resource://gre/components/ContentProcessSingleton.js

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/931f0b2487ef
Simplify about:studies code r=Gijs
https://hg.mozilla.org/integration/autoland/rev/220d96bc8cb9
Refactor test helper PreferenceExperiments.withMockExperiment to work like AddonStudies.withStudies. r=Gijs
https://hg.mozilla.org/integration/autoland/rev/2fe042527cad
Add preference studies to about:studies r=Gijs

Comment 13

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/931f0b2487ef
https://hg.mozilla.org/mozilla-central/rev/220d96bc8cb9
https://hg.mozilla.org/mozilla-central/rev/2fe042527cad

Comment 14

[Ryan VanderMeulen [:RyanVM]]

Is this something we want to consider uplifting to Beta or can it ride the 64 train?

Comment 15

[Michael Cooper [:mythmon]]

I think this should ride the 64 train. It is a nice feature, but I think it could use the usual amount of time baking.