1498940 - When no studies have run, about:studies shows "Nightly may install and run studies" despite studies being disabled

Status: VERIFIED FIXED

(Firefox :: Normandy Client, defect)

Description

[Oriol Brufau [:Oriol]]

In about:preferences#privacy,

 - Check "Allow Nightly to send technical and interaction data to Mozilla" (datareporting.healthreport.uploadEnabled=true)
 - Uncheck "Allow Nightly to install and run studies" (app.shield.optoutstudies.enabled=false)

Then go to about:studies. A message says

> Nightly may install and run studies from time to time

I need to disable "Allow Nightly to send technical and interaction data to Mozilla" in order to get "No new studies will run."

This is very confusing. If app.shield.optoutstudies.enabled=false alone doesn't prevent studies, then "Allow Nightly to install and run studies" in about:preferences should be changed to explain whatever the pref really does.

Otherwise about:studies should also check app.shield.optoutstudies.enabled instead of just datareporting.healthreport.uploadEnabled.

This started in https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=8fb0418948834f071dd99f6b8256b746b4a8adce&tochange=94f5c92fc711a1e1e76ea54d52482cbbd2db0401

Comment 1

[Michael Cooper [:mythmon]]

Thanks for the bug report. I was able to duplicate what you're seeing as well.

I'm skeptical that the bug started in the changeset you linked, since that commit tends to break history views. I suspect it was pre-existing when we moved all the code around. How did you narrow it down to that changeset?

I'm fairly sure, but haven't verified yet, that we are honoring the preferences in the actual behavior of installing add-ons, and the bug is only related to about:studies. I will also double check about that though.

Comment 2

[Oriol Brufau [:Oriol]]

(In reply to Michael Cooper [:mythmon] from comment #1)
> I'm skeptical that the bug started in the changeset you linked, since that
> commit tends to break history views. I suspect it was pre-existing when we
> moved all the code around. How did you narrow it down to that changeset?

I used mozregression-gui. Before that changeset, the message box was empty. After the changeset, I started getting "Nightly may install and run studies from time to time" despite app.shield.optoutstudies.enabled=false

Comment 3

[David Durst [:ddurst]]

[Tracking Requested - why for this release]: we don't want this to get bigger than it already is; should be resolved ASAP.

Comment 4

[Michael Cooper [:mythmon]]

I did more research here, and the problem happens happens specifically when no studies have ever run on the profile. When this is the case, the message chosen doesn't vary based on if studies are enabled or not.

To be clear, the pref is respected: when the preference is changed studies are not installed and existing studies are ended. The bug here is the wrong message being shown only.

Comment 5

[Michael Cooper [:mythmon]]

Attachment: Bug 1498940 - Reflect study opt-out in about:studies when Normandy is generally enabled. r=Gijs

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f0ed99b29aaf
Reflect study opt-out in about:studies when Normandy is generally enabled. r=Gijs

Comment 7

[Andrei Ciure[:aciure]]

Backed out 2 changesets (bug 1501877, bug 1498940) for failing browser_about_studies.js

push that caused the backout: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=f0ed99b29aafb5fd7f613a000634b53927af4259&selectedJob=207819106

failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=f0ed99b29aafb5fd7f613a000634b53927af4259&selectedJob=207819106&searchStr=linux%2Cx64%2Cdebug%2Cmochitests%2Cwith%2Ce10s%2Ctest-linux64%2Fdebug-mochitest-browser-chrome-e10s-4%2Cm-e10s%28bc4%29

backout: https://hg.mozilla.org/integration/autoland/rev/d08c28ed42f1bef0b77d38064759b797f377a998

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9de1bee6adc2
Reflect study opt-out in about:studies when Normandy is generally enabled. r=Gijs

Comment 9

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/9de1bee6adc2

Comment 10

[Ryan VanderMeulen [:RyanVM]]

Please nominate this for Beta approval when you get a chance.

Comment 11

[Michael Cooper [:mythmon]]

Comment on attachment 9019889 [details]
Bug 1498940 - Reflect study opt-out in about:studies when Normandy is generally enabled. r=Gijs

[Beta/Release Uplift Approval Request]

Feature/Bug causing the regression: None

User impact if declined: When a user has disabled studies but not Normandy, about:studies incorrectly shows that studies may run.

Is this code covered by automated tests?: Yes

Has the fix been verified in Nightly?: No

Needs manual test from QE?: No

If yes, steps to reproduce: 

List of other uplifts needed: None

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): Only a change to the display logic of about:studies, not any of the actual logic of running studies or Normandy

String changes made/needed:

Comment 12

[Ryan VanderMeulen [:RyanVM]]

This has conflicts with Beta - do we need bug 1501877 also?

Comment 13

[Michael Cooper [:mythmon]]

Oops. Yes, we need bug 1501877. We'll also need bug 1502169 to avoid some intermittent test failures. The three patches were developed together and split to separate concerns in review.

Comment 14

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9019889 [details]
Bug 1498940 - Reflect study opt-out in about:studies when Normandy is generally enabled. r=Gijs

[Triage Comment]
Fixes a confusing UI issue, approved for 64.0b6.

Comment 15

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/e240f9e29362

Comment 16

[Adrian Florinescu [:aflorinescu]]

Reproduced the issue on Fx65 29-10-2018 / verified as fixed on Nightly 31-10-2018 on Windows 10/Windows 8.1. Given the nature of the fix, I don't think a cross platform verification is required.

With the fix, when "Allow Nightly to install and run studies" is checked off, the about:studies lists: "This is a list of studies that you have participated in. No new studies will run."

Comment 17

[Adrian Florinescu [:aflorinescu]]

Verified as fixed on 64.0b6 2018-11-01 on OSX 10.13/ Windows 10 / Ubuntu 16.04.