1191421 - Don't show unsigned addon blocked notifications for experiments

Status: VERIFIED FIXED

(Toolkit :: Telemetry, defect)

Description

[Georg Fritzsche [:gfritzsche]]

Bug 1191018, comment 0 says that we show unsigned addon blocking notifications for experiments.

We should not show this for experiments.

Comment 1

[Georg Fritzsche [:gfritzsche]]

Dave, do you know who can take this?

Comment 2

[Dave Townsend [:mossop]]

I have to make some changes to this notification anyway

Comment 3

[Dave Townsend [:mossop]]

Attachment: patch

This applies on top of bug 1191008 and filters out experiments from the lists before deciding whether to show the notification.

Comment 4

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/2b359bf46360

Comment 5

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/2b359bf46360

Comment 6

[Dave Townsend [:mossop]]

Comment on attachment 8643925 [details] [diff] [review]
patch

Approval Request Comment
[Feature/regressing bug #]: Add-on Signing
[User impact if declined]: Users may see unexpected warnings about telemetry experiments.
[Describe test coverage new/current, TreeHerder]: Tested manually, we don't have good automated tests for the notifications shown at startup
[Risks and why]: Low risk, the code is straightforward
[String/UUID change made/needed]: None

Comment 7

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Comment on attachment 8643925 [details] [diff] [review]
patch

Given that add-on signing is a Aurora41 feature, let's uplift to Aurora.

Comment 8

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Requesting QE team to verify the fix.

Comment 9

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Georg, if possible, could you please verify this fix? Thanks.

Comment 10

[Georg Fritzsche [:gfritzsche]]

Vladan, as you saw this before, can you confirm?

If not, Kamil put up QA documentation for experiments here:
https://wiki.mozilla.org/QA/Desktop_Firefox/Telemetry_Experiments

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/d6c73ce94b10

Comment 12

[Vladan Djeric (:vladan)]

This is not fixed, with this latest Nightly, I still get the "Some add-ons have been disabled" screen with ***Heartbeat*** listed. 

I also still see the message "Heartbeat Telemetry Experiment 1 could not be verified use in Nightly and has been disabled" in about:addons.

Is Heartbeat a specific type of Telemetry Experiment?

Comment 13

[Dave Townsend [:mossop]]

(In reply to Vladan Djeric (:vladan) -- please needinfo! from comment #12)
> This is not fixed, with this latest Nightly, I still get the "Some add-ons
> have been disabled" screen with ***Heartbeat*** listed. 

Is this listed in the experiments section of the add-ons manager?

> I also still see the message "Heartbeat Telemetry Experiment 1 could not be
> verified use in Nightly and has been disabled" in about:addons.

That's expected, we're just disabling the notification bars here.

> Is Heartbeat a specific type of Telemetry Experiment?

Comment 14

[Georg Fritzsche [:gfritzsche]]

(In reply to Dave Townsend [:mossop] from comment #13)
> > I also still see the message "Heartbeat Telemetry Experiment 1 could not be
> > verified use in Nightly and has been disabled" in about:addons.
> 
> That's expected, we're just disabling the notification bars here.

Can't we hide those kind of notifications/messages for the experiments entries in about:addons?

Comment 15

[Dave Townsend [:mossop]]

(In reply to Georg Fritzsche [:gfritzsche] from comment #14)
> (In reply to Dave Townsend [:mossop] from comment #13)
> > > I also still see the message "Heartbeat Telemetry Experiment 1 could not be
> > > verified use in Nightly and has been disabled" in about:addons.
> > 
> > That's expected, we're just disabling the notification bars here.
> 
> Can't we hide those kind of notifications/messages for the experiments
> entries in about:addons?

We can, but should we? I guess I've always been confused about the user expectations for experiments, are users expecting to be able to manage them in some way?

Comment 16

[Georg Fritzsche [:gfritzsche]]

(In reply to Dave Townsend [:mossop] from comment #15)
> (In reply to Georg Fritzsche [:gfritzsche] from comment #14)
> > (In reply to Dave Townsend [:mossop] from comment #13)
> > > > I also still see the message "Heartbeat Telemetry Experiment 1 could not be
> > > > verified use in Nightly and has been disabled" in about:addons.
> > > 
> > > That's expected, we're just disabling the notification bars here.
> > 
> > Can't we hide those kind of notifications/messages for the experiments
> > entries in about:addons?
> 
> We can, but should we? I guess I've always been confused about the user
> expectations for experiments, are users expecting to be able to manage them
> in some way?

No, there is no user control over this (except turning off Telemetry etc. and thus all experiments).
The experiments section is there for user transparency, the management of the addons is done by the experiments system.

I don't think showing "outdated" or "blocked" notifications for experiments adds any value.

Comment 17

[Dave Townsend [:mossop]]

(In reply to Georg Fritzsche [:gfritzsche] from comment #16)
> (In reply to Dave Townsend [:mossop] from comment #15)
> > (In reply to Georg Fritzsche [:gfritzsche] from comment #14)
> > > (In reply to Dave Townsend [:mossop] from comment #13)
> > > > > I also still see the message "Heartbeat Telemetry Experiment 1 could not be
> > > > > verified use in Nightly and has been disabled" in about:addons.
> > > > 
> > > > That's expected, we're just disabling the notification bars here.
> > > 
> > > Can't we hide those kind of notifications/messages for the experiments
> > > entries in about:addons?
> > 
> > We can, but should we? I guess I've always been confused about the user
> > expectations for experiments, are users expecting to be able to manage them
> > in some way?
> 
> No, there is no user control over this (except turning off Telemetry etc.
> and thus all experiments).
> The experiments section is there for user transparency, the management of
> the addons is done by the experiments system.
> 
> I don't think showing "outdated" or "blocked" notifications for experiments
> adds any value.

File a bug for that then, I don't have time to get to it unless it is high priority but I can offer advice and reviews

Comment 18

[Benjamin Smedberg]

Perhaps the core issue is: are we going to sign experiment XPIs? If not, we need to hide the notifications. But it sounds like we should sign them and then this bug becomes much less interesting. I don't know how we go about getting them signed without a lot of AMO pain, though.

Comment 19

[Dave Townsend [:mossop]]

Some confusion here, Heartbeat is an old extension, not an experiment which is why it is showing up in warnings. Removing it is covered by bug 1191018.

Comment 20

[Ada [:adalucinet]]

Dave, based on the [qe-verify +] flag, can you please provide some guidance on how to verify this fix?
As far as I know, there are no active experiments for now [1]. Thanks in advance!

[1] https://telemetry-experiment.cdn.mozilla.net/

Comment 21

[Dave Townsend [:mossop]]

Not really, maybe Georg can

Comment 22

[Georg Fritzsche [:gfritzsche]]

There is documentation on running Telemetry experiments locally here:
https://wiki.mozilla.org/QA/Telemetry#Telemetry_Experiments.2FFHR_Documentation

Comment 23

[Ada [:adalucinet]]

(In reply to Georg Fritzsche [:gfritzsche] from comment #22)
> There is documentation on running Telemetry experiments locally here:
> https://wiki.mozilla.org/QA/Telemetry#Telemetry_Experiments.
> 2FFHR_Documentation

Thanks, Georg!; unfortunately it didn't help - tried to install telemetry experiment, but without success; Is there any other way to reproduce/verify this issue here that doesn't involve setting up a staging server?

Comment 24

[Georg Fritzsche [:gfritzsche]]

Not really, i think we need to figure out whats not working on your end.
Did you fail to set things up? Or did you run into any errors?

Comment 25

[Georg Fritzsche [:gfritzsche]]

(In reply to Georg Fritzsche [:gfritzsche] from comment #24)
> Not really, i think we need to figure out whats not working on your end.
> Did you fail to set things up? Or did you run into any errors?

Per bug 1207607 it looks like we require experiments to be signed - i think we need to wait for the outcome of that bug first.

Comment 26

[Georg Fritzsche [:gfritzsche]]

Per that bug you need to set xpinstall.signatures.required;false to be able to install unsigned experiments.

Comment 27

[Ada [:adalucinet]]

Reproduced with 42.0a1 (from 2015-08-05) - 'Some extensions could not be verified' notification is displayed for experiments.
Verified fixed with 42.0b1 (Build ID: 20150921151815) and 41.0 RC build 3 (Build ID: 20150917150946), across platforms [1], by changing/creating the following prefs via about:config page:
- devtools.chrome.enabled; true
- experiments.logging.level; 0
- experiments.manifest.cert.checkAttributes; false
- experiments.force-sample-value; 0.0
- experiments.manifest.uri = path to this https://goo.gl/fbHD0h manifest.json download sample (Many thanks, Georg!)

[1] Windows 7 64-bit, Mac OS X 10.10.5 and Ubuntu 14.04 32-bit