1415284 - [meta] Remove client-side support for Telemetry Experiments

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P3)

Description

[Robert Helmer [:rhelmer]]

Telemetry Experiments have been replaced by Shield Studies.

From talking with folks and looking at https://hg.mozilla.org/webtools/telemetry-experiment-server/ it doesn't appear that we've had any active studies for at least the past few months.

There's quite a bit of code in the addons manager, and also a few other places throughout the tree. I'd like to just remove all of it as part of this bug if there are no objections.

Comment 1

[Chris H-C :chutten]

Telemetry Experiments is the only current user of TelemetryLog, so we may wish to consider removing TelemetryLog while we're at it.

Comment 2

[Georg Fritzsche [:gfritzsche]]

I'd like to flag this first with some people next week to make sure nobody gets surprised by this or plans with it.
Otherwise as the de-facto owner i'm happy to see this code go and to have our experiments generally run through SHIELD!

Client-side, there is browser/experiments, add-on manager code and a bit of usage in TelemetryEnvironment.jsm.

Telemetry-wise, we can probably remove the TelemetryLog and environment/addons/activeExperiments.
This goes best into a separate bug where we check into pipeline dependencies.
If you just leave the activeExperiments field in as a stub, that should be sufficient for this bug.

We should also file a separate bug into Data Platform :: General, for investigating if there are any remaining data jobs that need to be killed.

Felipe, any other parts that you can think of?
Going around some server repos & wiki pages, marking them as obsoleted?

Comment 3

[:Felipe Gomes (needinfo for replies!)]

The other thing that I can think of is to see if we can decommission the server for the experiments: https://telemetry-experiment.cdn.mozilla.net/

That would require testing to see if Firefox behaves properly if it can't reach it. And making sure we don't want to run any more experiments on already-released versions  (maybe that should wait for the next ESR?)


There's code in about:support to list the existing experiments, and I imagine there are some SUMO pages with screenshots from the Add-ons Manager showing the Experiments tab


Also, it would probably be good to leave a mark in the code to reserve the number for addon type=experiment (128) so that it doesn't get reused in the future.

Comment 4

[Chris H-C :chutten]

I would be 100% not surprised at all to learn that we have a policy of _never_ experimenting on ESR users, so we should be okay on that front. Double-check with a Data Peer.

Comment 5

[Georg Fritzsche [:gfritzsche]]

Thomas wanted to raise this next week and see if anyone is concerned.
The dependencies outside of Firefox code are probably good for non-blocking follow-ups.

Comment 6

[Andrew Swan [:aswan]]

(In reply to Georg Fritzsche [:gfritzsche] [away Nov 24 - Dec 3] from comment #2)
> Client-side, there is browser/experiments, add-on manager code and a bit of
> usage in TelemetryEnvironment.jsm.

Should we have separate bugs for these?  I'm keen to do the addons manager part of it but would not be the best person to touch the bits in toolkit/telemetry and elsewhere...

Comment 7

[Chris H-C :chutten]

(In reply to Andrew Swan [:aswan] (PTO through 11/27) from comment #6)
> Should we have separate bugs for these?  I'm keen to do the addons manager
> part of it but would not be the best person to touch the bits in
> toolkit/telemetry and elsewhere...

Sounds like a plan. I'll go ahead and meta this so we can start filing subordinate bugs. I'll let you file the addons manager one, and I'll file the browser/experiments+toolkit/telemetry bug and one for TelemetryLog.

Though before starting work, I'd recommend waiting on Thomas getting back to us. I expect no problems, so if you're motivated then by all means go ahead... but if you're looking for an excuse to slow down, there it is :)

Comment 8

[Andrew Swan [:aswan]]

(In reply to Chris H-C :chutten from comment #7)
> Though before starting work, I'd recommend waiting on Thomas getting back to
> us. I expect no problems, so if you're motivated then by all means go
> ahead... but if you're looking for an excuse to slow down, there it is :)

The needinfo from a few weeks ago is still present here, are we good to go on this?  More urgently, are there any open questions here that would be more easily addressed talking face-to-face before the end of the week?

Comment 9

[Chris H-C :chutten]

I think we're good to go. SHIELD is quite completely stepping up to answer the questions Telemetry Experiments. 

I'll leave the ni? for now and we can go ahead.

Comment 10

[Georg Fritzsche [:gfritzsche]]

I caught up with Thomas about this.
I don't expect anyone to need Telemetry Experiments, so we'll just communicate this as going away in Fx 59+.
We're good to go to remove this on the client.

Comment 11

[Andrew Swan [:aswan]]

Moving this over to Telemetry since it became [meta], AddonManager-specific work can happen in bug 1450801