1435714 - [Dedicated profiles] Clear appropriate telemetry data when cloning profiles

Status: RESOLVED INVALID

(Toolkit :: Startup and Profile System, defect, P3)

Description

[Adrian Florinescu [:aflorinescu]]

[Environment:]
-Ubuntu 16.04 x64
-Firefox 60.0a1 (trybuild from https://treeherder.mozilla.org/#/jobs?repo=try&revision=6b67fc750f609d3eaa4097528528458e43bb4873&selectedJob=158774545)
-Firefox 58.0.1	20180128191252

[Pre-requisites:]
Delete all Firefox data and cache.

[Steps:]
1. Start Firefox 58.0.1 
2. Open about:config, search and record the value for toolkit.telemetry.cachedClientID
3. Close Firefox 58.0.1 and start the Nightly try build that contains the dedicated profile feature.
4. Open about:config, search and record the value for toolkit.telemetry.cachedClientID

[Actual Result:]
On step 3 - new profile is created(default-nightly) - (cloned after default) 
The ClientID values for both versions are the same, which means that telemetry probably identifies the two profiles as being the same data: default (created at step1) and default-nightly (step3)

[Expected Result:]
The values for toolkit.telemetry.cachedClientID are different for default and default-nightly profiles.
Also, I also could guess that probably what's in the ./datareporting folder shouldn't be copied either.

Comment 1

[Dave Townsend [:mossop]]

From fx-data-dev:

- We need to make sure that the client id gets reset, as this for us is a profile identifier. This means: resetting the pref toolkit.telemetry.cachedClientID, clearing datareporting/state.json
- We need to clear datareporting/session-state.json
- Unsent pings being sent from the new profile sounds right. We'd probably rather have duplicates. @mreid to confirm?
- What about the profile age (ProfileAge.jsm, times.json)? We use this to judge the "age" of a user. I could see arguments for both retaining and clearing.
- We should track that this is a "per-channel-profile copy" in Telemetry. Then we can trace down oddities etc. This might live in the environment?

Comment 2

[Romain Testard [:RT]]

(In reply to Dave Townsend [:mossop] from comment #1)
> From fx-data-dev:
> 

> - What about the profile age (ProfileAge.jsm, times.json)? We use this to
> judge the "age" of a user. I could see arguments for both retaining and
> clearing.
Per discussions with Alessio on fx-data-dev:
We should make sure to retain the profile creation date when creating and populating a new clone profile, otherwise they will look like new users.
http://searchfox.org/mozilla-central/source/toolkit/components/telemetry/TelemetryEnvironment.jsm#1411

Comment 3

[Adrian Florinescu [:aflorinescu]]

(In reply to Romain Testard [:RT] from comment #2)
> (In reply to Dave Townsend [:mossop] from comment #1)
> > From fx-data-dev:
> > 
> 
> > - What about the profile age (ProfileAge.jsm, times.json)? We use this to
> > judge the "age" of a user. I could see arguments for both retaining and
> > clearing.
> Per discussions with Alessio on fx-data-dev:
> We should make sure to retain the profile creation date when creating and
> populating a new clone profile, otherwise they will look like new users.
> http://searchfox.org/mozilla-central/source/toolkit/components/telemetry/
> TelemetryEnvironment.jsm#1411

And aren't they new users from the profile and channel perspective?

Comment 4

[Romain Testard [:RT]]

> > > - What about the profile age (ProfileAge.jsm, times.json)? We use this to
> > > judge the "age" of a user. I could see arguments for both retaining and
> > > clearing.
> > Per discussions with Alessio on fx-data-dev:
> > We should make sure to retain the profile creation date when creating and
> > populating a new clone profile, otherwise they will look like new users.
> > http://searchfox.org/mozilla-central/source/toolkit/components/telemetry/
> > TelemetryEnvironment.jsm#1411
> 
> And aren't they new users from the profile and channel perspective?
Technically these are new profiles but since cloning happens after app update to 60 (the channel existed before that, it was just running another default profile) in this case, data analysts should not interpret these profiles as  being new.

NI to Alessio in case he wants to add a comment

Comment 5

[Alessio Placitelli [:Dexter]]

(In reply to Romain Testard [:RT] from comment #4)
> > > > - What about the profile age (ProfileAge.jsm, times.json)? We use this to
> > > > judge the "age" of a user. I could see arguments for both retaining and
> > > > clearing.
> > > Per discussions with Alessio on fx-data-dev:
> > > We should make sure to retain the profile creation date when creating and
> > > populating a new clone profile, otherwise they will look like new users.
> > > http://searchfox.org/mozilla-central/source/toolkit/components/telemetry/
> > > TelemetryEnvironment.jsm#1411
> > 
> > And aren't they new users from the profile and channel perspective?
> Technically these are new profiles but since cloning happens after app
> update to 60 (the channel existed before that, it was just running another
> default profile) in this case, data analysts should not interpret these
> profiles as  being new.
> 
> NI to Alessio in case he wants to add a comment

This is the conversation Romain is referring to: https://mail.mozilla.org/pipermail/fx-data-dev/2017-October/000081.html
However, looks like a new conversation is going on: https://mail.mozilla.org/pipermail/fx-data-dev/2018-February/000109.html (:mossop is quoting this in comment 1).

I personally don't have a strong opinion, as there might be arguments both ways. This could probably benefit from the opinion of the people relying on this data (e.g. :joy).

Comment 6

[Romain Testard [:RT]]

With NI to :joy now
For info we're considering only cloning profiles from users who do switch channels regularly so cloning would happen a lot less (discussing with Saptarshi the share of users that would get impacted for a given channel).

Comment 7

[Dave Townsend [:mossop]]

The profile cloning feature never shipped.