1474285 - Implement profile-per-install

Status: RESOLVED FIXED

(Toolkit :: Startup and Profile System, enhancement)

Description

[Dave Townsend [:mossop]]

This is an alternate plan to bug 1373244 wherein we use a dedicated profile for each install of Firefox on the system as identified by a hash of the installation directory. In order to identify which install to give the current default profile to we look in compatibility.ini to see which install last launched the profile. Other installs receive a clean new profile.

Most users with just one install of Firefox on their system will see no effect from this change.

To support downgrade scenarios we create an old-style default profile in profiles.ini for older versions of Firefox to use.

Comment 1

[Dave Townsend [:mossop]]

Attachment: Bug 1474285: Make nsINIParser mutable.

In order to implement profile-per-install we need a new mutable datastore
alongside profiles.ini. Using an INI file makes sense however the C++ INI
parser is not mutable. This adds mutability and a method for writing the INI
data to disk.

In the future this can replace nsIINIParser.

MozReview-Commit-ID: L4VoIPe1OGS

Comment 2

[Dave Townsend [:mossop]]

Attachment: Bug 1474285: Expose a unique hash for the application install directory.

Profile-per-install uses a hash of the install directory to identify different
installs of Firefox. This exposes the existing cityhash generated hash from
nsXREDirProvider and makes it available on all platforms.

Comment 3

[Dave Townsend [:mossop]]

Attachment: Bug 1474285: Implement dedicated profiles per install. r=froydnj

Uses a different profile depending on the install directory of the application.
installs.ini is used to map a profile directory to the install directory hash.
If no profile is marked as default for the current install the old-style profile
default is used assuming it was last run with the current application.
Always creates an old-style default profile if one does not exist to allow
previous versions of the application to use a different profile.

Comment 4

[Dave Townsend [:mossop]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=04bc62dae7320c5ebe0b99360f299bceedccc9a6

Comment 5

[Dave Townsend [:mossop]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=cac225222d95c68264173da6db6e3662fdf152d4

Comment 6

[Dave Townsend [:mossop]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=31ab26e5fabf2f171a51a78da2b26cea352db316

Comment 7

[Su-Young Hong]

Hey, wanted to get some info on this proposed change. 

This would be an additional, kind of meta-profile that maps 1 to 1 with installations of Firefox, in addition to the existing profiles we have now, right? 

This wouldn't change how current profiles operate, correct? 

Would this exist on a per-computer-per-installation-per-OS-userprofile level? Or would it be able to exist per-computer-per-installation?

Comment 8

[Dave Townsend [:mossop]]

(In reply to Su-Young Hong from comment #7)
> Hey, wanted to get some info on this proposed change. 
> 
> This would be an additional, kind of meta-profile that maps 1 to 1 with
> installations of Firefox, in addition to the existing profiles we have now,
> right? 

Not sure I really understand this.

> This wouldn't change how current profiles operate, correct? 

Yes, it changes how the default profile is chosen.

> Would this exist on a per-computer-per-installation-per-OS-userprofile
> level? Or would it be able to exist per-computer-per-installation?

Per-computer-per-install.

More details will be coming in a newgroup post soon.

Comment 9

[Su-Young Hong]

Hi Dave, 

Thanks so much for meeting last week and explaining the profile creation changes to me. 

Like we discussed, the following telemetry would be very helpful for us to deal with this change. 

* is_stubprofile: 
	- indicates if a profile was created as a stub profile (as opposed to the primary profile)
* profile_creation_date:
	- keep this probe the way it is (so no changes to it's current behavior)
* first_run_date: 
	- field which indicates the timestamp of when the first run of the profile is. 

:chutten, :janerik, do you agree with the above? are there any other measurements we should take in light of the changes to profile creation behavior?

Comment 10

[Chris H-C :chutten]

I think that'll cover it.

To me the method of collection is the biggest question mark. We have a `profile` section in the Telemetry Environment[1] for the profile creation date, but a scalar[2] for any date of profile directory age scans.

For stub profile and first run date I would think we'll need to add them to the profile section of the Environment, which means managing and persisting them outside of the usual profile mechanism and code for storing and loading that information. It also means coming up with plans for corner cases (for instance, what happens when the profile is reset? And do we want some sort of integrity mechanism in case the user tries to duplicate the persisted profile information?). It also means updating the schema[3], and getting it included in relevant derived datasets (main_summary as the minimum, but I expect you, Su, have a longer list than that). And documentation and testing and Data Collection Review, of course.

If we're going to this effort it might be worth documenting some shortcomings of our current profile data collections and seeing if there are holes we can address "along the way" as it were.

[1]: https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/environment.html#profile
[2]: https://searchfox.org/mozilla-central/source/toolkit/modules/ProfileAge.jsm#122
[3]: https://github.com/mozilla-services/mozilla-pipeline-schemas/blob/dev/templates/include/telemetry/environment.1.schema.json

Comment 11

[Jan-Erik Rediger [:janerik]]

:chutten pretty much covered it all.

Comment 12

[Dave Townsend [:mossop]]

(In reply to Su-Young Hong from comment #9)
> Hi Dave, 
> 
> Thanks so much for meeting last week and explaining the profile creation
> changes to me. 
> 
> Like we discussed, the following telemetry would be very helpful for us to
> deal with this change. 
> 
> * is_stubprofile: 
> 	- indicates if a profile was created as a stub profile (as opposed to the
> primary profile)

Filed bug 1495794

> * profile_creation_date:
> 	- keep this probe the way it is (so no changes to it's current behavior)
> * first_run_date: 
> 	- field which indicates the timestamp of when the first run of the profile is.
> is. 

Filed bug 1495792

Comment 13

[Anthony Shipman]

The terms in the issue description are not clear to me.  What exactly is an installation or installation directory?  I normally download a tarball of a new version of firefox and unpack it into /usr/local/firefox-NN and I might then update the 'firefox' command to point to it.  Do these count as distinct installations?

Comment 14

[Dave Townsend [:mossop]]

(In reply to als from comment #13)
> The terms in the issue description are not clear to me.  What exactly is an
> installation or installation directory?  I normally download a tarball of a
> new version of firefox and unpack it into /usr/local/firefox-NN and I might
> then update the 'firefox' command to point to it.  Do these count as
> distinct installations?

Yes, because they are in different directories.

Comment 15

[Anthony Shipman]

Then everytime I upgrade I will have all settings erased. That doesn't sound like a good design.

Comment 16

[Anthony Shipman]

Will I be able to force a particular profile no matter what the installation directory?  

You'll need to make this clear to Linux distributions so that desktop short cuts have a fixed profile built into them, just in case the installation directory changes.

Comment 17

[Dave Townsend [:mossop]]

(In reply to als from comment #16)
> Will I be able to force a particular profile no matter what the installation
> directory?

Firefox will continue to respect -P and --profile command line arguments?

Comment 18

[Dave Townsend [:mossop]]

(In reply to Dave Townsend [:mossop] (he/him) from comment #17)
> (In reply to als from comment #16)
> > Will I be able to force a particular profile no matter what the installation
> > directory?
> 
> Firefox will continue to respect -P and --profile command line arguments?

Should have been no question mark there.

Comment 19

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

This looks like it could break package managers where the canonical path to the package's files includes a version number or other package instance identifier (similar to comment #13, but automated).  NixOS is a well-known example of this, but it also appears to be the case for our Snap package, which we directly support and promote (see https://snapcraft.io and bug 1297513).

I've verified that compatibility.ini in the profile of a Snap-managed Firefox contains paths with the Snap package revision number; if I understand this patch correctly, that means we'd create a new profile for every upgrade.

Comment 20

[Dave Townsend [:mossop]]

Attachment: Bug 1474285: Make sure to clear the default profile correctly when removing a profile. r=froydnj

Comment 21

[Dave Townsend [:mossop]]

Attachment: Bug 1474285: Only snatch the old default profile if it has not been explicitly selected by another install.

When setting the default profile for an install mark it as locked so another
install will not steal it. The exception is a non-OS default install selecting
the old default profile on startup where we want to allow an OS default install
to steal it if run.

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e69cac07b209
Implement dedicated profiles per install. r=froydnj, r=Gijs

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/58babf220962
Implement dedicated profiles per install. r=froydnj, r=Gijs

Comment 24

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/58babf220962

Comment 25

[Magnus Melin [:mkmelin]]

At some point there were discussions about having some runtime environment variable or similar to allow developers to opt out of profile per install. Did that get implemented? I can't find it.

Comment 26

[Dave Townsend [:mossop]]

(In reply to Magnus Melin [:mkmelin] from comment #25)

At some point there were discussions about having some runtime environment variable or similar to allow developers to opt out of profile per install. Did that get implemented? I can't find it.

I don't recall this. There is a way to build with it disabled.

Comment 27

[Magnus Melin [:mkmelin]]

Found it now, it's implemented: You just start with -allow-downgrade

Comment 28

[Gabriela [:gaby2300]]

I am sorry if this is not the correct place to make a comment regarding this new feature, but I recently made a new Nightly install. I got the message regarding the new profile.

I understood that syncing this profile with Firefox Account I would get back my previous Nightly windows and tabs.

This didn't happen.

I'd like to if I was mistaken and this is the expected behavior, or if it's a bug I should file. Thanks!

Comment 29

[Dave Townsend [:mossop]]

(In reply to Gabriela [:gaby2300] from comment #28)

I am sorry if this is not the correct place to make a comment regarding this new feature, but I recently made a new Nightly install. I got the message regarding the new profile.

I understood that syncing this profile with Firefox Account I would get back my previous Nightly windows and tabs.

This didn't happen.

I'd like to if I was mistaken and this is the expected behavior, or if it's a bug I should file. Thanks!

It should allow you to access the tabs on your other install through the synced tabs feature. If that isn't working please file a bug at https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox&component=Sync

Comment 30

[Gabriela [:gaby2300]]

(In reply to Dave Townsend [:mossop] (he/him) from comment #29)

It should allow you to access the tabs on your other install through the synced tabs feature. If that isn't working please file a bug at https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox&component=Sync

I surely used Sync so I think tabs should been synced too. I think I will file the bug. The worst that could happen is that it's invalid or wont fix. I would appreciate your thoughts!

Comment 31

[Gabriela [:gaby2300]]

Filed bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1533745

Comment 32

[mailinglists35]

this is not resolved for me.

I just installed nightly on windows and at the end of installer process if the checkbox to run is enabled (default), when I press finish it tells me firefox is already running.

nightly should automatically create the nightly profile even if regular firefox is running during nightly install.

Comment 33

[mailinglists35]

(In reply to mailinglists35 from comment #32)

nightly should automatically create the nightly profile even if regular firefox is running during nightly install.

correction: it seems to create the profile, but the installer checkbox issue is present. so in the installer i must uncheck the box to launch nightly, then only then I can finish the installing and it launches correctly from installed shortcuts

Comment 34

[Dave Townsend [:mossop]]

(In reply to mailinglists35 from comment #32)

this is not resolved for me.

I just installed nightly on windows and at the end of installer process if the checkbox to run is enabled (default), when I press finish it tells me firefox is already running.

nightly should automatically create the nightly profile even if regular firefox is running during nightly install.

Please file a new bug.