Open Bug 259356 Opened 20 years ago Updated 2 days ago

Support for the Freedesktop.org XDG Base Directory Specification

Categories

(Core :: XPCOM, enhancement)

Product:
Core
Component:
XPCOM
Platform:
All
Linux
Type:
enhancement

Tracking

()

Status:
REOPENED

People

(Reporter: contact, Assigned: gerard-majax)

References

(Depends on 5 open bugs, Blocks 4 open bugs,
URL
)

Details

(Keywords: parity-chrome)

Attachments

(2 files, 4 obsolete files)

Bug 259356 - Add support for the XDG Base Directory Specification r?stransky,glandium,gcp
46 bytes, text/x-phabricator-request
Details | Review
The XDG Base Directory Specification is based on the following concepts: There is a single base directory relative to which user-specific data files should be written. This directory is defined by the environment variable $XDG_DATA_HOME. There is a
5.47 KB, text/plain
Details 
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20040804 Firefox/0.9.3
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20040804 Firefox/0.9.3

Firefox should support [1] under Linux. This means using the variables defined
there for storing config files, data files and cache files. It would be nice for
a good desktop integration under X (Linux) to follow these specs (because I've
also noticed Firefox follows others too).

This might not require many lines in the code, but will require moving some
config & data files from ~/.phoenix to probably $XDG_CONFIG_HOME/phoenix and
$XDG_CACHE_HOME/phoenix (& even $XDG_DATA_HOME/phoenix), depending on the needs.

[1] http://freedesktop.org/Standards/basedir-spec (current version 0.6)

Reproducible: Always
Steps to Reproduce:
Hardware -> All, Severity -> enhancement, marking NEW

This is pretty low-level stuff, Product -> Browser?
URL: http://freedesktop.org
Severity: normal → enhancement
Status: UNCONFIRMED → NEW
Ever confirmed: true
Hardware: PC → All
Err... this is the first that came to my mind.
Sorry for not thinking globally but at the time I wrote the bug-report (actually
a feature)...

Yes, this should apply to all the Mozilla apps... unfortunately, I don't know
exactly which product that would be that matches all of them. I'll switch it to
"Browser" for now.

Note: if Browser reffers to the internals (cache etc) and Firefox to the
externals (GUI etc), maybe both should be selected. This is kind of tricky. I'll
leave the choice to someone more experienced than me in these projects.
Component: OS Integration → Browser-General
Product: Firefox → Browser
Version: unspecified → 1.0 Branch
(In reply to comment #2)
> Note: if Browser reffers to the internals (cache etc) and Firefox to the
> externals (GUI etc), maybe both should be selected. This is kind of tricky. I'll
> leave the choice to someone more experienced than me in these projects.

Sorry, that wasn't directed at you, but some of the higher-ups.

If made, this kindof change would probably be done in Mozilla, which would carry
over to Firefox. Just need some functions to return the initial starting point,
then append 'mozilla' or 'mozilla/firefox' to that.
Product: Browser → Seamonkey
Blocks: 140751
Well, no, not Browser/Suite. Since it's asking that we locate the profile and default download directory based on env vars, things which Firefox and Seamonkey do separately, choosing different places, there's no single place to cover both products.
Assignee: bugs → nobody
Component: General → OS Integration
Product: Mozilla Application Suite → Firefox
Version: 1.0 Branch → Trunk
Hmm.. then.. shouldn't this bug be duplicated for other Mozilla apps too?
This bug is quite important since Linux-users can't easily backup their config & data files if these files are not stored at the right place

see that article for an illustration : 
Cleaning user preferences, keeping user data
http://ploum.frimouvy.org/?184-cleaning-user-preferences-keeping-user-data
I tI think we depend on Bug 399500 here.
Depends on: 399500
Actually, it's more of a dupe... (technically the other should have been duped here, but that one has a patch already, so duping up...)
Status: NEW → RESOLVED
Closed: 16 years ago
No longer depends on: 399500
Resolution: --- → DUPLICATE
Could please check if these two bugs are about the same things ? 

Bug 399500 seems to be about choosing the good directory when firefox downloads a file, but xdg freedesktp specs are also about where installing date & configuration files when Firefox itself is installed :

The default for $XDG_CONFIG_HOME is $HOME/.config, the default for $XDG_DATA_HOME is $HOME/.local/share. So all applications should look for those environment variables and use those default values if the variables are not set.

that's why i've made comment#7 above concerning backup

Are this bug and bug 399500 really duplicates ?
In reply to Comment #10:
Yeah, that's why I said in bug 399500 comment 11 that I'm not even sure anymore that this bug depends on bug 399500. They are similar, but address different problems.
Ugh, not enough coffee.  I was looking at the summary and ignored what the bug was actually about.  Fixing summary to be more explicit and unduping.  Sorry for the noise.
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Summary: Support for the Freedesktop.org XDG variables → Support for the Freedesktop.org XDG basedir variables
Here, let's make the summary even better, and fix the URL field.
URL: http://freedesktop.org → http://freedesktop.org/wiki/Specifica...
Summary: Support for the Freedesktop.org XDG basedir variables → Support for the Freedesktop.org XDG Base Directory Specification
well i think that the title and the description of this bug are both ok, whereas the title og bug 399500 is unclear 
(Actually making use of these directories for the profile/local profile paths will be a further patch.)
Attachment #311040 - Flags: review?(caillon)
Now, it is necessary to define which files should go to which directories.

For example, for Firefox, ~/.mozilla/firefox/<profile>/ currently contains a mess of configuration, data and cache files that must be separated and organized into something like:

$XDG_CONFIG_HOME/firefox/<profile>/   (config files)
$XDG_DATA_HOME/firefox/<profile>/     (data files)
$XDG_CACHE_HOME/firefox/<profile>/    (browser cache)

It is easy to define what should go to the cache directory, but for everything else, there are some ambiguities about what is configuration and what is data.

Extensions, dictionaries, chrome and password files are a few things that may fit in either directory, depending on how you consider them.
Bug 399498 -  use XDG downloads dir as default downloads dir  
has been fixed and it's great 

But what about the present bug which concerns where installing data &
configuration files when Firefox itself is installed ?
In a similar vein to Comment 7, an additional issue that this bug causes is that backup programs like Deja Dup automatically exclude the XDG_CACHE_HOME directory, whereas the pathname for Firefox's cache is always different.
https://answers.launchpad.net/deja-dup/+question/99015
Also, Firefox should use $XDG_CACHE_HOME for both the cache folder and the urlclassifier3.sqlite folder.
i have applied that patch, but firefox still using ~/.mozilla. anyone have workable patch to get rid of ~/.mozilla?
That would be great if a fix could happen for Firefox 6 (since Firefox 5 is about to get its code frozen)
Integration within Linux is important and this would allow better backups and so on
Thanks in advance
(In reply to comment #19)
> Also, Firefox should use $XDG_CACHE_HOME for both the cache folder and the
> urlclassifier3.sqlite folder.

Yes, it really should. urlclassifier3.sqlite can be a pain for anyone who has to deal with a home directory with a quota applied to it.

Firefox should put everything which it will currently put in to XRE_PROFILE_LOCAL_PATH (if you can persuade Firefox to use it) in to a sub directory of XDG_CACHE_HOME. The Mac OS X version of Firefox effectively does this already and has done for years. It puts Cache, urlclassifier etc  in  ~/Library/Caches/Firefox


Someone has recently submitted a patch for using XDG_CACHE_HOME on a different bug https://bugzilla.mozilla.org/show_bug.cgi?id=239254#c8
(In reply to comment #23)
> Someone has recently submitted a patch for using XDG_CACHE_HOME on a
> different bug https://bugzilla.mozilla.org/show_bug.cgi?id=239254#c8

It was me :) I'm currently waiting for the review.
I'm using firefox with the patch from Bug 239254 applied and have the following files under ~/.cache/mozilla/firefox/<salt>.<profile_name>
  Cache/
  startupCache/
  urlclassifier3.sqlite
  XUL.mfasl
The above directory is seen as a "local data directory" just like if it was specified by XRE_PROFILE_LOCAL_PATH, but the patch works for all profiles.

While the patch attached in comment #15 exposes XDG-related directories to the rest of the code, the patch in bug 239254 actually determines the path to use (by default). When the former is committed the later may make use of new API, but currently it's just "hard-coded" to use XDG_CACHE_HOME environment variable and fall back to "~/.cache".
(In reply to comment #16)
> It is easy to define what should go to the cache directory, but for
> everything else, there are some ambiguities about what is configuration and
> what is data.
> 
> Extensions, dictionaries, chrome and password files are a few things that
> may fit in either directory, depending on how you consider them.

I remember running across a good explanation of how to look at the split a couple of months ago.

"config" is stuff that is more likely to be specific to a given system and, if you lose it, it's not the end of the world. (Basically, stuff that may be omitted from a roaming profile under certain circumstances)

"data" is stuff that's irreplaceable and should always roam if at all possible. (eg. bookmarks, saved sessions, and so on)

I'd say that, based on that rationale, password stores, personal dictionaries, and the like are definitely data and a decent case could be made for putting any extensions without platform-specific components in data as well.
While this issue is set to "platform: all linux", this issue applies to all unix-like OSs, except MAC OS X (I'm mentioning this just in case this gets commited to somewhere that excludes other OSs).

By the way, the latest spec is 0.7: http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

I've also seen no mention of $XDG_RUNTIME_DIR. Just in case, this is where user-specific runtime files should be stored (ie: sockets).
Blocks: 726939
The same issue has been raised for Thunderbird as Bug 735285.
Blocks: 735285
Just giving this a bump. Is this still on the agenda?
* ping
Blocks: 818686
Well... Just bumping this again !
According to XDG Base directory specification, Firefox should not have its own folder anymore:
User data should go into $XDG_DATA_HOME (which default to ~/.local/share),
user preferences should go into $XDG_CONFIG_HOME (which default to ~/.config)
and cached data should go to $XDG_CACHE_HOME (which default to ~/.cache).
More details at :
http://ploum.net/post/207-modify-your-application-to-use-xdg-folders
https://live.gnome.org/GnomeGoals/XDGConfigFolders

Full specification can be found at: 
http://standards.freedesktop.org/basedir-spec/latest/

The Freedesktop.org XDG base directory specification have good de facto adoption.
It has been adopted by:
- GNOME ( https://live.gnome.org/GnomeGoals/XDGConfigFolders )
- GTK+ ( https://bugzilla.gnome.org/show_bug.cgi?id=646631 )
- KDE ( http://techbase.kde.org/KDE_System_Administration/XDG_Filesystem_Hierarchy#Freedesktop.org_and_Standard_Directories )
- QT ( http://harmattan-dev.nokia.com/docs/library/html/qt4/qsettings.html#setPath )
- XFCE ( http://docs.xfce.org/xfce/xfce4-session/advanced in Files and Environment Variables )
- LXDE
- Razor-qt
- VLC ( https://trac.videolan.org/vlc/ticket/1267 )
- GStreamer ( https://bugzilla.gnome.org/show_bug.cgi?id=518597 )
- Chrome ( http://code.google.com/p/chromium/issues/detail?id=16976 )
- many more upstream applications
- Ubuntu ( http://brainstorm.ubuntu.com/idea/6557/ & http://packages.ubuntu.com/fr/source/precise/libxdg-basedir )
- Debian ( http://packages.debian.org/squeeze/libxdg-basedir1 )
- Red Hat
- Fedora
- Suse
- many more distributions

I think that Firefox should use same locations than the vast majority of Desktop environment and applications.


There are real advantages of following this specification :
- a lot less cluttered $HOME
- Make backups a lot more safer and easier.
  Backuping your $XDG_DATA_HOME along with your files is enough 
  (or just excluding $XDG_CACHE_HOME)
- A lot easier to reset a default configuration if you want/need it (and 
  without any risk to loose informations). Even for the software itself 
  could choose to reset $XDG_CONFIG_HOME if needed.
- Avoid some strange bugs that happens because you had a old version of 
  some configuration file
- A lot more of flexibility and portability because no path are hardcoded.
The problem with XDG Base Directory Specification is that broken the copy of profile. 
When we swift windows to Linux, it's more easy to copy all %APPDATA%\Mozilla\Firefox\Profiles\ to ~/.mozilla before the first start of Firefox on the new GNU/Linux distribution. The new user will keep all this preferences. It's a good help to give this switch smooth.

And some users have a dual-boot with a profile share between windows and GNU/Linux, XDG Base Directory Specification could break i too.

I don't know if Microsoft have similar recommendation for windows
(In reply to vulcain from comment #33)
> I don't know if Microsoft have similar recommendation for windows

They do have the Local/Roaming split which could be roughly equated to putting everything in either $XDG_DATA_HOME or $XDG_CACHE_HOME like most Humble Bundle Linux game ports do.
(In reply to vulcain from comment #33)
> The problem with XDG Base Directory Specification is that broken the copy of
> profile. 
> When we swift windows to Linux, it's more easy to copy all
> %APPDATA%\Mozilla\Firefox\Profiles\ to ~/.mozilla before the first start of
> Firefox on the new GNU/Linux distribution. The new user will keep all this
> preferences. It's a good help to give this switch smooth.
> 
> And some users have a dual-boot with a profile share between windows and
> GNU/Linux, XDG Base Directory Specification could break i too.
> 
> I don't know if Microsoft have similar recommendation for windows

Code will have to be written to migrate the current setting to their new location.

This code can be kept around, so when a firefox starts and finds stuff in .mozilla, it will move it to their proper location.
This will handle the scenario you propose just fine.
(In reply to Stephan Sokolow from comment #34)
> (In reply to vulcain from comment #33)
> > I don't know if Microsoft have similar recommendation for windows
> 
> They do have the Local/Roaming split which could be roughly equated to
> putting everything in either $XDG_DATA_HOME or $XDG_CACHE_HOME like most
> Humble Bundle Linux game ports do.

Actually, cache and temporary files SHOULD NOT be in roaming, while user data and config SHOULD be.
(In reply to Hugo Osvaldo Barrera from comment #36)
> (In reply to Stephan Sokolow from comment #34)
> > (In reply to vulcain from comment #33)
> > > I don't know if Microsoft have similar recommendation for windows
> > 
> > They do have the Local/Roaming split which could be roughly equated to
> > putting everything in either $XDG_DATA_HOME or $XDG_CACHE_HOME like most
> > Humble Bundle Linux game ports do.
> 
> Actually, cache and temporary files SHOULD NOT be in roaming, while user
> data and config SHOULD be.

That's what I meant. Since I didn't say "respectively", I didn't think to pay attention to the order in which I listed the XDG variable names.
(In reply to vulcain from comment #33)
> The problem with XDG Base Directory Specification is that broken the copy of
> profile. 
> When we swift windows to Linux, it's more easy to copy all
> %APPDATA%\Mozilla\Firefox\Profiles\ to ~/.mozilla before the first start of
> Firefox on the new GNU/Linux distribution. The new user will keep all this
> preferences. It's a good help to give this switch smooth.
> 
> And some users have a dual-boot with a profile share between windows and
> GNU/Linux, XDG Base Directory Specification could break i too.

With Firefox Sync built in by default, and other sync engines available, is it really such a big problem?
(In reply to Rimas Kudelis from comment #38)
> (In reply to vulcain from comment #33)
> > The problem with XDG Base Directory Specification is that broken the copy of
> > profile. 
> > When we swift windows to Linux, it's more easy to copy all
> > %APPDATA%\Mozilla\Firefox\Profiles\ to ~/.mozilla before the first start of
> > Firefox on the new GNU/Linux distribution. The new user will keep all this
> > preferences. It's a good help to give this switch smooth.
> > 
> > And some users have a dual-boot with a profile share between windows and
> > GNU/Linux, XDG Base Directory Specification could break i too.
> 
> With Firefox Sync built in by default, and other sync engines available, is
> it really such a big problem?

I agree. Worrying about shared profiles is a waste of time since sharing through sync should be the supported option.
Separating data in local/roaming profile is bug 239254
(In reply to Rimas Kudelis from comment #38)
> 
> With Firefox Sync built in by default, and other sync engines available, is
> it really such a big problem?

When i help to switch i don't want to use a web tool just for get history, preference & Co one time. Copy profile or export/import bookmarks are more easly.

Sync is a good tool for an user with a lot PC or Android but not for an extemporaneous use.

After, some people who prefer use a partition to share Firefox and Thunderbird instead of an Internet’s server (aka the cloud). That their choice.

After, it's not a big problem, just a regression, i will use export/import (and need to launch windows only to do the export T_T )
Comment on attachment 703317 [details] [diff] [review]
Updated to latest trunk

Benjamin, can you check it please? When it gets in I'll update patch in Bug 239254 for that.
Attachment #703317 - Flags: review?(benjamin)
Attachment #311040 - Flags: review?(caillon)
Component: Shell Integration → General
Product: Firefox → Core
Comment on attachment 703317 [details] [diff] [review]
Updated to latest trunk

Review of attachment 703317 [details] [diff] [review]:
-----------------------------------------------------------------

Well... I'm probably of absolutely no use as I never really developped for firefox, but here is what I understood :
 * A bunch of new "atoms" is defined ;
 * A lot of new functions is defined ;
 * Some code is written if the argument atom is one of the new ones ;
 * Some other code is written in case PR_GetEnv("HOME") fails ; but just returns an error (BTW, wouldn't that belong to a second patch, separated, fixing another bug ?) ;

But where is the code that actually uses all of these new functions ? In which way does the patch solve the mentioned issue ?
Is the code re-dispatching the files (using the new "atoms") left to be implemented ?

Hoping I didn't tell nonsense,

Leo Gaspard
Attachment #703317 - Flags: review?(benjamin) → review?(mh+mozilla)
Comment on attachment 703317 [details] [diff] [review]
Updated to latest trunk

Review of attachment 703317 [details] [diff] [review]:
-----------------------------------------------------------------

Why do you need these to be exposed through the directory service with specific atoms?
(In reply to Mike Hommey [:glandium] from comment #45)
> Why do you need these to be exposed through the directory service with
> specific atoms?

Actually I don't need that, but isn't that a way how to do it? For instance for the patch in Bug 239254 - nsXREDirProvider::GetUserDataDirectoryHome. We have to use a hack there for now. And I guess another step is to move ~/.mozilla or ~/.thunderbird to ~/.config on linux.
(In reply to Martin Stránský from comment #46)
> (In reply to Mike Hommey [:glandium] from comment #45)
> > Why do you need these to be exposed through the directory service with
> > specific atoms?
> 
> Actually I don't need that, but isn't that a way how to do it? For instance
> for the patch in Bug 239254 - nsXREDirProvider::GetUserDataDirectoryHome. We
> have to use a hack there for now.

If there's only one place where each is going to be used, I'm not sure of the added value.

> And I guess another step is to move
> ~/.mozilla or ~/.thunderbird to ~/.config on linux.

I think we shouldn't move existing profiles, but only create new ones in the new location.
(In reply to Mike Hommey [:glandium] from comment #47)
> If there's only one place where each is going to be used, I'm not sure of
> the added value.

The added value is that the code will use a standard way how to get the directories and not some in-place hack. But yes, you're right, the .cache or .config dir is not used by mozilla yet. 

Actually it looks like there's something similar in GetUnixXDGUserDirectory() as a part of nsDirectoryService.cpp but for folders like Downloads, Music and so.

> > And I guess another step is to move
> > ~/.mozilla or ~/.thunderbird to ~/.config on linux.
> 
> I think we shouldn't move existing profiles, but only create new ones in the
> new location.

Sure, sounds reasonable.
Most software I know, that added a compatibility patch for the XDG Config Spec also included some migration and compatibility code detect and move old profiles to the new location and in that process split up its contents.

Also please note, that it is not just about moving the whole profile to ~/.config (or $XDG_CONFIG_HOME) but also to separate the contents into CONFIG, CACHE and DATA parts, such that it becomes easier for example to clear the Mozilla cache (CACHE) or reset the preferences (CONFIG), without deleting the bookmarks or saved passwords (DATA).

If the profiles were not moved but only new ones are created in the new location, then Mozilla would require code to read from both the old and new locations. In the case the profiles are moved it is enough to keep the migration code long enough (e.g until the end of the ESR support period) and then remove it again.

P.S. Many thanks to the persons working on this bug, it is very very appreciated.
(In reply to sbastig from comment #49)
> Most software I know, that added a compatibility patch for the XDG Config
> Spec also included some migration and compatibility code detect and move old
> profiles to the new location and in that process split up its contents.
> 
> Also please note, that it is not just about moving the whole profile to
> ~/.config (or $XDG_CONFIG_HOME) but also to separate the contents into
> CONFIG, CACHE and DATA parts, such that it becomes easier for example to
> clear the Mozilla cache (CACHE) or reset the preferences (CONFIG), without
> deleting the bookmarks or saved passwords (DATA).

And you don't need to move the entire profile to do that. Again, bug 239254 does this for cache.

> If the profiles were not moved but only new ones are created in the new
> location, then Mozilla would require code to read from both the old and new
> locations.

No, there would only need to be code returning the old or the new location from the directory service.

Also note the profile contains a lot of things, and that there's no distinction between config and data at the moment, so moving things under .config would just move them for the sake of it, which is not helpful. Moving the caches is already a big step forward, but this is not what this bug is about.
Comment on attachment 703317 [details] [diff] [review]
Updated to latest trunk

I think we don't need these exposed through the directory service. It doesn't seem to me there's more need than to expand GetUnixXDGUserDirectory to return appropriate values for existing directory service atoms. That would help make the patch in bug 239254 feel less hackish.
Attachment #703317 - Flags: review?(mh+mozilla) → review-
(In reply to Mike Hommey [:glandium] from comment #47)
> (In reply to Martin Stránský from comment #46)
> > (In reply to Mike Hommey [:glandium] from comment #45)
> > > Why do you need these to be exposed through the directory service with
> > > specific atoms?
> > 
> > Actually I don't need that, but isn't that a way how to do it? For instance
> > for the patch in Bug 239254 - nsXREDirProvider::GetUserDataDirectoryHome. We
> > have to use a hack there for now.
> 
> If there's only one place where each is going to be used, I'm not sure of
> the added value.
> 
> > And I guess another step is to move
> > ~/.mozilla or ~/.thunderbird to ~/.config on linux.
> 
> I think we shouldn't move existing profiles, but only create new ones in the
> new location.

I don't think that's a good idea.
If users know that firefox is XDG-Basedir compliant, they'll just happily backup ~/.config when needing to backup (for example, to install a different OS).

This will result in data loss for users who didn't know their profiles were still in an older location.

Also, keeping older profiles in an older location means a lot of code to handle both layouts. If we move to the XDG-Basedir directories, the old code can be deleted, and we can just keep the migration code, and the code that deals with the new locations.

Finally, I don't think firefox can claim to be XDG-Basedir compliant if *some* profiles are still managed elsewhere.
I totally agree with Hugo in this point. Mozilla software should aim to be fully XDG-Basedir compliant and not leave behind any legacy folders.
Mike, if I understand correctly you propose to close this bug and get the XDG directories separately in each case (.config, .cache and so).
(In reply to Martin Stránský from comment #54)
> Mike, if I understand correctly you propose to close this bug and get the
> XDG directories separately in each case (.config, .cache and so).

We can keep this bug open as a tracker bug.
Blocks: 239254
(In reply to Hugo Osvaldo Barrera from comment #52)
> I don't think that's a good idea.
> If users know that firefox is XDG-Basedir compliant, they'll just happily
> backup ~/.config when needing to backup (for example, to install a different
> OS).
> 
> This will result in data loss for users who didn't know their profiles were
> still in an older location.

Far from all applications are XDG complaint. Someone backuping .config only will have a lot of data loss other than their Firefox profile.

> Also, keeping older profiles in an older location means a lot of code to
> handle both layouts. If we move to the XDG-Basedir directories, the old code
> can be deleted, and we can just keep the migration code, and the code that
> deals with the new locations.

See comment 50.
(In reply to Mike Hommey [:glandium] from comment #56)
> (In reply to Hugo Osvaldo Barrera from comment #52)
> > I don't think that's a good idea.
> > If users know that firefox is XDG-Basedir compliant, they'll just happily
> > backup ~/.config when needing to backup (for example, to install a different
> > OS).
> > 
> > This will result in data loss for users who didn't know their profiles were
> > still in an older location.
> 
> Far from all applications are XDG complaint. Someone backuping .config only
> will have a lot of data loss other than their Firefox profile.

If a user finds out firefox IS XDG-complaint, he'd ignore .firefox (assuming it was just old stuff), and backup .config; that's my point.

I've seen some software that copied it's data to XDG-Basedir locations, and left the old data where it was. When backing up, I'd ignore the old data, assuming it was deprecated.

> 
> > Also, keeping older profiles in an older location means a lot of code to
> > handle both layouts. If we move to the XDG-Basedir directories, the old code
> > can be deleted, and we can just keep the migration code, and the code that
> > deals with the new locations.
> 
> See comment 50.

Wrong, you won't need code to just read from the new (or old) location; the layouts inside each of is completely different.
Being XDG-Basedir compliant means some stuff is in ~/.cache, some stuff is in ~/.config, etc, NOT all together in a single directory. I'd also expect data to be organized differently inside each directory.
Because Firefox does not currently make any distinction between "data" and "config", and we really don't have plans to support that distinction in the future, I think that we should store the entire "roaming" profile as config, and the "local" profile as cache. We should not treat any Firefox things as data.

I agree that we should *not* move existing profiles.
(In reply to Hugo Osvaldo Barrera from comment #57)
> Wrong, you won't need code to just read from the new (or old) location; the
> layouts inside each of is completely different.
> Being XDG-Basedir compliant means some stuff is in ~/.cache, some stuff is
> in ~/.config, etc, NOT all together in a single directory. I'd also expect
> data to be organized differently inside each directory.

Things that would go in ~/.cache are currently in ~/.mozilla/firefox ;
Things that would go in ~/.config are currently in ~/.mozilla/firefox ;
etc.

You can's say "move the profile under ~/.config" and be done with it, obviously. You need to say: "this goes in ~/.config", "that goes in ~/.cache", etc.
But in the end, this and that both used to be in ~/.mozilla/firefox. There's nothing inherently difficult is supporting both locations. The difficulty is actually to put everything in the right place. And since there are many such things, and we add new ones regularly, you can be sure it won't happen in a snap of fingers, and even better, considering the windows and OSX code don't have distinction between config and data, you can be sure there will be mistakes, too.
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #58)
> Because Firefox does not currently make any distinction between "data" and
> "config", and we really don't have plans to support that distinction in the
> future, I think that we should store the entire "roaming" profile as config,
> and the "local" profile as cache. We should not treat any Firefox things as
> data.
> 

What are the "roaming" and "local" profiles?  As far as I can see, there's just a single profile ATM, stored in ~/.firefox/

I don't think configuration and cache should be stored together. Cache can grow a lot, while config is usually a few megabytes. Being unable to separate both while backing up makes backups grow a lot.

I also see no point in stuffing anything firefox-related into a single directory when we have proper locations for each type of file.
(In reply to Hugo Osvaldo Barrera from comment #60)
> What are the "roaming" and "local" profiles?  As far as I can see, there's
> just a single profile ATM, stored in ~/.firefox/

Windows and OSX code handle cache and other config/data separately. Linux code doesn't currently. That's bug 239254 (again).
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #58)
> Because Firefox does not currently make any distinction between "data" and
> "config", and we really don't have plans to support that distinction in the
> future, I think that we should store the entire "roaming" profile as config,
> and the "local" profile as cache. We should not treat any Firefox things as
> data.

If the firefox code "knows" what is config and what is data, then it should not be difficult to dedicate different locations for either of them on linux, but use the same one on Windows and Mac, for example on Windows both CONFIG and DATA would go into the same directory but could still have different subfolders (e.g AppData/RoamingProfile/firefox/{data,config}, where as on Linux they would go into completely separate folders (e.g. ~/.config/firefox/ and ~/.local/share/firefox). But if there is know plan to make such a distinction known to Firefox, then there is really no much to discuss in this bug. Just wait until the Caching  bug is closed and move/copy what ever is left to either ~/.config or ~/.local/share.

IMHO the interesting thing to discuss would be, why such a distinction is not planed. I think there is plenty of stuff that could be separated for example Passwords and Bookmarks (DATA) vs. Preferences and Extensions (CONFIG).

Of course the caching bug is a separate bug, so lets not discuss that again here.

@Hugo:
> What are the "roaming" and "local" profiles?  As far as I can see, there's
> just a single profile ATM, stored in ~/.firefox/
"Roaming" is a Windows Term und refers to the part of the users directory that can be copied to and from a server (for example in an active directory environment). Where as the "local" refers to a folder in the users directory that only exists in the local computer. So if applications want their configuration and data to be available on different computers, then they need to put that data into the "roaming" directory. Caching data is usually saved locally, to avaid coping potentially huge amounts of data to a server and back when the users logs on/off.
I have as module owner decided that we do not plan on trying to separate "config" and "data". The distinction is blurry to begin with: bookmarks are sure to be one mans data and another mans config, as are things like root certificates, certificate overrides, and other things. Because no other platform makes the distinction, coders working on the project are unlikely to know or care about the difference, leading to subtle XDG-only bugs.
IMHO, a means to distinguish between data and config is to ask what happens if the data is deleted. For cache data it is obviously recreated automatically and the user might only see a short slowdown. For config data the user does not loose anything that he can't easily recreate, e.g. he can configure the preferences again, add certificates again etc.
Data is everything that if deleted is permanently lost (e.g. if the user does not remember his password or his bookmarked places, then he cannot recreate them). Extensions are a bit difficult, because they can be easily installed again, but the user needs to remember which modules he had previously installed. A solution would be to store the names of extensions in DATA and if config is deleted, then ask the user if he wants to install previously installed extensions.

Since on platforms where there is not distinction between config and data we could still put everything into a common parent folder but then create different subfolders, so I don't think the XDG-only bugs argument will be a big problem.
(In reply to sbastig from comment #62)
> (In reply to Benjamin Smedberg  [:bsmedberg] from comment #58)
> > Because Firefox does not currently make any distinction between "data" and
> > "config", and we really don't have plans to support that distinction in the
> > future, I think that we should store the entire "roaming" profile as config,
> > and the "local" profile as cache. We should not treat any Firefox things as
> > data.
> 
> If the firefox code "knows" what is config and what is data, then it should
> not be difficult to dedicate different locations for either of them on
> linux, but use the same one on Windows and Mac, for example on Windows both
> CONFIG and DATA would go into the same directory but could still have
> different subfolders (e.g AppData/RoamingProfile/firefox/{data,config},
> where as on Linux they would go into completely separate folders (e.g.
> ~/.config/firefox/ and ~/.local/share/firefox). But if there is know plan to
> make such a distinction known to Firefox, then there is really no much to
> discuss in this bug. Just wait until the Caching  bug is closed and
> move/copy what ever is left to either ~/.config or ~/.local/share.

Data and config are completely different things, that's why they each have their location.  Even if the cache is moved elsewhere, data and config need to be separated.

> 
> IMHO the interesting thing to discuss would be, why such a distinction is
> not planed. I think there is plenty of stuff that could be separated for
> example Passwords and Bookmarks (DATA) vs. Preferences and Extensions
> (CONFIG).
> 
> Of course the caching bug is a separate bug, so lets not discuss that again
> here.
> 
> @Hugo:
> > What are the "roaming" and "local" profiles?  As far as I can see, there's
> > just a single profile ATM, stored in ~/.firefox/
> "Roaming" is a Windows Term und refers to the part of the users directory
> that can be copied to and from a server (for example in an active directory
> environment). Where as the "local" refers to a folder in the users directory
> that only exists in the local computer. So if applications want their
> configuration and data to be available on different computers, then they
> need to put that data into the "roaming" directory. Caching data is usually
> saved locally, to avaid coping potentially huge amounts of data to a server
> and back when the users logs on/off.

I was familiar with roaming/local in windows NT enviroments; I though you were refering to something *nix-related so I got confused.

So why is it ok for firefox to conform to windows standard, but so much debate when it comes to implementing *nix standards?
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #63)
> I have as module owner decided that we do not plan on trying to separate
> "config" and "data". The distinction is blurry to begin with: bookmarks are
> sure to be one mans data and another mans config, as are things like root
> certificates, certificate overrides, and other things. Because no other
> platform makes the distinction, coders working on the project are unlikely
> to know or care about the difference, leading to subtle XDG-only bugs.

The line is not blurry at all. Bookmarks are not configuration. You don't "configure" firefox with your bookmarks, they're clearly data.

I find it rather sad that this would be closed as WONTFIX - standard directories are respected for other OSs, so I think this is another example of firefox giving the linux community the "we don't care" response by not adjusting itself to XDG specifications.
(In reply to Hugo Osvaldo Barrera from comment #66)
> The line is not blurry at all. Bookmarks are not configuration. You don't
> "configure" firefox with your bookmarks, they're clearly data.

The line is so not blurry that epiphany puts bookmarks in .config/epiphany, not .local/share/epiphany.
That does not make it more right, a bug should be filed for epiphany to correct that.
This is not a question of right and wrong, it is a question of engineering a cross-platform application. Because the distinction between data and config is not a concept on any of our other platforms, it is not something that engineers working on Firefox will think about. That means they will use the wrong directory and there will be subtle Linux-only bugs. (And the distinction is definitely still blurry).

Because of this engineering reality, I have made the decision that we will not support this distinction, and will store everything in one place. Given the current distinction, I believe that place will be "data".
My Two Cents

Microsoft does it's own thing. If software is to be run on a Microsoft OS, it conforms to Microsoft.

Google Chrome implements the XDG standard, but apparently does not distinguish between data and configuration. Google Chrome cache files are in ~/.cache/{Profile Name}/Cache and ~/.cache/{Profile Name}/MediaCache. All other Google Chrome user files are stored in ~/config/Google Chrome.

The cache location should be discussed here because it is part of the standard. When discussing the standard, we should include Firefox, SeaMonkey and Thunderbird. While incorporating the standard, we should do something about the legacy mozilla directory. 

I do think that some folders are quite obvious as data. Add-ons and plug-ins should be stored in the data directory. Additionally, GUI folders like chrome and other files that were historically in the Netscape application data directory but were later moved to the profile directory. This does not include configuration files such as prefs.js that should be in the configuration directory.

Mail that is stored on the computer should be in the data directory, while mail configuration should be in the configuration directory. Cached emails should be in the cache directory.

From reading the newest specification from http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html, it sounds like the most volatile date should be stored in the cache directory, the least volatile data that is set or changed by the application should be stored in the data directory (~/.local/share), and items configured by the user should be stored in the configuration directory.

It seems that data that can be synced and therefore roam from computer to computer is in the configuration directory. Data that is more specific to the current computer and operating system appear to belong in the data directory.

My Recommendation

If it is too difficult to separate application data from configuration data, that data could remain in the configuration data. However, I recommend at least separating plug-ins and add-ons.

The firefox directory should be removed from the mozilla directory. The mozilla directory may remain separately but be used only for data that can be used or ignored by other mozilla based applications.

Other gecko applications should still be able to read the mozilla directory for plug-ins.

Bookmarks could also be stored in the mozilla folder and made readable by other browser applications. The format could be HTML format or the new XBEL standard (http://pyxml.sourceforge.net/topics/xbel/docs/html/xbel.html). The actual format or availability of bookmarks in this directory can be up to those working on bookmarks for the Mozilla core.

Migration

The installation script must look for the existence of ~/.config/{Application Name}.
(In reply to Thomas Sisson from comment #70)
> 
> Mail that is stored on the computer should be in the data directory, while
> mail configuration should be in the configuration directory. Cached emails
> should be in the cache directory.
> 

My one caution there would be that, for ease of backup, it'd probably be a good idea to consider all config required to access the data as data itself. Otherwise, it'd be like storing an MP3's raw data stream in one place and the frame headers required to decode it in another.

Lose those frame headers, and it's a ridiculous amount of trial-and-error to restore your backups.

> 
> It seems that data that can be synced and therefore roam from computer to
> computer is in the configuration directory. Data that is more specific to
> the current computer and operating system appear to belong in the data
> directory.
> 

Where did you get that impression? Everything I've read and heard tells me it was under the impression it was the other way around.

(Configuration data being stuff that is easier to replace and more machine-specific. Data being less machine-specific and harder to replace.)

Think of it in the context of something like Thunderbird address books vs. something like what SMTP server a given machine uses or some saved window geometry. 

Your address book is at least as important and system-agnostic as a LibreOffice document but you don't access it through an open dialog, so it goes under data where tools will prioritize backing it up.

Your configuration, like what SMTP server you use, may be different on different PCs and, while it'll be a pain to re-enter, it's not the end of the world if you lose it.

> If it is too difficult to separate application data from configuration data,
> that data could remain in the configuration data. However, I recommend at
> least separating plug-ins and add-ons.

The only way I can see that making sense is if you consider plugins and addons to be "configuration". Otherwise, what's there to separate when things like bookmarks belong under "data"?
We are not going to separate config and data. That discussion is closed, and further discussion of it is unwelcome in this bug. I have delegated to glandium the decision about whether we store our profile data in the config or data XDG subtree.
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #72)
> We are not going to separate config and data. That discussion is closed, and
> further discussion of it is unwelcome in this bug. I have delegated to
> glandium the decision about whether we store our profile data in the config
> or data XDG subtree.

I remain confused. Does Mozilla support GNU/Linux, or doesn't it?

It seems that you put attention into the fact that the code builds, and package it, but you don't adapt to the OS's standard. (Though you *do* for other Operating Systems).

Are you slowly dropping support for GNU/Linux? Because you either support, or don't, it confuses users if you say you do, and then ship half-ported software.
Migration (continued)

Sorry I'm so long-winded.

The installation script would also look for the existence of ~/.mozilla/{Application Name} or ~/.{Application Name} if the new profile location is not found. If no profiles are found, the profile directory would be created in the XDG compliant location. Migration would be similar to the old Netscape migration. The post-installation would offer the user the option to keep the old profile(s) or delete them when finished. For deletion, a warning would be made that this cannot be undone and that this change is not backwards compatible for previous versions. For the option to keep the the old profile, the user is told that any new changes will not show up in the old profile.

The option to migrate and delete old profiles should be made available through multiple updates but eventually be removed. At that time download pages would explain that old profiles would not be used. Any users who have not updated or upgraded the application would be able to find information on how to migrate old profiles.

Someone could choose to write an add-on that would migrate old profiles at any time.

When this is implemented, it provides an opportunity to implement any major changes that would not be compatible with previous versions of the application.
(In reply to Hugo Osvaldo Barrera from comment #73)
> 
> I remain confused. Does Mozilla support GNU/Linux, or doesn't it?
> 
> It seems that you put attention into the fact that the code builds, and
> package it, but you don't adapt to the OS's standard. (Though you *do* for
> other Operating Systems).
> 
> Are you slowly dropping support for GNU/Linux? Because you either support,
> or don't, it confuses users if you say you do, and then ship half-ported
> software.

Please don't use those "you're either with us or against us" tactics. They poison the discourse and, if people did rise to your bait, they'd be more likely to drop Linux support to spite you than do what you want.

As a Linux user, I honestly don't care that much. The config-data divide is somewhat orthogonal to how a browser does things and, given how much of Firefox's profile is data, splitting out the small parts that are config would probably harm more than it helps.

As long as Firefox properly separates out content meant for XDG_CACHE_HOME and XDG_RUNTIME_DIR (user-specific locks and sockets), I have no problem with profiles just being plopped into XDG_DATA_HOME.

My main concerns are:

1. Backup tools should be able to offer sane defaults for things like excluding caches and, even if I do configure manually, I should have to babysit the config file as little as possible as I install and remove applications.

2. If an application gets wedged and concludes that another copy still exists when it doesn't, there should be one standard place I look for stale locks and sockets.

3. There shouldn't be 50 million different folders in my homedir when I choose to show hidden files.
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #72)
> We are not going to separate config and data. That discussion is closed, and
> further discussion of it is unwelcome in this bug. I have delegated to
> glandium the decision about whether we store our profile data in the config
> or data XDG subtree.

I was only offering "My Two Cents" as my opinion. I had not realized such discussion was deemed closed. I further acknowledge the difficulty of separating the two and suggested the config XDG subtree.

I don't know who you are, what you contribute to the project, or what status you hold in relation to this bug, so I say this with trepidation and as much respect as possible intended. If you don't wish to hear about what people think about which is which kind of data, couldn't you check the box labeled "Ignore Bug Mail?" I do see your point though, and leave that to glandium for implementation.
(In reply to Stephan Sokolow from comment #71)
> (In reply to Thomas Sisson from comment #70)
> > 
> > Mail that is stored on the computer should be in the data directory, while
> > mail configuration should be in the configuration directory. Cached emails
> > should be in the cache directory.
> > 
> 
> My one caution there would be that, for ease of backup, it'd probably be a
> good idea to consider all config required to access the data as data itself.
> Otherwise, it'd be like storing an MP3's raw data stream in one place and
> the frame headers required to decode it in another.
> 
> Lose those frame headers, and it's a ridiculous amount of trial-and-error to
> restore your backups.
> 
> > 
> > It seems that data that can be synced and therefore roam from computer to
> > computer is in the configuration directory. Data that is more specific to
> > the current computer and operating system appear to belong in the data
> > directory.
> > 
> 
> Where did you get that impression? Everything I've read and heard tells me
> it was under the impression it was the other way around.
> 
> (Configuration data being stuff that is easier to replace and more
> machine-specific. Data being less machine-specific and harder to replace.)
> 
> Think of it in the context of something like Thunderbird address books vs.
> something like what SMTP server a given machine uses or some saved window
> geometry. 
> 
> Your address book is at least as important and system-agnostic as a
> LibreOffice document but you don't access it through an open dialog, so it
> goes under data where tools will prioritize backing it up.
> 
> Your configuration, like what SMTP server you use, may be different on
> different PCs and, while it'll be a pain to re-enter, it's not the end of
> the world if you lose it.
> 
> > If it is too difficult to separate application data from configuration data,
> > that data could remain in the configuration data. However, I recommend at
> > least separating plug-ins and add-ons.
> 
> The only way I can see that making sense is if you consider plugins and
> addons to be "configuration". Otherwise, what's there to separate when
> things like bookmarks belong under "data"?

That demonstrates the confusion in separating into config and data and differences in specification and implementation which is why, despite my comments, the discussion is closed.

See Benjamin Smedberg [:bsmedberg] 2013-05-10 13:12:29 PDT [Comment 72] https://bugzilla.mozilla.org/show_bug.cgi?id=259356#c72.
(In reply to Stephan Sokolow from comment #75)
> (In reply to Hugo Osvaldo Barrera from comment #73)
> > 
> > I remain confused. Does Mozilla support GNU/Linux, or doesn't it?
> > 
> > It seems that you put attention into the fact that the code builds, and
> > package it, but you don't adapt to the OS's standard. (Though you *do* for
> > other Operating Systems).
> > 
> > Are you slowly dropping support for GNU/Linux? Because you either support,
> > or don't, it confuses users if you say you do, and then ship half-ported
> > software.
> 
> Please don't use those "you're either with us or against us" tactics. They
> poison the discourse and, if people did rise to your bait, they'd be more
> likely to drop Linux support to spite you than do what you want.
>

That might not be a bad thing. Mozilla is keeping firefox-for-linux almost frozen. Linux specific issues keep getting rejected, and I've seen plenty of builds with just windows-specific enhancements.

At least if Mozilla compeltely dropped Firefox for *nix, some other foundation would have the motivation to pick it up and really maintain it.

As-is, it just stagnates. All you get is security updates.
And no, a single person can't just branch it; you need something with way larger resources.
(In reply to Hugo Osvaldo Barrera from comment #78)
> That might not be a bad thing. Mozilla is keeping firefox-for-linux almost
> frozen. Linux specific issues keep getting rejected, and I've seen plenty of
> builds with just windows-specific enhancements.

Please cite sources. So far, the only thing which comes to mind which might explain your claim that it's frozen is that Mozilla isn't bending over backward to integrate with all the unproven ideas the Ubuntu and GNOME UI teams are hopping on.

Personally, I'd call that "taking a cautious approach to avoid wasting manpower".
I'm restricting further comments on this bug. Bugzilla isn't intended as a debate forum. Benjamin's the module owner in this area, and he's already asked glandium to look into it. Both are longtime Mozilla contributors, and glandium's commitment to Linux is beyond reproach (http://glandium.org/about/). No further flames, please.

[Comments are restricted to users with the editbugs privilege -- with privilege comes responsibility, so please consider carefully before commenting if you are able.]
Restrict Comments: true
Just a clarification, after such a comment flood, the status of this bug is that it's waiting for a new patch taking into account comment 51. I'll happily review such patch.
Mike, which (existing ones?) directory service atoms would you like to utilize by GetUnixXDGUserDirectory()? Bug 239254 uses XDG_CACHE_HOME or $HOME/.cache only.
(In reply to Martin Stránský from comment #82)
> Mike, which (existing ones?) directory service atoms would you like to
> utilize by GetUnixXDGUserDirectory()? Bug 239254 uses XDG_CACHE_HOME or
> $HOME/.cache only.

The sXDG* atoms and NS_UNIX_XDG_* strings are pretty much useless. Just expose the "right" value from nsXREDirProvider::GetUserDataDirectory. Adjustments will need to be done to e.g. nsXREDirProvider::AppendSysUserExtensionPath, and maybe others. That still leaves the problem of data migration, which i still think we'd just better avoid entirely, especially with all the things that keep file paths in preferences and whatnot. The reset feature from about:support could probably be made to force the switch.
Blocks: 903874
Restrict Comments: false
Is there any progress with this? So far Firefox still uses $HOME/.mozilla and doesn't conform to XDG base directory specification.
There is no progress on this bug. It is assigned to nobody and is not a priority for the team.
I use NFS /export/home/tom between multiple machines.
XDG means that I can have more than one machine logged in.
With Firefox, I can "pick the profile" at launch.
With Thunderbird, it seems I just "can't launch it", if it is open on another host sharing home.
???
>Just expose the "right" value from nsXREDirProvider::GetUserDataDirectory. 
>Adjustments will need to be done to e.g. nsXREDirProvider::AppendSysUserExtensionPath, and maybe others.

I presume others are 

nsAppFileLocationProvider::GetProductDirectory
nsAppFileLocationProvider::GetDefaultUserProfileRoot

and friends.

As for the config+data discussion, our sandbox currently *allows* access to XDG_CONFIG_PATH and XDG_CONFIG_DIRS to let GTK and friends pick up their config files. So you can't store profiles there. Conversely, we only allow access to specific subdirs in XDG_DATA_HOME/DIRS (for fonts), so .local/share/mozilla would work.

My thinking would be to get GetUserDataDirectory home probe for ~/.mozilla, and if present, use the current logic (so no migration, and no bad surprises for user relying on it), and if not, test for existence of XDG_DATA_HOME and use /mozilla there (so we are nicer on clean installs on new systems).
Mass bug change to replace various 'parity' whiteboard flags with the new canonical keywords. (See bug 1443764 comment 13.)
Keywords: parity-chrome
Whiteboard: parity-chrome
What's the state on this issue? anyone working on it? I seem to have a .cache/mozilla folder which is cool, but .config/mozilla is non-existant and there's still a .mozilla in my $HOME directory.
legacy configuration (check if '$HOME/.mozilla' exists; if it does, we use
this path. Otherwise, we use the XDG path).
Attachment #9012358 - Attachment description: Bug 259356 - Add support for the XDG Base Directory Specification. We also check for → Bug 259356 - Add support for the XDG Base Directory Specification.
Assignee: nobody → edenisfa

I will kindly re-ask what Edênis Freindorfer Azevedo asked on Phabricator's D6995 on Jan 6, 2020:
"@glandium Hello! Can we try again to check on this? I can still work on this if more changes are required."

There was some discussion on Matrix/Element over risk/benefit of taking the change:

Pro:

  • Makes Firefox behave like other software when backing up things, no need to special case .mozilla etc.
  • XDG is supported by competing browsers.
  • Easier to store profile / caches in a specific location without having to symlink things around.

Contra:

  • Third party software that expects to find Firefox profiles (only) in .mozilla will break.
  • If XDG dirs get unset or change unexpectedly, Firefox will no longer find the profile and the user might have a hard to diagnose dataloss.

In general I feel like the current patches are a solution that's workable and will be seamless for the majority of users, and makes us a better behaved Linux citizen, but it's unavoidable that a change like this will break some people's setup as well, and not go far enough for others (because we don't force-migrate etc).

To address some of the cons:

By default, Firefox could default to .mozilla similarly to how git works, however, this would mean that having .mozilla in a different location would only be available to those who explicitly want that to be the case.

Also most people leave the XDG environment variables unused (and just use the default locations, .config, .local/share, ...) and for those who explicitly set them, the unsetting of those variables shouldn't be a big problem, at least not a problem that would be specific to Firefox (in such cases a lot of things will have an altered behaviour).

If XDG dirs get unset or change unexpectedly, Firefox will no longer find the profile and the user might have a hard to diagnose dataloss.

I half-agree with nils here. If .mozilla is detected then it should use that, if not, make the profile in XDG_CONFIG_HOME. So all new configs would be in XDG dirs, and people with existing configs wouldn't have to bother migrating or anything, unless they want to, in which case they can just mv it.

If .mozilla is detected then it should use that, if not, make the profile in XDG_CONFIG_HOME

The case described is where XDG_DATA_HOME changes or becomes unset. There would be no .mozilla because you were always in the new mode.

the unsetting of those variables shouldn't be a big problem, at least not a problem that would be specific to Firefox (in such cases a lot of things will have an altered behaviour).

There's a fair amount of other apps that don't follow the spec, and for many people the browser is one of the most important ones, hence the "hard to diagnose" concern. I have no idea if this is going to be common (I really hope not!), but experience says at least one person will hit it :)

If someone explicitly sets any XDG variables, that is on them to maintain in their system. This is something for the distribution or individual tinkerer to deal with. Sure there are applications that don't follow this, but there are many that do, and it would cause a broken system state that someone savy enough to change XDG vars would be savy enough to diagnose and fix.

This should not be a concern in supporting XDG vars, and I do agree with the use .mozilla if detected, otherwise go with the spec approach.

The case described is where XDG_DATA_HOME changes or becomes unset. There would be no .mozilla because you were always in the new mode.

Yes, a very vast majority of the time the XDG variables are set by the user, and most of that time it's not a new user, the other times it's set by someone who probably installed someone else's dotfiles and is removing random stuff, and to that I say: If you installed someone else's dotfiles and changed stuff without knowing what you're doing, stuff not working is 100% your fault and no program should willingly keep a bad functionality just to account for you.

There's a fair amount of other apps that don't follow the spec

Yep, sadly there are.

Most of the time I either try find an alternative, or in the case of firefox/steam I change my $HOME variable so it doesn't make a dotfile in my real home, then i make a symlink to where it should be. Being a native feature would be a far better solution.

And I completely agree with @me's comment.

As an update, I've reached out to some major distros to know if they have any feelings about this. Some highlights that came up during that discussion:

  1. There's some preference for XDG_CONFIG_HOME over XDG_DATA_HOME. I believe most comments above, and the original patch, had that as well. Of course, you can always argue the other way around as well: https://bugs.chromium.org/p/chromium/issues/detail?id=129861
    There's some good discussion there about what XDG_DATA_HOME means exactly, and the fact that the default is ".local/share" leads to me to tend to agree that it probably is not supposed to be the kind of "user data" that a profile would be even though the XDG spec isn't worded very clearly there.

  2. The push to use XDG_DATA_HOME instead was related to how sandboxing interacts with this, but this revealed a bug. I mentioned XDG_CONFIG_PATH in comment 83 above, and that's actually what the code uses, but this doesn't exist! It should have been XDG_CONFIG_HOME. I made a mistake in bug 1399392 but this wasn't caught because there's a fallback to the default if the env var isn't set. I'll file a follow-up for that.

  3. I understand the argument that it would be annoying if Firefox uses XDG_DATA_HOME and Chromium uses XDG_CONFIG_HOME. Unfortunately for security that would mean either hunting down all the subdirs that are used in practice and whitelisting those (instead of the root), or adding support for blacklisting in the sandboxing code (ugh!). That means the patch in here wouldn't be enough to fix this problem, if we decide we want XDG_CONFIG_HOME after all.

(In reply to Gian-Carlo Pascutto [:gcp] from comment #113)

As an update, I've reached out to some major distros to know if they have any feelings about this. Some highlights that came up during that discussion:

I don't have the URLs handy but what I read on various mailing lists was to think of it as:

  • XDG_CONFIG_HOME is for stuff that may depend on things which vary from system to system like monitor resolution and which is less worthy of tears if lost. (eg. persisted GUI settings like window geometry and toolbar setup)
  • XDG_DATA_HOME is for userdata that isn't accessed via open/save dialogs, like browser bookmarks or the contents of desktop sticky notes or a personal wiki.

...and I just realized that I already said almost the same thing 9 years ago. Sorry about that.

(I really shouldn't go online when I had a bad night's sleep. It keeps me from double-checking what I say/write.)

There's some preference for XDG_CONFIG_HOME over XDG_DATA_HOME.

Data such as crash reports, downloaded extensions, and website storage should mostly go under XDG_DATA_HOME. I would also argue that bookmarks are more about user data than user configuration.

Personally, I would assume that what should be stored in ${XDG_CONFIG_HOME}/firefox are: prefs.js, handlers.json, pkcs11.txt, extension-settings.json. I don't know if containers.json is shipped with Firefox or provided by the relevant extension. That would also be about config.

The rest in my opinion should end up under ${XDG_DATA_HOME}/firefox.

Some users are using git to version their config directories, but they don't want to do the same for data. Some users would like to only backup their $HOME/.config directory, but not the $HOME/.local data directory. YMMV.

It's a minor point, but I feel it's at least worth taking an informed decision now before landing this, rather than risking another future migration to re-adjust.

Some users are using git to version their config directories, but they don't want to do the same for data. Some users would like to only backup their $HOME/.config directory, but not the $HOME/.local data directory. YMMV.

Thank you for mentioning this, I happen to be one such person!

A good rule of thumb to follow is moving configuration into its own files. Configuration should be anything that's user-driven, and these config files shouldn't hold program state.
An example of state is that Firefox shouldn't note the last tabs I had open within a configuration file. They should be in a file dedicated to that purpose. Otherwise, it would would create a non-permanent change to a permanent config file. I don't know if Firefox currently does this to be clear, this is just an illustrative example. I know some history Linux software like KDE does it like that currently, but they are committed to work it away in the next major version.

So care should be taken to separate configuration, program state, data and cache. There is a separate XDG folder for caches specifically, though I'm not sure if any data that Firefox generates fits under XDG_CACHE_HOME. Rarely do programs use this from what I've seen, since most data is to some degree essential; most stick to XDG_CONFIG_HOME and XDG_DATA_HOME from the scheme.

There is no point in discussing splitting up things between XDG_CONFIG_HOME and XDG_DATA_HOME, and the reason why has been repeatedly explained in the comments.

We respect XDG_CACHE_HOME already.

XDG_DATA_HOME is for userdata that isn't accessed via open/save dialogs, like browser bookmarks or the contents of desktop sticky notes or a personal wiki.

I thought this as well, which is why I proposed that, but as already explained above, this doesn't appear to be correct. See for example https://bugs.chromium.org/p/chromium/issues/detail?id=129861#c8. The XDG spec seems to assume the reader has a substantial background to be able to understand what is meant there, which is really unfortunate.

(In reply to Gian-Carlo Pascutto [:gcp] from comment #119)

XDG_DATA_HOME is for userdata that isn't accessed via open/save dialogs, like browser bookmarks or the contents of desktop sticky notes or a personal wiki.

I thought this as well, which is why I proposed that, but as already explained above, this doesn't appear to be correct. See for example https://bugs.chromium.org/p/chromium/issues/detail?id=129861#c8. The XDG spec seems to assume the reader has a substantial background to be able to understand what is meant there, which is really unfortunate.

When he makes that argument, he ignores four things:

  1. The XDG spec's goal is to eliminate the forest of dotdirs in the user's home directory but, if XDG_DATA_DIR is meant to be for files that are conceptually read-only, then it provides no alternative for per-user userdata files that aren't conceptually read-only... suggesting that it's interpreting the FHS's definition of /usr/share as read-only, not as a property of share, but as a property of it being inside /usr rather than /home.

    This also makes sense when you look at how, in the modern day, applications generally don't bother to draw a big distinction between "I installed this plugin in a per-user location" and "I stored some bulky userdata in a per-user location" beyond the Windows-inspired one of "Is this too bulky to inflict on profile-roaming?" ...a distinction the XDG spec makes no mention of.

    ...unless they expected to follow the Windows "convention" of cluttering up My Documents with tons of garbage like My Saved Games that really should go in Application Data because nobody's going to want to browse to it with Explorer or an Open/Save dialog under normal operation. (Microsoft really needs to better educate their designers on the conceptual separation between document-centric and application-centric desktop paradigms and how to make them coexist harmoniously.)

  2. The XDG trash specification says to put a user's trash folder at $XDG_DATA_HOME/Trash. That's about as opposite to "read-only data" as you can get.

  3. Chrome may put its userdata in XDG_CONFIG_HOME, but Linux-first WebKit-based browsers like LuaKit and Midori put their userdata in XDG_DATA_HOME.

  4. The "I already had this discussion" post he links is from 2009 and a lot has changed since he wrote "I looked in my ~/.local/share and the only things stored there are .desktop files and associated data. It doesn't look like any app stores "real" user data there."

For example:

  1. KDE 5 chose to use ~/.local/share in the manner I described when they broke apart the monolithic ~/.kde from KDE 4 and earlier.

  2. The decision has already been made in the world of games and they chose to agree with me.

    1. The vast majority of Unity games that don't have their game-saving redirected by something like Steam will store their prefs file (stuff like keybindings and resolution settings) in $XDG_CONFIG_HOME/.unity3d/<developer>/<game> and their userdata in either $XDG_DATA_HOME/<developer>/<game> or $XDG_DATA_HOME/<game>. The ones that don't, which store their userdata alongside prefs, appear to be consistently using older versions of Unity. (For anyone not familiar with the game development ecosystem, Unity is the most popular game engine in use by a large margin.)

    2. The Godot game engine (the most promising open-source competitor to Unity) stores userdata at ~/.local/share/godot/app_userdata/<game>.

    3. SDL 2, which is pushed by Valve as the platform abstraction all game engines should sit on top of, directs games to save userdata in XDG_DATA_HOME via SDL_GetPrefPath. (SDL_GetPrefPath is explicitly documented as being the only location an application is allowed to assume to be writable, intended for both prefs and userdata, and the only alternative is SDL_GetBasePath which is intended to resolve to something like /usr/share.)

  3. As mentioned, Linux-first WebKit-based browsers like LuaKit and Midori put their userdata in XDG_DATA_HOME, as do the Comix and MComic CBZ/CBR viewers, which also have a user bookmarks system. (Note that Comix saw its last release in 2009, the year he said he couldn't find any examples of programs doing it that way.)

  4. GVFS, GNOME's bundle of virtual filesystem backends for GTK's GIO abstraction, uses ~/.local/share/gvfs-metadata/ to store metadata you attach to files using applications like Nautilus.

  5. As a few more concrete counterexamples to his claim, Midnight Commander (The orthodox file manager that's as well-known on Unixy OSes as Norton Commander was on DOS), NeoVim (The Vim fork trying to be the Phoenix/Firebird/Firefox to Vim's Mozilla Suite), the Geeqie image viewer (a maintained fork of GQView), and the Parcellite clipboard manager popular on lightweight X11 desktops are all examples of non-game applications that store conceptually non-read-only userdata in XDG_DATA_HOME.

Thanks, that's a nice overview.

Well, when I compare a content of ~/.local/share and ~/.config on my box I clearly see that most of desktop applications use ~/.config to store the data, while some of them use both (KDE/Plasma/mc/vlc...).

There's the list, Fedora 32 system:

~/.config
abrt
akonadi
Atom
autostart
autostart-scripts
baloofilerc
bluedevilglobalrc
caja
caja-actions
cef_user_data
chromium
configstore
dconf
deepin
devcert
d-feet
emaildefaults
emailidentities
enchant
eog
epiphany
evince
evolution
fedora-plasma-cacherc
fedoraproject.org
filezilla
fontconfig
gatsby
gconf
gedit
GIMP
gnome-boxes
gnome-control-center
gnome-games
gnome-initial-setup-done
gnome-session
gnote
goa-1.0
google-chrome
gtk-2.0
gtk-3.0
gtk-4.0
gtkrc
gtkrc-2.0
gwenviewrc
hexchat
htop
ibus
imsettings
Intel
kactivitymanagerdrc
kactivitymanagerd-statsrc
kateschemarc
kcmfonts
kcminputrc
kcmshell5rc
kconf_updaterc
kded5rc
kdeglobals
kglobalshortcutsrc
khotkeysrc
kmail2rc
kmailsearchindexingrc
kmixrc
knfsshare
konq_history
konquerorrc
konsolerc
korgacrc
krunnerrc
kscreenlockerrc
ksmserverrc
ktimezonedrc
kwinrc
kwinrulesrc
kxkbrc
libaccounts-glib
libreoffice
libvirt
mc
menus
mimeapps.list
mozilla
mpv
nautilus
pavucontrol.ini
pdfmod
phishingurlrc
pipewire-media-session
plasma-localerc
plasma-nm
plasmanotifyrc
plasmashellrc
powerdevilrc
powermanagementprofilesrc
pulse
QMatrixClient
QtProject.conf
sealert.conf
session
specialmailcollectionsrc
startupconfig
startupconfigfiles
startupconfigkeys
systemsettingsrc
Thunar
totem
transmission
Trolltech.conf
VirtualBox
vlc
webengineurlinterceptoradblockrc
wireshark
xfce4
xsettingsd
yelp

~/.local/share
akonadi
applications
apps
backgrounds
baloo
contacts
dde-osd
deepin
evolution
flatpak
folks
gegl-0.4
gnome-settings-daemon
gnome-shell
grilo-plugins
gsettings-data-convert
gvfs-metadata
ibus-typing-booster
icc
icons
kactivitymanagerd
kded5
keyrings
klipper
konsole
kscreen
kwalletd
kxmlgui5
mc
nautilus
pki
qrenderdoc
RecentDocuments
recently-used.xbel
rhythmbox
sounds
Steam
themes
totem
tracker
Trash
user-places.xbel
virtualenv
vlc
vulkan
webkitgtk
xorg

btw. I don't think the question is "What's the most obvious place where to put the config" but "Do we want to follow that".

To be honest, I don't really care if you follow the specification, all I care about is not having an entire folder in my home for a single software directory, and that's probably the case for most people.
So you can just move ~/.mozilla to ~/.local/share/mozilla and most people people will be happy.

That would be a very... hacky solution, to say the least.

So you can just move ~/.mozilla to ~/.local/share/mozilla

How would you suggest migrating? What about people who don't have ~/.local/share (Controlled by XDG_DATA_HOME, a part of the spec, which you "don't care about")

Moving from one hard-coded location to another hard-coded location is quite literally just moving the problem (since then .local/share would be the dotfile made by mozilla apps, and only mozilla apps (for me!))

and most people people will be happy.

Incorrect. Most people are happy with how it is right now. Do not take this thread or your beliefs as the majority opinion.

If you're going to move it, you're going to have to make everyone who wants it moved happy, as well as keeping the people who don't care happy.

So, I'm not sure if I already said this, or if it contradicts something I already said here, but my proposed solution would be:

If ~/.mozilla exists, great! Use it.

If it doesn't exist, use $XDG_DATA_HOME/mozilla (if the var is unset fall back to ~/.local/share/mozilla) as the config location.

This means that all new configs will be in XDG_DATA, and all existing configs won't have to bother moving (Unless the user does it manually)

As an example, in sh, this would be as simple as

# existing configs
if [ -d "$HOME/.mozilla" ]; then
    CONFIG_DIR=$HOME/.mozilla
else  # non-existing or new configs
    CONFIG_DIR=${XDG_DATA_HOME:=$HOME/.local/share}/mozilla
fi
# if the dir doesn't exist
mkdir -p "$CONFIG_DIR"

I just want to note that the implementation in the Phabricator D6995 patch has been approved by Martin Stránský [:stransky] and is awaiting review from Mike Hommey [:glandium].

(In reply to gk from comment #126)

That would be a very... hacky solution, to say the least.

So you can just move ~/.mozilla to ~/.local/share/mozilla

How would you suggest migrating? What about people who don't have ~/.local/share (Controlled by XDG_DATA_HOME, a part of the spec, which you "don't care about")

Moving from one hard-coded location to another hard-coded location is quite literally just moving the problem (since then .local/share would be the dotfile made by mozilla apps, and only mozilla apps (for me!))

and most people people will be happy.

Incorrect. Most people are happy with how it is right now. Do not take this thread or your beliefs as the majority opinion.

If you're going to move it, you're going to have to make everyone who wants it moved happy, as well as keeping the people who don't care happy.

So, I'm not sure if I already said this, or if it contradicts something I already said here, but my proposed solution would be:

If ~/.mozilla exists, great! Use it.

If it doesn't exist, use $XDG_DATA_HOME/mozilla (if the var is unset fall back to ~/.local/share/mozilla) as the config location.

This means that all new configs will be in XDG_DATA, and all existing configs won't have to bother moving (Unless the user does it manually)

As an example, in sh, this would be as simple as

# existing configs
if [ -d "$HOME/.mozilla" ]; then
    CONFIG_DIR=$HOME/.mozilla
else  # non-existing or new configs
    CONFIG_DIR=${XDG_DATA_HOME:=$HOME/.local/share}/mozilla
fi
# if the dir doesn't exist
mkdir -p "$CONFIG_DIR"

Ah, I meant most people that are annoyed by the ~/mozilla dir will be happy.
Also your idea is much better.

(In reply to Tadej Janež from comment #127)

I just want to note that the implementation in the Phabricator D6995 patch has been approved by Martin Stránský [:stransky] and is awaiting review from Mike Hommey [:glandium].

Oh, I wasn't aware of that.
Thanks for the reminder.

Hi Mike! The patch is waiting on a second review from you. Thanks.

Flags: needinfo?(mh+mozilla)
Component: General → XPCOM

I tried contacting Mike a few times, but sadly I haven't heard back so far.

That's unfortunate. Can it be reassigned to someone who can get on it quickly?

I looked at this again because we were fixing bug 1672421. As explained in comment 89 and comment 113, we can't use XDG_CONFIG_HOME because of how it interacts with sandboxing, at least not without adding blacklisting support to the sandboxing code.

Currently the sandbox allows reading XDG_CONFIG_HOME (bug 1672421 existing doesn't change this because the fallback there worked) for various GTK related libraries that are loaded in web content. While these eventually might go away (bug 1635451 and bug 1129492), we still need those to work for now. So if you put the profiles in that dir, a compromised web content process could read all cookies, stored passwords, etc. That's bad.

I thought I looked at an earlier version of this patch using XDG_DATA_HOME, but the current one seems to be using XDG_CONFIG_HOME so that can't land - it would open up security holes.

If we want XDG_CONFIG_HOME, which seems to have a slight preference (but see comment 120 for some arguments why XDG_DATA_HOME would be reasonable too), the sandbox needs a way to overlay a blacklist (for the profile dir...) onto its whitelist.

If using XDG_CONFIG_HOME isn't an option now, and XDG_DATA_HOME is arguably a better option anyway, can we skip the yak shaving and just use XDG_DATA_HOME? For most users the difference is negligible anyway, and as explained in the prior comment, this scheme follows conventions set by most Linux-first applications (and therefore makes things like git-based backups of configuration feasible).

If there is still appetite for using XDG_CONFIG_HOME, I think this can be done in the future with a more fine comb to split apart actual application/config data, assuming this original issue ever gets fixed.

(In reply to tm from comment #142)

If using XDG_CONFIG_HOME isn't an option now, and XDG_DATA_HOME is arguably a better option anyway, can we skip the yak shaving and just use XDG_DATA_HOME? For most users the difference is negligible anyway, and as explained in the prior comment, this scheme follows conventions set by most Linux-first applications (and therefore makes things like git-based backups of configuration feasible).

If there is still appetite for using XDG_CONFIG_HOME, I think this can be done in the future with a more fine comb to split apart actual application/config data, assuming this original issue ever gets fixed.

I don't know if devs will accept using one of them from opinions alone. As you can see, most comments here are me-too asking for this feature and some topics are outdated. My first patch used XDG_DATA_HOME, but due to interpretation issues about the spec (version 0.7), it could not go forward. Now, we have version 0.8 that also has a new variable XDG_STATE_HOME and (imo) further endorses using XDG_DATA_HOME how we are trying to do here (and used by other apps, as [comment 120])(https://bugzilla.mozilla.org/show_bug.cgi?id=259356#c120).

Keep in mind that version 0.8 was released in 08th May 2021, while 0.7 is from 24th November 2010. My first patch is from 2018. I think we could use XDG_DATA_HOME considering the newly spec. Given that we can agree on this, it takes 1 minute to replace the use of XDG_CONFIG_HOME by XDG_DATA_HOME on phabricator. I know nothing about the sandbox, though. Probably a headache for the future.

Reading the comments here, I agree with XDG_DATA_HOME proponents. XDG_DATA_HOME is arguably a better place to put Firefox if you guys can't split application from user data.

It's unusual, but much better than the status quo.

If using XDG_CONFIG_HOME isn't an option now, and XDG_DATA_HOME is arguably a better option anyway, can we skip the yak shaving and just use XDG_DATA_HOME?

It's not an option "now" because a feature isn't implemented. We can implement the feature, or it might arrive as a side effect of other work (bug bug 1129492). If we make a bad pick for the dir and want to change it later on, we'll end up breaking Unix admin's scripting twice, and cause all the code to have to handle 3 cases. Or we end up being the odd one out in .local/share and annoying everyone.

Given that the distro maintainers (...who are also actively working on Firefox for Linux, making their opinion rather relevant to me) said they preferred XDG_CONFIG_HOME, I think I'm going to look at how hard it is exactly to add blacklisting to the sandbox, before we "rush" (ahem 😉) to a solution here. I can also confirm that I have about 10x as many apps in .config, and notably a lot of the crossplatform apps that have similar problems with the config/data distinction are in .config.

I'm going to remove the needinfo for glandium. I've talked to him about this bug before, and while he doesn't oppose either solution on technical grounds, he's not willing and/or able to deal with the fallout that will likely result from this change, especially given that the benefit is marginal. I might be willing to live with the consequences of fixing this and glue everything we break in the process, but then I'd prefer to have the major external contributors to Firefox on Linux on board with the change, and it looks like that means XDG_CONFIG_HOME.

Flags: needinfo?(mh+mozilla)
Depends on: 1718084

(In reply to Gian-Carlo Pascutto [:gcp] from comment #145)

Given that the distro maintainers (...who are also actively working on Firefox for Linux, making their opinion rather relevant to me) said they preferred XDG_CONFIG_HOME, I think I'm going to look at how hard it is exactly to add blacklisting to the sandbox, before we "rush" (ahem 😉) to a solution here. I can also confirm that I have about 10x as many apps in .config, and notably a lot of the crossplatform apps that have similar problems with the config/data distinction are in .config.

For me the numbers are about the same, and quite a few applications have entries in both, but I use few crossplatform apps and most of them have been banished to hand-restricted flatpaks - except Firefox ;)

I understand there are a lot of opinions to consider of course, it just seemed like you posed the two as real options in your first comment; the voice of distro maintainers wasn't clear in the bits of the thread I'd seen, I've only followed the last two years or so of this "rush".

I'm obviously biased towards using DATA_HOME, so I thought I'd ask the obvious question. Thanks for the explanation :)

Depends on: 1719279

(In reply to Edênis Freindorfer Azevedo from comment #143)

(In reply to tm from comment #142)

If using XDG_CONFIG_HOME isn't an option now, and XDG_DATA_HOME is arguably a better option anyway, can we skip the yak shaving and just use XDG_DATA_HOME? For most users the difference is negligible anyway, and as explained in the prior comment, this scheme follows conventions set by most Linux-first applications (and therefore makes things like git-based backups of configuration feasible).

If there is still appetite for using XDG_CONFIG_HOME, I think this can be done in the future with a more fine comb to split apart actual application/config data, assuming this original issue ever gets fixed.

I don't know if devs will accept using one of them from opinions alone. As you can see, most comments here are me-too asking for this feature and some topics are outdated. My first patch used XDG_DATA_HOME, but due to interpretation issues about the spec (version 0.7), it could not go forward. Now, we have version 0.8 that also has a new variable XDG_STATE_HOME and (imo) further endorses using XDG_DATA_HOME how we are trying to do here (and used by other apps, as [comment 120])(https://bugzilla.mozilla.org/show_bug.cgi?id=259356#c120).

Keep in mind that version 0.8 was released in 08th May 2021, while 0.7 is from 24th November 2010. My first patch is from 2018. I think we could use XDG_DATA_HOME considering the newly spec. Given that we can agree on this, it takes 1 minute to replace the use of XDG_CONFIG_HOME by XDG_DATA_HOME on phabricator. I know nothing about the sandbox, though. Probably a headache for the future.

The work on the sandbox is done, and bug 1718084 should land on central soonish now. Hopefully it will stick there.

Are you still looking to work on this, now that XDG_CONFIG_HOME should work ? If not, then I can gently steal the bug and continue your work to land this feature.

Flags: needinfo?(edenisfa)

(In reply to Alexandre LISSY :gerard-majax from comment #147)

The work on the sandbox is done, and bug 1718084 should land on central soonish now. Hopefully it will stick there.

Are you still looking to work on this, now that XDG_CONFIG_HOME should work ? If not, then I can gently steal the bug and continue your work to land this feature.

You do not even need to ask, my friend. Go right ahead and finish the job. I would really love to see this landed too. If the sandbox fix stays, I think most is done already, but beware nevertheless.

Flags: needinfo?(edenisfa)

(In reply to Edênis Freindorfer Azevedo from comment #148)

(In reply to Alexandre LISSY :gerard-majax from comment #147)

The work on the sandbox is done, and bug 1718084 should land on central soonish now. Hopefully it will stick there.

Are you still looking to work on this, now that XDG_CONFIG_HOME should work ? If not, then I can gently steal the bug and continue your work to land this feature.

You do not even need to ask, my friend. Go right ahead and finish the job. I would really love to see this landed too. If the sandbox fix stays, I think most is done already, but beware nevertheless.

Thanks. I always prefer to ask gently, we never know why people are contributing and it might sometimes have consequences, so stealing a bug without warning can be problematic :).

Assignee: edenisfa → lissyx+mozillians
Attachment #311040 - Attachment is obsolete: true
Attachment #703317 - Attachment is obsolete: true
Attachment #9233503 - Attachment is obsolete: true
Attachment #9233505 - Attachment is obsolete: true
Attachment #9012358 - Attachment description: Bug 259356 - Add support for the XDG Base Directory Specification. → WIP: Bug 259356 - Add support for the XDG Base Directory Specification
Depends on: 1723753
Attachment #9012358 - Attachment description: WIP: Bug 259356 - Add support for the XDG Base Directory Specification → Bug 259356 - Add support for the XDG Base Directory Specification r?stransky,glandium,gcp
Depends on: 1724958

Since this bug has a lot of people in Cc, I'd like to take the opportunity to ask for some feedback on third party tooling that might be dependant on the ~/.mozilla/ path and might need proper handling of $XDG_CONFIG_HOME. As :glandium mentionned, there's mozci/mozregression which I filed a bug for as bug 1724958, but others are likely to be out in the wild.

(In reply to Alexandre LISSY :gerard-majax from comment #152)

Since this bug has a lot of people in Cc, I'd like to take the opportunity to ask for some feedback on third party tooling that might be dependant on the ~/.mozilla/ path and might need proper handling of $XDG_CONFIG_HOME. As :glandium mentionned, there's mozci/mozregression which I filed a bug for as bug 1724958, but others are likely to be out in the wild.

I don't know if web-ext depends on ~/.mozilla but it's definitely worth checking with the addons team.

(In reply to Danny Colin [:sdk] from comment #153)

(In reply to Alexandre LISSY :gerard-majax from comment #152)

Since this bug has a lot of people in Cc, I'd like to take the opportunity to ask for some feedback on third party tooling that might be dependant on the ~/.mozilla/ path and might need proper handling of $XDG_CONFIG_HOME. As :glandium mentionned, there's mozci/mozregression which I filed a bug for as bug 1724958, but others are likely to be out in the wild.

I don't know if web-ext depends on ~/.mozilla but it's definitely worth checking with the addons team.

They just confirmed me it is okay on their side.

(In reply to Danny Colin [:sdk] from comment #153)

(In reply to Alexandre LISSY :gerard-majax from comment #152)

Since this bug has a lot of people in Cc, I'd like to take the opportunity to ask for some feedback on third party tooling that might be dependant on the ~/.mozilla/ path and might need proper handling of $XDG_CONFIG_HOME. As :glandium mentionned, there's mozci/mozregression which I filed a bug for as bug 1724958, but others are likely to be out in the wild.

I don't know if web-ext depends on ~/.mozilla but it's definitely worth checking with the addons team.

web-ext relies on ProfD/extensions/, which is the standard path for extensions.
Relevant sources:

Depends on: 1725011

(In reply to Alexandre LISSY :gerard-majax from comment #152)

Since this bug has a lot of people in Cc, I'd like to take the opportunity to ask for some feedback on third party tooling that might be dependant on the ~/.mozilla/ path and might need proper handling of $XDG_CONFIG_HOME. As :glandium mentionned, there's mozci/mozregression which I filed a bug for as bug 1724958, but others are likely to be out in the wild.

The crash reporter client assumes the existence of that directory (see here). I don't know if the changes here would affect it but it might be worth keeping it in mind.

Depends on: 1725134

(In reply to Alexandre LISSY :gerard-majax from comment #152)

Since this bug has a lot of people in Cc, I'd like to take the opportunity to ask for some feedback on third party tooling that might be dependant on the ~/.mozilla/ path and might need proper handling of $XDG_CONFIG_HOME. As :glandium mentionned, there's mozci/mozregression which I filed a bug for as bug 1724958, but others are likely to be out in the wild.

You can use MOZILLA_CONFIG as a temporary solution until XDG is supported by default, and use MOZILLA_CONFIG as a configuration directory when MOZILLA_CONFIG exists, otherwise it falls back to ~/.mozilla. According to the XDG specification, it's obvious that you shouldn't put everything into .mozilla. But I don't know the directory structure of .mozilla, see https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html to make the directory as compliant as possible.

(In reply to ohosmp from comment #157)

(In reply to Alexandre LISSY :gerard-majax from comment #152)

Since this bug has a lot of people in Cc, I'd like to take the opportunity to ask for some feedback on third party tooling that might be dependant on the ~/.mozilla/ path and might need proper handling of $XDG_CONFIG_HOME. As :glandium mentionned, there's mozci/mozregression which I filed a bug for as bug 1724958, but others are likely to be out in the wild.

You can use MOZILLA_CONFIG as a temporary solution until XDG is supported by default, and use MOZILLA_CONFIG as a configuration directory when MOZILLA_CONFIG exists, otherwise it falls back to ~/.mozilla. According to the XDG specification, it's obvious that you shouldn't put everything into .mozilla. But I don't know the directory structure of .mozilla, see https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html to make the directory as compliant as possible.

Hi. It doesn't work with firefox 98.0.2 on linux64. Running ltrace -e '-*+*env*' suggests it never looks up for such environment variable.

Hi. It doesn't work with firefox 98.0.2 on linux64. Running ltrace -e '-*+*env*' suggests it never looks up for such environment variable.

I'm just giving the assignee some advice about the Freedesktop.org XDG base directory patch, which still doesn't go into any version of the code.

(In reply to ohosmp from comment #159)

Hi. It doesn't work with firefox 98.0.2 on linux64. Running ltrace -e '-*+*env*' suggests it never looks up for such environment variable.

I'm just giving the assignee some advice about the Freedesktop.org XDG base directory patch, which still doesn't go into any version of the code.

If you read my message carefully, we are worrying about third-parties. I fail to understand the relationship between your reply and the message you quoted.

I think third parties issue can be solved by making this configurable. I.e. add an option to turn XDG base directory on and off with an environment variable. Let's say MOZ_XDG_BASE_DIR_SUPPORT. Make it off by default and announce your plans to switch to it in some future. Those who want to use it now will flip the variable in their environment. Then after reasonable time (a year or whatever you decide?) flip the default for everyone. Third parties that care will update, before that, those who don't (consider them abandoned enough) will break.

Those who do depend on such third parties for whatever reason will just flip the variable the opposite way after that, so their workflow will have an option not to break.

Then after another grace period you can just remove that legacy fallback and they'll break for good then. That's a reasonable migration path.

Severity: normal → S3

(In reply to Shmerl from comment #164)

Then after another grace period you can just remove that legacy fallback and they'll break for good then. That's a reasonable migration path.

Yeah, but the ultimate outcome is that we continue carrying this burden for at least a few years, adding more complexity on already not super tested components, and in the end we might still end up breaking user's flow.

(In reply to :gerard-majax [PTO 01/08/2023-10/09/2023] from comment #169)

Yeah, but the ultimate outcome is that we continue carrying this burden for at least a few years, adding more complexity on already not super tested components, and in the end we might still end up breaking user's flow.

At least it's some way forward even if not perfect.

Alternatively, would you be interested in a relatively simple workaround like adding support for MOZ_HOME env variable to the components this bug depends on and then to Firefox itself? This way when it's not set, they will use default $HOME/.mozilla, and when it's set they would use that location. This way those who don't want wait for the complete fix would set it to something like $HOME/.local/share/mozilla and be done with it.

Alternatively, would you be interested in a relatively simple workaround like adding support for MOZ_HOME env variable to the components this bug depends on and then to Firefox itself? This way when it's not set, they will use default $HOME/.mozilla, and when it's set they would use that location. This way those who don't want wait for the complete fix would set it to something like $HOME/.local/share/mozilla and be done with it.

Please do this, at the bare minimum.

(In reply to Ignacio Taranto from comment #171)

Please do this, at the bare minimum.

I can try doing this, but developers need to ack first they are OK with it, and it will have to be also done in all blocker dependencies. They didn't ack it so far.

I think we can support the MOZ_HOME dir at least.

Flags: needinfo?(lissyx+mozillians)

(In reply to Shmerl from comment #170)

Alternatively, would you be interested in a relatively simple workaround
like adding support for MOZ_HOME env variable to the components this
bug depends on and then to Firefox itself? This way when it's not set,
they will use default $HOME/.mozilla, and when it's set they would
use that location. This way those who don't want wait for the complete
fix would set it to something like $HOME/.local/share/mozilla and
be done with it.

This is a fairly common solution e.g. :

Android SDK -> ANDROID_HOME
Bun         -> BUN_INSTALL
Cargo       -> CARGO_HOME
gnupg       -> GNUPGHOME
go          -> GOPATH
gradle      -> GRADLE_USER_HOME
nimble      -> NIMBLE_DIR
nvm         -> NVM_DIR
Rustup      -> RUSTUP_HOME
volta       -> VOLTA_HOME
w3m         -> W3M_DIR
wine        -> WINEPREFIX

I personally use these to keep my $HOME tidy, and it seems like an easy
and pragmatic solution to keep this ticket from stagnating any longer.

Yes, I didn't get a chance to do it. I'd start with blockers first. Also may be waiting for Mozilla repos to switch to git proper first. If anyone wants to do it for the blockers - feel free to help.

I think we can support the MOZ_HOME dir at least.

I'm not sure what this gains you over comment 162. If you add another directory config option, every dependency and the Firefox code itself now has to do a 3-way fallback, i.e. default to MOZ_HOME, then look in XDG_CONFIG_HOME, then look for ~/.mozilla. This will be fun with writing testcases :-) Adding MOZ_XDG_BASE_DIR_SUPPORT=false that disables the new support in Firefox only needs to do something at profile creation time (and add 2-way fallback everywhere else, still no fun but strictly easier and less error prone). If I check the patch that's attached to this bug, it already has a MOZ_LEGACY_HOME toggle, so the "env var toggle" solution was already coded, just with a different name/default value.

Practically speaking, I'm a bit wary of adding XDG support, and shipping it to release while NOT enabling it by default. If we don't default it, anyone running with the new setting is essentially running an untested configuration. Given that the dependencies include crash reporting, you're not only in an untested config, you're in one that might never get fixed. If we flip it by default, any new profile will have it, so it'll get lots of testing, and any existing user will be on the old code, so that will remain well tested too. So the current patch is just fine from that perspective.

Summarizing, what I think needs to happen is:
a) Patches for the dependencies to support XDG_CONFIG_HOME + .mozilla fallback (note some of these are outside of Mozilla)
b) Rebase the current patch
c) Test the current patch on our CI and testsuites, and fix all the breakage (this could mean fixing more Mozilla tooling we missed in (a))
d) Pick a period where someone with deep Linux expertise isn't buried in Snap/Flatpack/Wayland regressions (in case you wondered why this stalled the last ~3 years)
e) Land the patches in Nightly, enabling XDG support by default (or don't enable by default, but lock it to Nightly then)
f) Watch our CI, Bugzilla and various other Linux forums for additional breakage in 3rd party tooling and assist/evangelize as much as possible

a) Patches for the dependencies to support XDG_CONFIG_HOME + .mozilla fallback (note some of these are outside of Mozilla)

Are there any dependencies other than 1724958, 1725011 and 1725134 that need to be fixed?

If I check the patch that's attached to this bug, it already has a MOZ_LEGACY_HOME toggle, so the "env var toggle" solution was already coded, just with a different name/default value.

I think using a MOZ_LEGACY_HOME boolean setting is a bad approach, when a MOZ_HOME string setting is a lot more configurable. It is also useful for users that prefer a rather small XDG_CONFIG_HOME (mine is at 12kb, while .mozilla is at 800mb) backed up via git and would rather move everything to XDG_DATA_HOME just to get away with the ugly dotfile.

default to MOZ_HOME, then look in XDG_CONFIG_HOME, then look for ~/.mozilla

Since a perfect approach would require firefox and its dependencies to be able to totally separate xdg-config/xdg-data/xdg-state files, with MOZ_XDG_BASE_DIR_SUPPORT in the meantime not being enable to properly deal with the maybe conflicting behaviors of first moving all files to XDG_CONFIG_HOME and then to their proper xdg-dirs (or even new ones should the XDG spec receive a new update), I think having MOZ_HOME > ~/.mozilla would be future-proof enough for the next decade, as it would require the following workflow:

  1. Each component adds the two-way fallback MOZ_HOME > ~/.mozilla for their required files
  2. Each component discusses how to separate their files independently
  3. Each component updates the fallback chain MOZ_HOME > xdg-dir > ~/.mozilla as required
  4. Be aware of new XDG spec updates to update the middle fallback (instead of having to rely on a new boolean setting MOZ_XDG_BASE_DIR_SUPPORT2 etc)

And then some 20y from now just deprecate ~/.mozilla

Flags: needinfo?(gpascutto)

Are there any dependencies other than 1724958, 1725011 and 1725134 that need to be fixed?

Probably, but those are the ones identified at this point.

when a MOZ_HOME string setting is a lot more configurable

The previous comment already explains why this is more an argument against it as for it, notably because...

I think having MOZ_HOME > ~/.mozilla would be future-proof enough for the next decade

...it wouldn't even fix this bug because you still wouldn't get XDG support by default.

for users that prefer a rather small XDG_CONFIG_HOME (mine is at 12kb, while .mozilla is at 800mb) backed up via git

That's a valid concern, but as long as the CONFIG/DATA split isn't done, it will be an issue either way. You either back up too much (annoying) or you don't back it up at all (dataloss), or you special case Firefox somewhere (I assume that's what people do now).

I checked mine and it's 5GB, a lot of it due to VSCode, and indeed they are struggling with the same for the same reasons, e.g. https://github.com/microsoft/vscode/issues/126182#issuecomment-1034321175 and https://github.com/microsoft/vscode/issues/126182#issuecomment-1342766685 ("comment war about ... what is config" 😉). At least I think we mostly handle caches - they're easier to deal with because dataloss and migration aren't as much of an issue.

Flags: needinfo?(gpascutto)

...it wouldn't even fix this bug because you still wouldn't get XDG support by default.

I see now, looking back on the items that use a variable, I think only Discord and Chromium might be similar enough as to be properly compared (non-technical user base, require internet access), and they both use XDG_CONFIG_HOME for now without risking it, so it would indeed be better to first enable a boolean support and then work from there.

About the things that would need to be fixed, I'd like if any of these count as a dependencies or should be linked somehow:

Flags: needinfo?(gpascutto)
Flags: needinfo?(gpascutto)
Depends on: 1890234
Depends on: 1890235
See Also: → 1892505
See Also: → 1892507
Depends on: 1892936
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: