1658094 - Clearing site preferences on shutdown also clears exceptions for clearing on shutdown

Status: VERIFIED FIXED

(Toolkit :: Data Sanitization, defect, P2)

Description

[pf.arqon]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:79.0) Gecko/20100101 Firefox/79.0

Steps to reproduce:

Specify a site as "(permanently) Allow" for cookies.
Set FF to clear cookies (i.e. from sites NOT permanently allowed) on exit.
Check "Clear history when Firefox close", ensure "Cookies" is NOT checked.
Close FF.

Actual results:

Cookies from Allowed site(s) are all deleted.

Expected results:

Cookies from Allowed site(s) should not be deleted, obviously.

Comment 1

[pf.arqon]

The bug appears to be triggered by the "Site Preferences" checkbox incorrectly being used for the test instead.
On the bright side, at least it'll be easy to write the AT for, assuming you have AT.

Comment 2

[pf.arqon]

Attachment: breaks.png

Comment 3

[pf.arqon]

Attachment: works.png

That's the only difference in the settings between the two cases. Cookies is cleared in both, i.e. should NOT be deleted, but FF is deleting them based on the Site Preferences checkbox instead, which is supposed to just be for zoom levels etc.

Comment 4

[pf.arqon]

MHO (though the wranglers will doubtless disagree :P) is that this easily deserves S2 not for the broken functionality, but for the significant impact of lost user time.
I could restore my lost cookies from a relatively-recent backup (but only at the expense of losing all of the history since then since this appears to be yet another thing that's been moved into places.sqlite), but most users will experience "unexpected" behavior, including being forced to re-login to all sites, potentially losing access to some without a password reset; and after wasting hours on that will then be hit by the same bug again the next time they close FF even after confirming that it's correctly set up to not clear those cookies.

Comment 5

[BugBot [:suhaib / :marco]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 6

[pf.arqon]

ugh - I also lost all the new bookmarks since then, of course.

I know it's too late to hope for any return to managing data in sensible groups again, and it may be that there's just too much data for that to work any more, but this is another example of why the "put ALL the eggs in the same basket!" trend of the last few years could do with some reconsideration at some point.

Comment 7

[:Gijs (he/him)]

(In reply to pf.arqon from comment #4)

MHO (though the wranglers will doubtless disagree :P) is that this easily deserves S2 not for the broken functionality, but for the significant impact of lost user time.

Ironically, the fact that you set this likely meant nobody saw it until now.

Johann, triage ping?

Comment 8

[Johann Hofmann [:johannh]]

301 to Paul

Comment 9

[Emma Zühlcke [:emz]]

I can reproduce this issue. I suspect this is happening, because "site preferences" will also clear site permissions. Setting a cookie exception for a site sets the "cookie" permission. When Firefox clears "Site preferences" on shutdown, it will also clear the custom cookie permissions which protect sites from having their cookies deleted by the "Delete cookies and site data when Firefox is closed" option.

With the current implementation, if you want exceptions from the cookie lifetime policy (the checkbox), you can't set Firefox to clear "Site settings", because that's where they are stored in these site settings.

For Bug 1681493 we want to consolidate the checkbox and the sanitize on shutdown feature. If there is time, we can also take a look at this bug.

Comment 10

[jscher2000]

This item has been renamed from "Site Preferences" to "Site Settings" but users still do not understand what it means. Can it be renamed to something like "Site Permissions"? I believe it covers some other data like download paths and zoom levels, but permissions are the most important ones.

Comment 11

[Emma Zühlcke [:emz]]

(In reply to jscher2000 from comment #10)

This item has been renamed from "Site Preferences" to "Site Settings" but users still do not understand what it means. Can it be renamed to something like "Site Permissions"? I believe it covers some other data like download paths and zoom levels, but permissions are the most important ones.

Fair point! We are considering revamping this dialog, changing the categories to make them easier to understand and adding descriptions. Stay tuned!

Comment 12

[Thorin [:thorin]]

We are considering revamping this dialog

FYI jscher, that be Bug 1765533

Comment 13

[qaj1shi]

is there coming a fix for this issue, Paul?

Comment 14

[Daniel Veditz [:dveditz]]

FF is deleting them based on the Site Preferences checkbox instead, which is supposed to just be for zoom levels etc.

Why do you think it was "supposed to be" for just those? That category has always included your "preference" for whether a site could use cookies or not, or your "preference" for whether a site is allowed to do geolocation, use your camera, etc. If you "Forget about this site" we clear all these things, because they are all traces that you had visited that site.

The above is an honest question: if we can figure out what misled you maybe we can do a better job of explaining the functionality. For instance, as jscher mentioned above we have since renamed it from "Site Preferences" to "Site Settings" in an attempt to make this clearer. The new name matches the terminology Chrome and Edge are using for similar things and the consistency may help folks who come to Firefox from those browsers. If you go to Chrome's Site Settings (chrome://settings/content -- in Chrome, not Firefox) you'll see a collection of things including Permissions and "content settings". They put zoom levels under "Additional Content Settings".

If we rename it again to "Site Permissions" then people may not understand it includes non-permissions like zoom levels.

We're unlikely to split these into two different checkboxes/prefs -- what we've got is already pretty cluttered! Maybe the answer is to use more words in the UI, like: "Site Settings and Permissions"

Comment 15

[Hannah Peuckmann]

I’ve discussed this with Paul today. I will try to recap our thoughts on this:

• Since more people will be using sanitizeOnShutdown after deprecating the network.cookie.lifetimePolicy we should find some solution (although ‘site settings’ is not automatically selected after the migration).
• If we would exclude cookie permissions from the 'site settings' category we could persist the users exceptions and users could still clear them manually via the exceptions manager, but it would not be possible anymore to clear them automatically each shutdown.
• If we would exclude them from the site settings category we could consider only doing so for sanitizing on shutdown and still clear them when sanitizing the running system, but this could also lead to confusion (maybe add an info text that this will also clear the users cookie permissions)
• Also, clearing permissions but leaving the cookie permissions seems like we are not clearing everything and leaving data behind (some user might want to have deleted).
• Might be worth it to think about other ways to handle users exceptions than handling them as permissions.

Comment 16

[Thorin [:thorin]]

(maybe add an info text that this will also clear the users cookie permissions)

I don't want to go OT. Apologies for the noise. Maybe this is better under Bug 1765533

Suggestions

• Move the UI history section up under Cookies + Site Data (i.e nudge logins down)
• consolidate onShutdown + cpd dialogs and simplify code
  • deprecate .downloads (it has no UI anymore and in the the cpd dialog if .formdata is checked, downloads changes to that as well) and just use .formdata
  • combine .cookies + .offlineApps and .sessions (i.e deprecate the last two)
  • use the privacy.clearsitedata.cache.enabled code for .cache code so it respects site exceptions
    • I'm not sure where the engineering is on this: perf? hooking up time ranges? or if it's feasible
    • at the moment "delete cookies and site data" flips cache and implies exceptions work

simplified panels

History
[ ] Browsing and downloads <- .history
[ ] Form and search <- .formdata (also toggles .download)

Site Data (respects Site Exceptions)
[ ] Cookies and site data <- .cookies (also toggles .offlineApps, .sessions)
[ ] Cache <- .cache

Site Settings
[ ] Site exceptions and permissions <- .siteSettings

I am not a fan of making an exception for sanitizing site exceptions and going all inception on users. Keep it simple. I doubt is many users would want to wipe site settings: many permissions can be granted as temporary, and advanced users can use permissions.memory_only. Fix the UI, you fix the misconceptions.

Comment 17

[pf.arqon]

"Something" about this appears to have changed in the last few weeks, and looks like a massive regression from a user standpoint: Firefox wiped ALL cookies every time it was closed, logging me out of every site I use them for. This is withOUT any sort of "clear history" settings, and all of those sites set to a permanent "Allow" state.

I thought it was just a random glitch the first time, but it happened multiple times, including after removing cookies.sqlite in case that was damaged.

The workaround seems to be to turn "clear history" ON, and to INCLUDE cookies in it, which I would hoped anyone could see is more than a little counter-intuitive. The ones set to "Allow (Permanently)" do correctly survive under those conditions, but don't appear to under any others.

Comment 18

[pf.arqon]

(In reply to Daniel Veditz [:dveditz] from comment #14)

FF is deleting them based on the Site Preferences checkbox instead, which is supposed to just be for zoom levels etc.

Why do you think it was "supposed to be" for just those?

The above is an honest question: if we can figure out what misled you maybe we can do a better job of explaining the functionality.

erm... because cookies explicitly have their own controls for deletion etc? And have had for over a decade?

I don't think I've been "misled" at all. To be honest, since you asked, ISTM that you're the one with a misunderstanding of how these pieces have worked, consistently, for a very very long time now. You may argue that the behavior isn't what you personally think it should be, but it's what it was, and it was intentional.

While I understand the desire to change UI and behavior in an attempt to bring FF closer to what your vision for "How Things Should Work" is, what we actually have here is a significant change in behavior that is (or was at the time of the initial report) unplanned, undocumented, and "harmful" to users. If that doesn't count as a bug to you, then I'm kind of at a loss as to what would.

For instance, as jscher mentioned above we have since

"since". :)
Regardless, I care far less about adjusting the labels on things to match Chrome than I do the changes in behavior and the current incorrect functionality.

the consistency may help folks who come to Firefox from those browsers.

It may, should such a thing happen, but that's not my concern. Look at it from a simple user perspective: Thing X worked, as designed as documented, for a very large number of years. Then, someone broke it. That's it. That's the whole story. The various UI changes in the vicinity may well have caused the breakage, but they either weren't part of the plan or someone got very confused when they specced this out. :)

We're unlikely to split these into two different checkboxes/prefs -- what we've got is already pretty cluttered! Maybe the answer is to use more words in the UI, like: "Site Settings and Permissions"

I don't think so. I think the problem is that either (a) there's a bug in it; or (b) words don't actually mean what they say.

That aside, let's get to what matters: the goal that we're trying to reach. Which is that, when FF is closed:

1. Cookies that the user wants to keep, for site logins etc, should be kept.
2. Cookies that the user has not made an explicit choice about should follow the global setting for their persistence.

Are we agreed that that is the behavior we want? If not, why not? (i.e. What's the justification for not allowing users to have some degree of control over privacy without it having to be a Scorched Earth all-or-nothing that will force them to re-login and change on-site preferences like Dark Mode etc every single time they restart the browser?)

Assuming there are no objections to that as the correct story to follow, the following statement is true:

1. FF should clear all cookies NOT marked as "Allow", iff "Clear History" is checked, AND "Cookies" is checked within that.
   Let's keep it simple and start with just that. Do you agree that FF clearing ANY cookies if NEITHER of those is true is a bug?

Comment 19

[Hannah Peuckmann]

(In reply to pf.arqon from comment #17)

"Something" about this appears to have changed in the last few weeks, and looks like a massive regression from a user standpoint: Firefox wiped ALL cookies every time it was closed, logging me out of every site I use them for. This is withOUT any sort of "clear history" settings, and all of those sites set to a permanent "Allow" state.

I thought it was just a random glitch the first time, but it happened multiple times, including after removing cookies.sqlite in case that was damaged.

The workaround seems to be to turn "clear history" ON, and to INCLUDE cookies in it, which I would hoped anyone could see is more than a little counter-intuitive. The ones set to "Allow (Permanently)" do correctly survive under those conditions, but don't appear to under any others.

That really sounds strange, could you please tell me the Firefox version that you are using and have experienced this issue with? I am trying to reproduce this. Just to make sure I understand the issue right, Firefox is clearing your cookies without privacy.sanitize.sanitizeOnShutdown being set to true? Does this happen every time you close Firefox?

Comment 20

[pf.arqon]

(In reply to [e-mail] from comment #19)

That really sounds strange, could you please tell me the Firefox version that you are using and have experienced this issue with?

102.0 (64-bit) currently, on an Ubuntu install. Obviously though, the bug first presented back in 79, per the initial report.
Since then, having developed a workaround that gave the correct outcome, I've ignored the bug because I know nobody is working on it and spamming "still broken" comments into the thread every month doesn't benefit anybody. I only returned to it because the behavior appears to have changed again, from "strange and broken" to a different "strange and broken". :)

Just to make sure I understand the issue right, Firefox is clearing your cookies without privacy.sanitize.sanitizeOnShutdown being set to true?

I should probably emphasize that both this bug, and the original workaround for it, and the new workaround for it, have all been arrived at via the UI only - again, per the original screenshots; and also the one from today that I'll be uploading in a second.

sanitizeOnShutdown is enabled, because some aspects of session activity are intended to be cleared: the (literally hundreds) of tracking cookies that accumulate over a day or so on the modern web, for example. We're not talking about general over-eager sanitization here, but rather over-eager sanitization of specific subgroups that have distinct UI for control of them which is not being honored. Hopefully that's clearer?

I suspect that at heart this is a fairly simple case of a commit in 79 that resulted in incorrect precedence in the sanitization code, but I don't have the familiarity or the time to really dig into it. The more recent issue similarly appears to just be a trivial logic error, but the time cost of experimenting with it is enormous from a user perspective (having to locate and re-enter usernames and passwords for every site) so I'm limited in how much I can experiment with it right now.

Does this happen every time you close Firefox?

Yes.

Comment 21

[pf.arqon]

Attachment: Settings needed to NOT have permanently-'Allow'ed cookies incorrectly removed on exit

So yeah, this one is especially bizarre. :)
If the Cookies checkbox is not ticked, 102 deletes supposedly-permanent cookies on exit. To have FF keep them, the UI requires that "Clear all ... Cookies" is ticked.

It's possible that what's going on here is more involved than it appears from the outside, but the impact of this bug is large enough to strongly discourage further experimentation. That needs a new profile with a suitable test server in place, and I just don't have the free time for that right now. (Getting real accounts auto-suspended for spamming logins at them would be bad...)

Comment 22

[pf.arqon]

hmm - thinking about it from a programmer perspective, ISTM that sanitizeOnShutdown shouldn't really even exist in the first place.

There is dedicated UI for each aspect of history that the user may or may not want to keep across sessions, and on the code side of things each piece of it should similarly be specific and dependent on that related piece of UI. Even ignoring these two bugs, the "global" sanitize flag is at best almost completely redundant.

Isn't the flag really just an artifact left over from simpler times? I'm not sure I see any reason for it other than to populate the base about:preferences#privacy checkmark (which should really be a tri-state / partial checkbox), which can be inferred from an OR of the flags set in the corresponding Settings dialog.

The only downside to that is that - as a one time cost - a user who wants to turn on full scrubbing would have to expend a minuscule amount of additional effort. I'm not sure how realistic that case is though, given that Private now exists - and, on reflection, even the "extra effort" part of that is untrue, because transitioning the main checkbox from partial to full covers that case anyway. \o/

Still, that's getting a little too far into the future, I think. I only mention it at all because I know there are plans to overhaul both the UI and the behavior at some point. Let's worry about the getting the current implementation to match the current UI first. :)

Comment 23

[jscher2000]

(In reply to pf.arqon from comment #21)

Created attachment 9287168 [details]
Settings needed to NOT have permanently-'Allow'ed cookies incorrectly removed on exit

So yeah, this one is especially bizarre. :)
If the Cookies checkbox is not ticked, 102 deletes supposedly-permanent cookies on exit. To have FF keep them, the UI requires that "Clear all ... Cookies" is ticked.

Firefox 102.0 had a somewhat clumsy implementation of the coordination between

Cookies and Site Data
[X] Delete cookies and site data when Firefox is closed

and

History
[X] Clear history when Firefox closes

Firefox 102 would migrate the first setting to the second at the next startup (and then leave the first setting is unchecked). See bug 1777419. It works much better in Firefox 103 where the UI for the two features is synchronized in real time.

Comment 24

[pf.arqon]

(In reply to jscher2000 from comment #23)

Firefox 102.0 had a somewhat clumsy implementation of the coordination between
[X] Delete cookies and site data when Firefox is closed
and
[X] Clear history when Firefox closes

So the suggestion is that the second form of this bug is "really" 1777419? I don't think that tracks, given the observed behavior, though as you say it's generally not great to have two checkboxes for the same functionality at all, let alone also having them out of sync. :)

Regardless of whether they were ANDed, ORed, or different pieces of code just randomly picked one over the other, the end result in 102 remains incorrect, since FF still deletes cookies with none of the related boxes checked. While the UI will continue to be problematic until all these pieces are consolidated, my problem is, and always has been since the initial bug filed against 79, that the behavior is simply not matching what the UI says will happen.

I understand that you think the current UI is less than ideal, and I agree with that assessment. However, I want to ensure that we don't fall into the trap of using the UI flaws to justify not actually addressing the underlying bug(s) in the logic, and opt to just kick the can down the road until such time as it can all be redone "right". The issue isn't that the labels are poorly phrased, or poorly located, or similar cosmetic trivia: it's that the code behind that is misinterpreting the settings.

Since we now have a second set of "unexpected behavioral changes" as of 102, it seems that this area is surprisingly prone to collateral damage, and also that there is neither AT nor manual testing of it. I understand that it's a lot more stressful to work on code without that safety net, but the same will hold true for the "rewrite ALL the things!" long-term goal. My concern is that the bug is a problem today, and deflecting it as "just a UI issue", or deferring it until a wishlist rewrite, puts any resolution of it at "years from now" at best.

Since 1777419 resets the "Delete cookies and website data when closing Firefox" user preference, we know that only the checkboxes shown in yesterday's screenshot are relevant. I've just checked that setting, and it has indeed been cleared here as well, so that's now also excluded as a possible cause. (Which I expect you already knew, but I didn't). That notwithstanding, it's obviously worth checking the behavior in 103 in case it's transformed again. I'll do that over the next few days and update this accordingly.

Comment 25

[pf.arqon]

Okay - the bug is still present in 103, in the same more-broken form that it turned into with v102.

It's hard to believe that this isn't somehow an error on my part, but https://bugzilla.mozilla.org/show_bug.cgi?id=1777419#c14 shows that I'm not alone in this.

I can get the correct behavior on a clean profile (at least, I'm fairly sure I did - I'm starting to lose track at this point). I cannot get it to work on the existing profile by any UI means whatsoever.
Obviously, having to nuke the profile from orbit is last ditch act of desperation, and very much not one I'm keen on after collecting years worth of bookmarks, logins, and history.

I would guess that there was an additional error beyond the obvious one in the 102 changes, which has now set something "secret" to the wrong state. (Seems odd that the UI is failing to recover things properly when changed, but I can picture that happening if the tests are for e.g. (pref==2) when existing profiles have the value as 1, or something along those lines).

I don't see any diffs at all in about:config for "saniti", nothing suspicious-looking for "cookie", and too many prefs to count both in total and (apparently) customized at some point for "privacy".
If there are any specific pieces you'd like me to investigate, let me know.

Comment 26

[pf.arqon]

To be clear: the workaround that I was using in 102 no longer works in 103. It's now always all or nothing as far as cookies go, which makes it devastating for either privacy or convenience.

Comment 27

[jscher2000]

Attachment: Coordination of settings involved in deleting cookies and site data at shutdown

(In reply to pf.arqon from comment #26)

To be clear: the workaround that I was using in 102 no longer works in 103. It's now always all or nothing as far as cookies go, which makes it devastating for either privacy or convenience.

What was the workaround? Do your selections on the Settings page match mine (see image)?

I was a longtime user of this setting and the transition has worked as expected for me, as in your test profile. Firefox deletes all the cookies not protected by the exceptions list. Also cached web content, but that's a separate discussion.

Comment 28

[Hannah Peuckmann]

(In reply to pf.arqon from comment #26)

To be clear: the workaround that I was using in 102 no longer works in 103. It's now always all or nothing as far as cookies go, which makes it devastating for either privacy or convenience.

This Bug is about the behavior of the site settings checkbox, that it includes clearing cookie permissions, and how we want to handle this behavior in the future.
The discussion for the new issue that you experience, that is that Firefox deletes all your cookies regardless of the permissions you set (as far as I understand it from your reports) deserves a separate bug.
Please file a new bug for this new issue you experience and include about:support data, since we are not able to reproduce the issue this will be helpful.

Please try to keep the discussion here about the topic of the bug, how to handle the clearing of cookie permissions under the site settings category, Thank you :)

Comment 29

[pf.arqon]

What was the workaround?

If you mean "for 102", see #21.

Do your selections on the Settings page match mine (see image)?

Also see #21. Complete with pictures. :)

Comment 30

[pf.arqon]

This Bug is about the behavior of the site settings checkbox, that it includes clearing cookie permissions, and how we want to handle this behavior in the future.

No, this bug - which I filed in the first place, but thanks for telling me what you think I meant - is simply about Firefox unintentionally changing documented behavior.
As I said back in #18: "Look at it from a simple user perspective: Thing X worked, as designed and documented, for a very large number of years. Then, someone broke it. That's it. That's the whole story".
To Mozilla staff though, the breakage seems to be either "You're holding it wrong" or "We have always been at war with Eastasia".

The nebulous changes that you feel would result in a better UI are an aspect that you introduced to the ticket. While I appreciate your desire to improve the product and your willingness to share those idea here, ISTM that if anyone is trying to co-opt this bug, it's... well, let's just say it's not me. :)
Not saying I don't agree with you that it could be better, but that's very much not the point of this bug and never was.

While you could (and did) argue that the two bugs aren't directly related, I don't think it's unreasonable to tie them together into a larger issue, namely that this is the second time in two years that this has happened - i.e. that the same settings produce different results in consecutive releases, with no updates to any related documentation including release notes, and with a severe negative impact on users.
Having the response to both bugs be predominantly just attempts to gaslight users with claims that "the boxes don't mean what we said they did, and it's always been this way" in the face of years of evidence to the contrary is incredibly insulting.

If you prefer to pretend that sort of thing somehow "doesn't count" as a bug then you are of course free to do so. Similarly, if you want to ignore the bug in favor of pushing for yet another round of UI rewording then by all means go ahead. I don't see how that would actually fix either of the bugs, or stop the code from being prone to suddenly not working in new and creative ways on a regular basis, or even why you think "We'll get it right this time", but knock yourself out.

On a related note though, I did have an insight as to root cause earlier this evening, which is simply this: attempts to improve the feature over the years via more granular settings have led to a situation where nobody actually knows what the code "should" be doing any more. As a result, every time someone touches it they just go with what they personally think should happen.
Since there's no testing to validate it against the behavior of the previous release, and the UI is confusing at best, it's not really a surprise that even some of the Mozilla employees commenting on this bug have shown that they have absolutely no idea what the behavior actually is, or was, despite a willingness to loudly proclaim what they imagine it is.

Your diagram in https://bugzilla.mozilla.org/show_bug.cgi?id=1765533#c1 is potentially helpful (ignoring the issues annevk raised) but it's not clear to me whether that's based on what you want to see or how things currently are.

The discussion for the new issue that you experience, that is that Firefox deletes all your cookies regardless of the permissions you set (as far as I understand it from your reports) deserves a separate bug.

No, that's not what the comment was. The comment was that, as with the bug in v76, the settings in the UI were unchanged, but Firefox's behavior was not. Cookies that had correctly been kept for the past two years were now being deleted.

Please file a new bug for this new issue you experience and include about:support data, since we are not able to reproduce the issue this will be helpful.

Nah, I'm done, thanks. You're not "not able to reproduce the issue": the only post in here on that front ignored the bug report and tested a different set of settings based on, again, what they thought the behavior should be, for a case that is not the one described here. There's very little point to me wasting my time filing a second bug when it's already been made clear that the response is just going to be a mixture of denial, claims that are factually incorrect, and "well, we want to change it all anyway so let's talk about that instead".

As for the v102 bug, AFAICT it looks like "Active logins" was the cause this time, though I did reset a couple of preferences that were different (changed by FF, not me) at the same time. As you can see in the screenshots from two years ago "Active logins" wasn't a factor back then - though I'm sure someone will claim that it's always been the case despite those, just as they did for the original "Site Preferences" cause back in v76.
That's only a tentative diagnosis, but since I can't afford the time that the cookie wipe costs me it's as good as we're going to get unless someone from Mozilla steps up - which brings us to...

It feels like I'm the only one in this discussion who's willing to even admit to the bugs' existence, let alone investigate or test any of it. The responses from Mozilla staff have been big on trying to redirect the conversation to future plans, but completely devoid of any acknowledgement of responsibility for the changes that have already happened.
While I fully support the idea of undoing some of the UI mistakes that have been added over the years, the tone is overwhelmingly "We want to redesign and rewrite all this because that's fun, but fixing bugs isn't, and admitting that we keep breaking this piece makes us look bad". I don't have the time or energy to fight against that sort of mindset, so I think I'll just bow out here.

Good luck with the rewrite, if/when that happens. Hopefully it'll also include a sensible way to add simple exceptions instead of always requiring users to enter URLs by hand; and default to sorting the list by site name rather than the useless Allow/Block field ordering with no secondary sorting, to name but two of the UX failures in the current approach - but that's a topic for another day. :)

Comment 31

[jscher2000]

(In reply to pf.arqon from comment #30)

It feels like I'm the only one in this discussion who's willing to even admit to the bugs' existence, let alone investigate or test any of it.

Nobody is denying that clearing Site Settings (formerly Site Preferences) clears permissions, including cookie permissions. That has been true for a long time, judging from the code in these versions:

• Firefox 78 ESR: https://searchfox.org/mozilla-esr78/source/browser/modules/Sanitizer.jsm#494
• Firefox 68 ESR: https://searchfox.org/mozilla-esr68/source/browser/modules/Sanitizer.jsm#492
• Firefox 60 ESR: https://searchfox.org/mozilla-esr60/source/browser/modules/Sanitizer.jsm#574
• Firefox 45 ESR: https://searchfox.org/mozilla-esr45/source/browser/base/content/sanitize.js#480

Since it is intentional, the question is whether to change this behavior -- for example, to treat permissions separately from other footprints of your browsing, or to treat cookies specially in contrast to everything else Firefox clears through this checkbox -- or to keep it as is and work on the UI to reduce unintended data loss.

Hopefully it'll also include a sensible way to add simple exceptions instead of always requiring users to enter URLs by hand;

I use the Page Info dialog (Ctrl+i), Permissions panel, myself. But sometimes this isn't general enough, for example, you may need to create an exception for a base domain while the Page Info dialog saves it for the full host name.

I mentioned creating exceptions in a Reddit thread the other day and someone chastised me for making obsolete suggestions in light of Total Cookie Protection. I think people find cookies, and their role in tracking, difficult to understand.

Comment 34

[Cosme Benito]

It's not clear or intuitive that clearing the site settings also deletes all the exceptions. Is there a reasonable use case where the user wants to delete all exceptions when Firefox is closed? I would argue such a use case is a very marginal outlier and not the expected use case.

"Site settings" is a very broad and generic term which could mean almost anything, especially to regular users. It should at very least warn me that all my website exceptions are going to be permanently deleted.

Comment 35

[BugBot [:suhaib / :marco]]

The severity field for this bug is set to S3. However, the following bug duplicate has higher severity:

• Bug 1895058: S2

:hsohaney, could you consider increasing the severity of this bug to S2?

For more information, please visit BugBot documentation.

Comment 36

[Emma Zühlcke [:emz]]

William, would you like to look into this bug? It's a complex one so we can also meet and discuss it more.

We need to find a way to persist site exceptions when clearing data on shutdown. Currently the permissions that are used to store the exceptions get cleared as part of the "site settings" category.
Here is the clear data call in the siteSettings category: https://searchfox.org/mozilla-central/rev/0529464f0d2981347ef581f7521ace8b7af7f7ac/browser/modules/Sanitizer.sys.mjs#650,656
We eventually end up in the ClearDataService here: https://searchfox.org/mozilla-central/rev/0529464f0d2981347ef581f7521ace8b7af7f7ac/toolkit/components/cleardata/ClearDataService.sys.mjs#1299 which tells the permission manager to clear all permissions.
We need to find a way to exclude the exception permissions, but only when clearing data on shutdown.

An alternative would be to move the clear-on-shutdown exceptions to a different store than the permission manager. This would also resolve Bug 1767271 where we use the same permissions for anti-tracking exceptions and for clear-on-shutdown exceptions which is quite unexpected. This comment elaborates a bit on a possible solution: https://bugzilla.mozilla.org/show_bug.cgi?id=1767271#c7

Comment 38

[William Wen (:wwen)]

Yeah, I can pick this bug up. I'll get familiar with it today and we can discuss it whenever.

Comment 39

[Emma Zühlcke [:emz]]

For this bug I suggest to split off a new cleaner from the PermissionsCleaner so we effectively have two cleaners:

• Permissions cleaner: Removes all permissions except for the "cookie" permission.
• Clear on shutdown exceptions cleaner: Remove only "cookie" permissions.

The new cleaner needs a flag in nsIClearDataService. Since we don't want to clear exceptions in any of our clear data UIs the new cleaner flag should not be added there (e.g. Sanitizer.sys.mjs).

The work here should also fix Bug 1898533. By not including the new cleaner's flag we should also skip over "cookie" permissions for "Forget about this site".

The rest of the work, separating out clear on shutdown exceptions into its own permission, should be done in Bug 1767271. There we should also consider updating the UI so that clear on shutdown permissions get their own management dialog.

Comment 41

[William Wen (:wwen)]

Attachment: Bug 1658094 - Add new cleaner for "cookie" permissions. r=pbz

Comment 42

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d85a0c432b59
Add new cleaner for "cookie" permissions. r=pbz

Comment 43

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/d85a0c432b59

Comment 44

[Daniel Bodea [:danibodea]]

The original steps to reproduce are these:

1. Specify a site as "(permanently) Allow" for cookies.
2. Set FF to clear cookies (i.e. from sites NOT permanently allowed) on exit.
3. Check "Clear history when Firefox close", ensure "Cookies" is NOT checked.
4. Close FF.
   Actual results: Cookies from Allowed site(s) are all deleted.
   Expected results: Cookies from Allowed site(s) should not be deleted.

The original steps are invalid and incomplete because:

1. If the user <<sets FF to clear cookies on exit>> (from Cookies and Site Data section) and then <<checks "Clear history when Firefox closes" and ensures "Cookies" is NOT checked>> (from the History section), it will result in unchecking the setting to first <<clear cookies on exit>> (from Cookies and Site Data section), so "Cookies and site data" checkbox should remain checked!
2. Site settings checkbox is not mentioned in step 3, which is the cause of this issue.

The (supposedly) correct steps to reproduce would be:

1. about:preferences#privacy / Cookies and Site Data / check the "Delete cookies and site data when Firefox is closed"
   then click "Manage Exceptions..." and add a website as permanently allowed.
2. about:preferences#privacy / History / check the "Clear history when Firefox closes",
   then click "Settings..." and ensure that the "Cookies and site data" checkbox remains checked!
   and ensure that the "Site Settings" checkbox is also checked.
3. Close FF.
   Actual results: Cookies from Allowed site(s) are all deleted.
   Expected results: Cookies from Allowed site(s) are not deleted.

Using these steps to reproduce I have managed to reproduce Facebook losing the cookies in Firefox Release v128.0.2 on Windows 10 and Ubuntu 22. Furthermore, if the "Site Settings" is not checked, then the Facebook cookies are not lost.

I can confirm the fix in beta v128.0b8 in Windows 10, Ubuntu 22 and MacOS 11. The cookies from the permanently allowed site(s) are not deleted even if the Site Settings checkbox is checked. Thank you.

Comment 45

[pf.arqon]

(In reply to jscher2000 from comment #31)

Nobody is denying that clearing Site Settings (formerly Site Preferences) clears permissions, including cookie permissions.

That's good, given that it's explicitly what the bug was about. It would have been embarrassing to have been wrong for the intervening four-plus years. :)

Thank you for the dumps. I don't care to do the archeology at this point now that the bug is fixed, but FYI: as a developer, the diffs between versions claimed good and the one that introduced the regression are preferable to file snapshots, if you're comfortable enough with git.

... chastised me for making obsolete suggestions in light of Total Cookie Protection. I think people find cookies, and their role in tracking, difficult to understand.

Sorry to hear that. Though you can drop the "I think" there, as it implies opinion, which it is not: we have overwhelming evidence from studies (and testimony under oath, &c) that people do indeed not understand cookies, including the vast majority of webmasters developers. (One of the USA medical-industrial complex clowns coincidentally just today admitted to leaking the HIPAA data of over ~5M people this way for three years). If the fines for that aren't enough to incentivize even basic learning for people whose job it literally is, how can we expect non-technical end users to want to put in the effort? (j/k BTW - it's a $1B company, so it'll pay nothing, but I think the point stands).
This is why sanitization has to work, and has to do so reliably. The consequences for it going wrong can be life-altering - or indeed, life-ending. I know most users don't care, but failures in this area can be devastating for those who need at least some deniability. Watching bugs like this be deflected, denied, and delayed for multiple years over low-value UI riders until everyone just walks away from it exhausted and disgusted has left a very sour taste.

Anyway, to return to why I'm actually here: many thanks to hsohaney for filing the dup; and emz for addressing it so promptly after that. Hopefully none of us will ever have to return to this subject again. :)