1827988 - Most cookies dates are set in the far future when launching Firefox

Status: RESOLVED DUPLICATE of bug 1827669

(Core :: Networking: Cookies, defect)

Description

[dslayer]

Attachment: Cookies_wrong_dates.png

Steps to reproduce:

Updating from 111.0.1 (20230321111920) to 112 (20230406114409) made most cookies seem to disappear. They are here but have wrong dates, last used is set to "in 4,137 years". Safe mode is identical. Bookmarks and such also work as intended. Windows clock is working fine and the antivirus found nothing.

Actual results:

Websites ask you to accept cookies as if there were no valid cookie available. It keeps happening for every site after restarting Firefox. For example, I have to use my phone to validate my login attempt into gmail every time I open Firefox, which normally doesn't happen.

Expected results:

The websites should recognize the cookies, which would have the correct dates. At least, it should be able to keep the new ones working normally, even after restart.

Comment 1

[Dianna Smith [:diannaS]]

We are currently trying to debug this issue in bug 1827669, would you be willing to share the data in your about:support as well as a copy of your cookies DB? In particular we would like to see what prefs you have set to try to identify the culprit.

An alternative would be running mozregression to narrow down a regressor. There's a guide on how to do this on https://mozilla.github.io/mozregression/quickstart.html .

Comment 2

[dslayer]

(In reply to Dianna Smith [:diannaS] from comment #1)

We are currently trying to debug this issue in bug 1827669, would you be willing to share the data in your about:support as well as a copy of your cookies DB? In particular we would like to see what prefs you have set to try to identify the culprit.

An alternative would be running mozregression to narrow down a regressor. There's a guide on how to do this on https://mozilla.github.io/mozregression/quickstart.html .

I may have lost settings while trying some fixes as I only backed up the profile data from Roaming and not Local. But the problem still happens when updating from 111.0.1 to 112 and I start seeing some 4000+ years for "last used", even though the 111 (and 102 esr) did react by resetting the dates to startup time repeatedly, apparently. New cookies are possibly working but may be invisible or otherwise still corrupted. I wasn't expecting any better from a downgrade but it's still more usable than 112 if I need something temporary. I'll keep the backup for the profile anyway.

I am willing to share about:support but not cookies.sqlite, however I can tell you that the only thing that DB Browser could see was AB_test_privacy, atidvisitor and atuserid, 3 rows out of more than 6000 in the backup cookies.sqlite.bak from a few month ago. I guess I'll have to forget about the cookies and site data, but I want the new ones to work well and not have to use older versions as it breaks some things. What can i do now without loosing further data? I'll take a look at mozregression if that can be helpful. Thank you very much!

Comment 3

[dslayer]

Attachment: about_support_112.txt

Comment 4

[Dianna Smith [:diannaS]]

Thanks for helping us debug the issue!

could you possibly check if this build solves the problem for you ?
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/eM-HmnA6QXWHiAoZWpe8Cw/runs/0/artifacts/public/build/target.installer.exe which i took from this try build

Comment 5

[edgul]

In order to test the new build against your existing cookies you will want to point the new build at the old profile. You may want to backup the old profile as well.

1. To find the profile location, using the old version of fx go to the three-line button settings > help > more troubleshooting information > check Profile Directory. Note the name of the profile and back up the entire folder just in case.
2. Install the build provided
3. In the new fx navigate to the profile dir and point it to your old profile (noted in step 1).

It would also be a great help to get some logs if you are able to assist. For this particular problem they will need to be set from command line like so on windows x64:

set MOZ_LOG=timestamp,rotate:200,nsHttp:5,cookie:5
set MOZ_LOG_FILE=%TEMP%\log.txt "c:\Program Files\Mozilla Firefox\firefox.exe"

More details can be found here, but the cookie:5 is the important part here. If you are concerned about personal information in the logs then you can email to [e-mail].

Thanks for the help!

Comment 7

[dslayer]

The provided target version behaved the same as any other version I tried (other than regular 112) and it appears to work but keeps resetting the timestamps to startup time without fixing the root of the problem. It can read some more data than the other version and isn't a downgrade, but it's far from enough to help my situation, apparently. Maybe it DOES work and prevents the corruption in the first place but it doesn't work for me. It keeps happening even when trying to use the old cookies.sqlite.bak or remaking it from scratch, so the problem also happens somewhere else.

I can't test with a non-empty uncorrupted profile, unfortunately, and here I was about the backup my entire computer the very same day... I guess I'll have to try to make a new profile and import what I can and the bug may not happen at all when updating. I don't have any more time to try this now and the log files record way too much stuff with those settings. I hope the problem is really only due to cookies and maybe it will be possible to skip past the issue without having to lose settings or more. Thank you very much.

Comment 8

[edgul]

Thanks for your help on this. When exactly did you make the cookies.sqlite.bak file?

Comment 9

[dslayer]

It was made automatically (I think) on the 19th of October 2022 at ‏‎02:45:04. It's too outdated. I'll see if it can still be recycled in a new profile.

Comment 10

[edgul]

Sorry, I wasn't very clear. I'm trying to figure out if the file contains corrupted data. I'm assuming when you renamed cookies.sqlite to cookies.sqlite.bak you stopped using it as the database for fx.

Which version of fx were you on when you did this and what was the sequence of events?
Ie: Was this before you upped to v112 (pre-release), after your first upgrade to v112, after you downgraded back to v111 or other?

Comment 11

[dslayer]

I didn't rename files more than temporarily. What I referred to as the sqlite.bak was made months ago when Firefox current version was 108-106 (I lost the update history somehow). That file shouldn't have any problem and hasn't been modified since October. I just tried to slap it instead of whatever corrupted version I had at the time by renaming that bad file and then renaming the .sqlite.bak to .sqlite to see if the one which is supposed to be safe makes a difference despite the age discrepancy. It didn't work so I deleted the entire folder one more time to put another backup in it for another test. I obviously still have the initial .bak somewhere else. The problem in my opinion is somewhere else, where you can't reset it by just erasing all cookies and site data (unless maybe done manually, I didn't try because I don't have an exhaustive list of those).

Everything was fine until the switch to 112, I kept the initially corrupted file right after the update (backed up the whole profile from Roaming only), and I also made and use some new functional one (but there are still problems somewhere) that I obtained by erasing cookies and site date (from the settings menu) and then logging in everywhere. I keep those 3 versions of the cookies database around and none of them are the solution. I have another backup for current use that has both Local and Roaming profile data. I'll try to make a new profile later.

Comment 12

[dslayer]

I used mozregression on the corrupted profile and considered it good when the dates were reset and bad when the dates were in 4000+ years. In the "copy" profile mode, it made a new copy every time and it found bug 1819529 which has already been identified and reopened by folks, here. I tried "copy first" mode to check for reversibility but only the build known to be bad triggered the problem while being tested second. I'm putting the information here in case that's useful for somebody. The "copy first" was done twice to check if the opt build made a difference but it didn't. I'm going to try to make a new profile and see if the problem pops up again depending on what I put in from the old one.

Comment 13

[dslayer]

Attachment: mozreg_copy_first_only.txt

Comment 14

[dslayer]

Attachment: mozres_copy_first_only.png

Comment 15

[dslayer]

Attachment: mozreg_copy_each_time.txt

Comment 16

[dslayer]

Attachment: mozres_copy_each_time.png

Comment 17

[Emma Zühlcke [:emz]]

Thank you, this is very useful for confirmation!
If you can spare some more time, it would be helpful to find out if you have cookies in your affected cookies.sqlite db that may trigger this bug. See comment here: https://bugzilla.mozilla.org/show_bug.cgi?id=1827669#c19 .

Comment 18

[dslayer]

I looked at a bunch of variants of cookies.sqlite with the DB Browser and with HxD. The file that was an old backup from October 2022 has normal entries. A few very early creation time like 1538665537126000 rather than something like 132023218005714612, and last accessed is always around 1665020655790000. However, there are a few Expiry that are much higher than the rest like 157185115362 or 253402300799 if that matters. The content looks normal inside the Hex editor at first glance, too.

About the cookies.sqlite right after the update, the DB Browser sees almost nothing in it, and the Hex editor reveals that it's almost entirely 00 values and a few leftovers. I don't understand where they get the 4000+ years into the future, that's not anywhere in the files I have even though the cookie manager within the browser gives those values instead of zeroes or whatever that would read. It should be repeated many time...

When trying to make new cookies, it tends to try to write new entries and then you can see the time updated to startup time inside the file, though you may need to try with a new one for it to work. I still get weird behavior even when it seems the cookies can stick and appear to work. Can I try out those firefox 114 and such somewhere?

The status for the bugs changed a bunch of times. Am I supposed to skip versions or something like that to get a path that works? I'm still probably going to try and make a new profile, because I don't expect fixes to magically resurrect 00s into actual stuff without recent backups. I'll try keeping my settings to see if the bug still happens.

Comment 19

[Emma Zühlcke [:emz]]

(In reply to dslayer from comment #18)

I looked at a bunch of variants of cookies.sqlite with the DB Browser and with HxD. The file that was an old backup from October 2022 has normal entries. A few very early creation time like 1538665537126000 rather than something like 132023218005714612, and last accessed is always around 1665020655790000. However, there are a few Expiry that are much higher than the rest like 157185115362 or 253402300799 if that matters. The content looks normal inside the Hex editor at first glance, too.

About the cookies.sqlite right after the update, the DB Browser sees almost nothing in it, and the Hex editor reveals that it's almost entirely 00 values and a few leftovers. I don't understand where they get the 4000+ years into the future, that's not anywhere in the files I have even though the cookie manager within the browser gives those values instead of zeroes or whatever that would read. It should be repeated many time...

When trying to make new cookies, it tends to try to write new entries and then you can see the time updated to startup time inside the file, though you may need to try with a new one for it to work. I still get weird behavior even when it seems the cookies can stick and appear to work. Can I try out those firefox 114 and such somewhere?

The status for the bugs changed a bunch of times. Am I supposed to skip versions or something like that to get a path that works? I'm still probably going to try and make a new profile, because I don't expect fixes to magically resurrect 00s into actual stuff without recent backups. I'll try keeping my settings to see if the bug still happens.

Thanks! I'm not sure why you don't see cookies with future timestamps in the DB even though they do show up in the site data manager. Have you sorted the columns to ensure there are no larger timestamps for createdAt in the DB?

I recommend skipping 112 if possible and directly updating to the latest release 112.0.1 which should address the main issue. Please let me know if you still run into problems after applying this update.

Comment 20

[dslayer]

I don't know if everybody has the same problem as I do but the bug corrupted the database, as previously stated 99.99% of the file is 00 hex values even though it's 2.5Mb. Now that I've seen it in the hex editor I can actually only see 3 cookie entries that somehow survived the bug and they are for some mail server I barely use, so I guess it doesn't represent a big privacy issue to share it. I'll post it as "cookies_bad.sqlite".

I already tried several things with the "fixed" version but it does nothing to me. Whether I start from 111 or try to directly install 112.0.1 to skip the bug, the issues of restarting timestamps (and possibly other hidden defects) aren't solved. Maybe that did it for others but apparently the new version just removed some older fix so it's not too surprising it doesn't magically restore all functionality for me. Maybe the timestamps and the problems that keeps popping up are situated somewhere else, like site data that doesn't get reset as easily or some prefs.

Anyway, I'll post the files that I find safe to reuse from the current profile while making a new one that hopefully won't be as weird.

Comment 21

[dslayer]

Attachment: cookies_bad.sqlite

Comment 22

[Emma Zühlcke [:emz]]

Thanks for sharing the DB file!
I suspect that the file is this big because of old entries marked for deletion, perhaps due to invalid purging. When I run VACUUM on it the file size reduces to ~ 100kb. With the bug in Fx112 Firefox may have purged cookies wrongly for you.
While 112.0.1 doesn't fix the invalid timestamps itself we don't expect it to do any additional invalid purging. To confirm, if you use a DB that has never been loaded in Fx112 (but an older version) and you directly upgrade to Fx112.0.1, do you still loose cookies in the process?

but apparently the new version just removed some older fix

The patch that has been backed out for Fx112.0.1 is (/was meant to be) a test only change, so we're not removing a fix. This will also not fix up an already purged / corrupted DB.

Comment 23

[dslayer]

If I slap the cookies.sqlite from October to 111.0.1, it purges some outdated cookies and keep most of them and they seem to work fine. When updating directly to 112.0.1, nothing changes as far as I can see so it does what it's supposed to. I'm not sure it's exactly equivalent, though, and it still messes with the new timestamps at startup as usual. I'd like to minimize the number of potential problems I might encounter later, and the new total cookie protection thing may be worth trying to make this version work, even if I have to erase current cookies and site data.

Comment 24

[Emma Zühlcke [:emz]]

(In reply to dslayer from comment #23)

If I slap the cookies.sqlite from October to 111.0.1, it purges some outdated cookies and keep most of them and they seem to work fine. When updating directly to 112.0.1, nothing changes as far as I can see so it does what it's supposed to. I'm not sure it's exactly equivalent, though, and it still messes with the new timestamps at startup as usual. I'd like to minimize the number of potential problems I might encounter later, and the new total cookie protection thing may be worth trying to make this version work, even if I have to erase current cookies and site data.

You can try deleting any cookie that has a timestamp set to the future from the DB before starting Firefox. That should prevent new cookies that have this invalid timestamps from being created. We're also working on a fix for this in Bug 1828126 which will probably ship in Fx114.

Comment 25

[dslayer]

I don't have weird future timestamps in my old cookies file. I still don't know what triggered the bug and massive purge of the database.

After a lot of messing around with new profiles and prefs, it turns out that the session was somehow corrupted... For someone like me, with always several tabs open when quitting Firefox, I basically never try to start a new session and wouldn't ever want to disable the "restore tabs on startup" setting. If you do, though, the infinite reset of the timestamps at startup stops and you can enable it, again. Remember to bookmark your current tabs if you want to get them back in the new session. I hope there is no hidden problem remaining. Maybe I'll use the next esr build when it gets total cookie protection in the near future...

In the end, I can keep my profile and all. You still lose some data but it's not the most important part, so it's not that bad. Thank you very much!