753874 - FF10 does not import FF3.6 cookies, and sqlite3 3.7.11 cannot read FF10 cookies.sqlite

Status: RESOLVED INVALID

(Core :: Networking: Cookies, defect)

Description

[Ken Winiecki, Jr.]

Migrating from FF3.6 to FF10ESR did not retain any cookies.  Copying FF3.6 cookies.sqlite into a clean FF10 profile did not work, FF10 does not seem to understand FF3.6 cookies.sqlite and overwrites it.  If I lock the FF3.6 cookies.sqlite then FF10 does not overwrite it, but still does not use it.  The FF3.6 cookies.sqlite with 8 cookies is 4 kB; the FF10 cookies.sqlite with 0 cookies is 521 kB of mostly nulls.  After surfing with FF10 to acquire new cookies, a hex editor shows cookies in cookies.sqlite, but sqlite3 version 3.7.11 (March 2012) .dump says it's empty, so it can't be used to migrate them manually either.  The sqlite3_analyzer does show the cookies, but that doesn't help migration.

Comment 1

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

could you send me the cookies.sqlite that fails to be imported by mail? I'd like to take a look at its file format.
Ideally any version of SQLite is able to read any database created with a previous version (the opposite is untrue though).
Actually, could just be that the cookie service thinks that version is so old to not be worth trying to import it, and directly makes a new db.

Comment 2

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

cc-ing Lukas, since this may affect forced upgrades or unsupported versions, if confirmed.

Comment 3

[Ken Winiecki, Jr.]

Attachment: cookes.sqlite works in FF3.6 not FF10

This cookies.sqlite works in FF3.6 but not in FF10.  It was regenerated last April with the then-latest sqlite3.  But it doesn't explain why the latest sqlite3 can't read FF10's cookies.sqlite.

Comment 4

[Ken Winiecki, Jr.]

Oh, SEND by mail...I misparsed "imported by mail".  It's ok though, none of the cookies contain personal data (only personalized data).

Comment 5

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

what does "regenerated" means?

Comment 6

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

at first glance your database should have PRAGMA user_version = 2, but somehow it has user version 0, so maybe you rebuilt this database manually ignoring it's not just the data but also PRAGMAs matter?
From what I see cookies service tries to initialize the db, finds a broken user_version, checks if all the columns are there, but not finding all of them properly decides to start from scratch.

Comment 7

[Ken Winiecki, Jr.]

Sorry, I don't know that much about SQL or databases.  As I understand it, the procedure to modify a cookies.sqlite file is:

sqlite3 cookies.sqlite
.output cookies.txt
.dump
.exit
edit cookies.txt in text editor (occasionally I have to make minor mods to the cookie data)
sqlite3 cookies_new.sqlite
.read cookies.txt
.exit
mv cookies_new.sqlite cookies.sqlite

I called that "regenerating", but maybe there's a more appropriate term.  That's all I've been doing since whatever verson of Firefox created the very first one out of its predecessor's cookies.txt, unless I need a brand new cookie, and then I use whatever the current Firefox of the time generates.  I never mess with the database structure since I don't know nearly enough about it.  And through 3.6.28 it was never a problem.

If sqlite3's similar .dump of FF10's cookies.sqlite gave me something, then I could shoehorn my cookies into it, but it is always empty, even though I can see otherwise with hexedit.

Comment 8

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

(In reply to Ken Winiecki, Jr. from comment #7)
> I called that "regenerating", but maybe there's a more appropriate term.

yeah, this is not enough to properly "regenerate" a database, you are missing metadata, like user_version that applications rely on. There is other data like PRAGMA page_size, for example, that a dump can't copy.  Dump just copies contents and contents are hardly enough to describe how to use them.

> That's all I've been doing since whatever verson of Firefox created the very
> first one out of its predecessor's cookies.txt, unless I need a brand new
> cookie, and then I use whatever the current Firefox of the time generates.

Why you thought you had to do that? You can optimize the database just using the SQL command "VACUUM", while the browser is closed.  But generally there's no need to do that, each new version of Firefox knows how to handle the older versions databases (if they are properly built), and larger databases often don't mean worst performance (indeed it may be the opposite).

> I never mess with the database structure since I don't know nearly enough
> about it.  And through 3.6.28 it was never a problem.

You were just lucky, cause 3.6.x didn't upgrade the database version, it didn't read it as well, so missing metadata went unnoticed through those stability versions.

> If sqlite3's similar .dump of FF10's cookies.sqlite gave me something, then
> I could shoehorn my cookies into it, but it is always empty, even though I
> can see otherwise with hexedit.

if you want to recover your cookies, just do "PRAGMA user_version=2" on the original cookies.sqlite file and try to use it with FF10, it *should* work, but considered it may be missing other metadata, would be better to start from scratch and stop touching dbs... This is clearly valid only for this specific case, not for any sqlite db.

I'm resolving the bug as invalid cause it's due to a manually modified database with missing data, so not a Firefox bug, though if you need further assistance, feel free to ask.

Comment 9

[Ken Winiecki, Jr.]

> Why you thought you had to do that? You can optimize the database just using the SQL command

They were somebody's online instructions for how to manage your cookies without having to learn SQL and buy an SQL editor (my understanding is that OS X does not come with one).

First the Cookie Manager went away, then human-readable cookies.txt went away, and now free-sqlite3-manageable cookies have gone away.  Should not Firefox allow at least *one* method for its users to manage their own cookies without learning SQL and paying for an SQL editor?

Actually I did find one:  using FF3.6.28 to regenerate (properly this time I think) the cookies.sqlite database by having it add a cookie, write the file, delete the cookie, and write the file.  Then it worked with FF10.

Thanks for all the information.