1530298 - places.sqlite migration on upgrade

Status: RESOLVED INVALID

(Toolkit :: Places, enhancement)

Description

[ildar.mulyukov]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0

Steps to reproduce:

upgrade from 60.x to 63.x

Actual results:

"The bookmarks and history system will not be functional" message. It is documented: https://support.mozilla.org/en-US/kb/fix-bookmarks-and-history-will-not-be-functional
The file is completely correct. The root reason seems in the places.sqlite format change.

Expected results:

The automatic format migration. I.e. FF must convert old file to the new file in correct format.

Comment 1

[ildar.mulyukov]

You may have a script for that, aren't you?

Comment 2

[Marco Bonardo [:mak]]

Is you profile a common local profile on your disk, or is it on a network share? We made a change in Firefox 63 where we try to obtain an exclusive lock on the database, that may affect the behavior if either something else in the system opened the db, or on certain network filesystems.

Supposing it's not a problem with file locking, Firefox does indeed migrate automatically on upgrades, unless there's a problematic data coherence problem in the database (that may have a lot of different and exotic causes). If you don't care much about history, you can just delete places.sqlite (or rename it, better to have a backup) and it will generate a new one, importing bookmarks from the last backup.
You could even try to see what happens if you use the Places database integrity tool from about:support. Note the latest version is Firefox 65, anyway.
Otherwise, you may try to send a compressed copy of places.sqlite to me by mail (can use send.firefox.com), and I can try to debug the exact problem.

Comment 3

[Marco Bonardo [:mak]]

Any other detail that comes to your mind may be useful, like usage of third party tools to access the database, or third party syncing systems like XMarks, or so on.

Comment 4

[ildar.mulyukov]

Marco, thanks for answering.

If you don't care much about history, you can just delete places.sqlite

I know. I also references the support site where it's written.

You could even try to see what happens if you use the Places database integrity tool from about:support.

Oh! thanks. I start using it. So:

unless there's a problematic data coherence problem in the database

shouldn't be. The only peculiarity is the big history:

places.database.lastMaintenance 1550749513
places.history.expiration.max_pages 99999
places.history.expiration.transient_current_max_pages 99999

Then using about:support I verify Integrity in FF 60:

Task: checkIntegrity

• The database is sane

Task: invalidateCaches

• The caches have been invalidated

Task: checkCoherence

• Error(s) encountered during statement execution: cannot commit - no transaction is active

Task: expire

• Database cleaned up

Task: vacuum

• Unable to vacuum database

Task: stats

• The task queue was cleared by an error in another task.

Task: _refreshUI

• The task queue was cleared by an error in another task.

I'll do that in FF63 a few days later.

Comment 5

[Marco Bonardo [:mak]]

(In reply to ildar from comment #4)

• Error(s) encountered during statement execution: cannot commit - no transaction is active

Uh, this is very strange, first time I see this error. Off-hand this looks like a nested transaction :/
Is this a build made by your Linux distribution or a Mozilla build?

Comment 6

[ildar.mulyukov]

this is distro-built, https://packages.altlinux.org/en/p8/srpms/firefox/rpms . Is this important?

Comment 7

[ildar.mulyukov]

I seem to have found the root. This is space constraints: with places db of 43-46MB and free space of 6 MB it just couldn't rebuild the DB.
Sorry for the noise.
P.S. DB Integrity check also fixed.