1180419 - Cannot load URLs from the address bar if places.sqlite is corrupt

Status: RESOLVED FIXED

(Firefox :: Bookmarks & History, defect)

Description

[Raamachandhiran Vasudevan]

User Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.130 Safari/537.36

Steps to reproduce:

1. Upgraded Firefox from 37 to 39
2. Restarted browser
3. Entered a URL in the address bar
4. Pressed the enter key


Actual results:

The website won't launch. I have disabled hardware acceleration and restarted. Still no change.

Comment 1

[Loic]

Could you try the basic troubleshooting steps, please:
1) safe mode: https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-mode
2) fresh profile: https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles

Comment 2

[[:philipp]]

i'm confirming this due to the number of similar questions we see in support. reportedly safemode doesn't help but a new profile does. some affected users have figured that it's related to their places.sqlite file.

Comment 3

[Alice Wyman]

(In reply to philipp from comment #2)
> i'm confirming this due to the number of similar questions we see in
> support. reportedly safemode doesn't help but a new profile does. some
> affected users have figured that it's related to their places.sqlite file.

That would explain why a Firefox refresh doesn't fix the problem but removing the places.sqlite file from the Firefox profile(or creating a new  profile) does .... as reported in this thread:
http://forums.mozillazine.org/viewtopic.php?f=38&t=2944871 FF 39.0 can't navigate from address bar

Comment 4

[Vladan Djeric (:vladan)]

Affected users, can you copy your about:support and post it in the comments or attach it to the bug (Add an attachment" above)?
the data in about:support describes your Firefox configuration, including installed extensions

Comment 5

[[:philipp]]

hi vladan, here are a couple of existing user reports with about:support info already attached:

https://support.mozilla.org/en-US/questions/1070539 (under "more system details" on the right; doesn't list modified preferences)
https://support.mozilla.org/en-US/questions/1070278
http://www.camp-firefox.de/forum/viewtopic.php?p=970636#p970636 german)

it doesn't look like it's caused by an extension on first sight. i'll tag all new threads on sumo i come across for easier retrieval: https://support.mozilla.org/en-US/questions/all?tagged=bug1180419&show=all

Comment 6

[[:philipp]]

and another one: https://pastebin.mozilla.org/8838727

Comment 7

[Loic]

Reported on the French board too: https://forums.mozfr.org/viewtopic.php?f=5&t=125128

Comment 8

[Vladan Djeric (:vladan)]

Thanks guys, you're right there doesn't seem to be much common in those about:support configurations aside from Flash 18.0.

Can we get affected users to check their error console output?

It's in the Firefox menu -> Developer -> Browser Console, or Ctrl+Shift+J on Windows, or Cmd+Shift+J on Mac
Also explained here: https://developer.mozilla.org/en-US/docs/Tools/Browser_Console

Comment 9

[Marco Bonardo [:mak]]

I wonder if this add-on would have helped https://addons.mozilla.org/en-US/firefox/addon/places-maintenance/

Having by mail one of those places.sqlite that cause the issue would be extremely useful.

Comment 10

[John Hesling [:John99] (NeedInfo me)]

(In reply to Vladan Djeric (:vladan) -- please needinfo! from comment #8)
> 
> Can we get affected users to check their error console output?
>
NI Vladan 
What is the output needed please. Is it JS & Net Errors ?

I have tried asking some of the people in sumo threads for more information and will wait for responses.
If you wish to post your own responses & questions note the quick registration link is 
https://support.mozilla.org/users/authcontributor 

Please mention in this bug when you have the required information, because the majority posting support mozilla questions will probably find it rather inconvenient trying to provide error output, naturally enough they are looking for quick simple fixes.

(In reply to Marco Bonardo [::mak] from comment #9)
> I wonder if this add-on would have helped
> https://addons.mozilla.org/en-US/firefox/addon/places-maintenance/
> 
> Having by mail one of those places.sqlite that cause the issue would be
> extremely useful.

IIRC from the mozillazine & sumo threads some have tried that without success already.

My own responses in some sumo threads has asked some affected to backup the profile & places sqlite, and others will still have original places.sqlites.
Presumably we could email to Vladan, that would address any privacy concerns because the data would be going to a mozilla address, then NI Vladan to check for the emails.

Comment 11

[Marco Bonardo [:mak]]

(In reply to John Hesling [:John99] from comment #10)
> What is the output needed please. Is it JS & Net Errors ?

javascript errors.

> that would address any privacy concerns because the data would be going to a mozilla address

well, mak AT mozilla.com works :)

Comment 12

[Marco Bonardo [:mak]]

I suspect there's a relation with keywords changes, when text is entered in the locationbar we ask the keywords API about it, it's likely it is throwing for some reason.

The reason could be related to a bogus migration after bug 1125113, that is, the database contained or contains some unexpected data that confuses the API.

I really need a places.sqlite db to debug this.

Comment 13

[Ben Philipp]

(In reply to Marco Bonardo [::mak] from comment #12)
> ...
> I really need a places.sqlite db to debug this.

Can I just send you a link to mine at mak AT mozilla.com?
It's reported to be corrupt by third party tools too, btw...
Can't the critical call just be wrapped inside a "try"-like call, without an error resulting in termination?

Comment 14

[Marco Bonardo [:mak]]

(In reply to Ben Philipp from comment #13)
> Can I just send you a link to mine at mak AT mozilla.com?

Sure, links to download, with optional password, wfm too.

> It's reported to be corrupt by third party tools too, btw...

which tools and which kind of corruption do they report? If it's physically corrupted there's not much we can do...

> Can't the critical call just be wrapped inside a "try"-like call, without an
> error resulting in termination?

First of all, we should find what is the critical call...

Comment 15

[Ben Philipp]

(In reply to Marco Bonardo [::mak] from comment #14)

> Sure, links to download, with optional password, wfm too.
mail sent.

> which tools and which kind of corruption do they report? If it's physically
> corrupted there's not much we can do...
No, it's not like that. Here is what SQLite Shell gave me:
Page 5211: btreeInitPage() returns error code 11
> On tree page 2851 cell 1: Child page depth differs
> Page 5213 is never used
> Page 5890 is never used
> Page 5891 is never used
> row 1878 missing from index sqlite_autoindex_moz_hosts_1
... (4 more), as well as
> row 23911 missing from index moz_historyvisits_placedateindex
...and...
> row 50946 missing from index moz_historyvisits_fromindex
...a bunch.

The FF Plugin SQLite Manager returns pretty much exactly the same upon a complete integrity check.

Comment 16

[Ben Philipp]

Oh, since nobody supplied it yet:
Here's the error from the console that seems to refer to the problem at hand:
> A promise chain failed to handle a rejection. Did you forget to '.catch', or did you forget to 'return'?
> See https://developer.mozilla.org/Mozilla/JavaScript_code_modules/Promise.jsm/Promise
and
> Date: Tue Jul 07 2015 18:32:44 GMT+0200
> Full Message: Error: Error(s) encountered during statement execution: database disk image is malformed
> Full Stack: ConnectionData.prototype<._executeStatement/pending<.handleCompletion@resource://gre/modules/Sqlite.jsm:685:25

Comment 17

[Ben Philipp]

PS: I got mine running btw.
Following the steps here (https://developer.mozilla.org/en-US/doc ... leshooting) I managed to get a repaired version of my places.sqlite db. When I overwrote the corrupt one with it, in my v39 installation, the problem was gone.

The db is now almost half the size than it used to be (61+mb -> 39mb), I don't know what's missing now...
But since I'm not seeing anything that's obviously missing, I'm good.

Comment 18

[Ben Philipp]

Umh... scratch that last one... :(
I got my versions mixed up, it was v38.0.5 where it was working fine (naturally). Error still persists in v39

Comment 19

[Ben Philipp]

the link is https://developer.mozilla.org/en-US/docs/Mozilla/Tech/Places/places.sqlite_Database_Troubleshooting

Comment 20

[Ben Philipp]

Dammit, I didn't mean to send the 2015-07-07 09:43:25 PDT comment about me being mistaken, that message must've been stuck behind the scenes somehow... THAT was me being mistaken, it DOES work; just had overwritten the working copy.

(the system told me something about "mid-air collision" and asked me what I wanted to do, so I chose "only send my latest comment" - but it was ambiguously phrased, and it sent the older message too.

Can I remove the older comment somehow?

Comment 21

[Marco Bonardo [:mak]]

Happy that the troubleshooting page I wrote is useful and you could recover your database. I will look into the file you sent me, off-hand looks like real corruption and somehow we actually hit that corruption with 39 changes. Will let you know.
(I marked your wrong comment as [obsolete])

Comment 22

[Ben Philipp]

(In reply to Marco Bonardo [::mak] from comment #21)
> [...} the troubleshooting page I wrote [...]

Oh that was you too! :) Thank you so much, your help is very much appreciated and goes a long way :)

BTW: Had to change some things, the syntax wasn't quite working for me:
> sqlite3 places.sqlite-corrupt
->
> .open places.sqlite-corrupt
and so on. The console started as sqlite3> too.

How come this way of just salvaging the usable entries aren't part of a standard routine? It seems like many people would be happy with just that, if the alternative is to throw the whole db away.

Thought: let FF offer to salvage usable data, set a flag, backup the old file, recover the working entries. Upon restart, offer to restore the backup, until the user accepts a version -> turn off the flag.

Comment 23

[Vladan Djeric (:vladan)]

(In reply to John Hesling [:John99] from comment #10)
> NI Vladan 
> What is the output needed please. Is it JS & Net Errors ?

THe JS errors as Marco said, but Ben's comment 16 seems to have pinpointed the exception throwing code in SQLite.jsm :)

Marco: 
- are you going to write the patch to catch this Promise rejection, or do you need Yoric to?
- will you have time to work on auto-fixing corrupt Places databases in Q3? I think it would go hand-in-hand with the bug we discussed to update Places DBs to the better-performing page(?) size

Comment 25

[Marco Bonardo [:mak]]

(In reply to Vladan Djeric (:vladan) -- please needinfo! from comment #23)
> - are you going to write the patch to catch this Promise rejection, or do
> you need Yoric to?

First I need to know what is throwing and why, I got some dbs by mail and will look at those today. Some kind of corruptions are not recoverable.

> - will you have time to work on auto-fixing corrupt Places databases in Q3?
> I think it would go hand-in-hand with the bug we discussed to update Places
> DBs to the better-performing page(?) size

I don't know yet, I'm head down in the awesomebar work, on the other side some performance work will affect also the awesomebar so I'm trying to make that re-enter there.
I don't think I have time to work on auto-fixing corrupt databases though. We should basically reimplement the code in comment 19. the hard part is figuring how to detect corruption and when to handle it.

Comment 26

[John Hesling [:John99] (NeedInfo me)]

This bug priority is only marked as normal. 
Yet those affected are unable to use Firefox and the problem is not fixed by a clean reinstall unless the profile is deleted. It is not fixed by a Firefox Refresh.

End users have resorted to deleting profiles or places and those are not very good solutions as they result in important data loss.

What is the fix to recommend for end users ?
From comment 18 & comment 19 following steps in MDN https://developer.mozilla.org/en-US/docs/Mozilla/Tech/Places/places.sqlite_Database_Troubleshooting fixes the issue.
Which actual steps are needed ? 

Would exporting bookmarks HTML renaming places and reimporting be likely to work and have minimal data loss ?

Comment 27

[Marco Bonardo [:mak]]

(In reply to John Hesling [:John99] from comment #26)
> This bug priority is only marked as normal.

We don't use the priority field usually.

> What is the fix to recommend for end users ?
> From comment 18 & comment 19 following steps in MDN
> https://developer.mozilla.org/en-US/docs/Mozilla/Tech/Places/places.
> sqlite_Database_Troubleshooting fixes the issue.
> Which actual steps are needed ? 

All of them, provided the problem is a database corruption.

> Would exporting bookmarks HTML renaming places and reimporting be likely to
> work and have minimal data loss ?

just removing places.sqlite and letting firefox generate a new one should do the same, it will lose history, but bookmarks should be restored properly.

Comment 28

[Ben Philipp]

(In reply to John Hesling [:John99] from comment #26)
> Would exporting bookmarks HTML renaming places and reimporting be likely to
> work and have minimal data loss ?
Well, no, as the whole history would be gone after that, and all the functions relying upon it, like the auto complete functionality (You would only get suggestions out of your bookmarks).
Maybe that's not important for many users, and it's a question of weighing time+effort against results, but technically, the loss isn't "minimal" this way.

Comment 29

[Alice Wyman]

(In reply to John Hesling [:John99] from comment #26)
> What is the fix to recommend for end users ?
I would also recommend just removing places.sqlite and letting Firefox generate a new one, as outlined here: http://kb.mozillazine.org/Locked_or_damaged_places.sqlite#Rebuild_Places_database
This results in lost history but bookmarks should be restored from the latest JSON bookmarkbackup.

Comment 30

[John Hesling [:John99] (NeedInfo me)]

Thanks for the quick reply.

I guess I was also trying to ask the likely impact and damage of this problem.
It is not picked up in crash stats and breaks Firefox totally.
Many affected users may not be able to report the issue.
Many affected users may simply be migrating to an alternative browser.
Presumably any fix will take time and then ride the trains.

I realise a regression is involved, but presumably the places corruption is likely to be an ongoing occurrence. Presumably we will continue to get new cases of broken Firefox over time until the bug is fixed.

Our suggested workaround destroys History. End users will often not realise that will affect the awesomebar, and presumably the new tiles.


> > Would exporting bookmarks HTML renaming places and reimporting be likely to
> > work and have minimal data loss ?
> 
> just removing places.sqlite and letting firefox generate a new one should do
> the same, it will lose history, but bookmarks should be restored properly.

Yes I realised that after I had posted.
I was trying to think about any recent bookmarks that would be lost. But if that were a concern to a particular end user just exporting as HTML and using that as a searchable file probably suffices.   
Anyhow that is an edge case, and may be resolvable by your troubleshooting steps. 
Loosing recent bookmarks probbaly is not a great problem

Comment 31

[Marco Bonardo [:mak]]

(In reply to John Hesling [:John99] from comment #30)
> I guess I was also trying to ask the likely impact and damage of this
> problem.
> It is not picked up in crash stats and breaks Firefox totally.
> Many affected users may not be able to report the issue.
> Many affected users may simply be migrating to an alternative browser.
> Presumably any fix will take time and then ride the trains.

4% of the users have at least one keyword, and this issue only affects a small percentage of those having some particular setup that I'm investigating. Moreover users using keywords are usually more technical savvy, so they know how to ask for help. So it should be a very tiny percentage of disruption. If this issue would be more common, we'd have noticed it in Nightly/Aurora/Beta already, where we have many technical users using keywords.

> I realise a regression is involved, but presumably the places corruption is
> likely to be an ongoing occurrence. Presumably we will continue to get new
> cases of broken Firefox over time until the bug is fixed.

We have tools already in the product for corruption handling, and bookmarks backups. We can't predict some sort of corruptions, cause they are often bound to system issues, memory faults, disk faults, antivirus software and other external causes. So a pretty good and normal fix in some cases can become an issue.

> I was trying to think about any recent bookmarks that would be lost.

Bookmarks are backed up every day by default.

Comment 32

[Marco Bonardo [:mak]]

One of the databases I got through e-mail allowed me to figure out a workaround for the problem.

Thanks to everyone who sent me their files I also could confirm the issue is not due to the schema change in Firefox 39, but rather to the fact urlbar code in Firefox 39 makes evident pre-existing database corruptions.

All of the databases I received had some sort of corruptions, that can be handled (at the moment) through the small tutorial I put here https://developer.mozilla.org/en-US/docs/Mozilla/Tech/Places/places.sqlite_Database_Troubleshooting

So, the fix here will workaround the problem by making the locationbar work (so one can browser to search for a solution), but it won't fix the underlying corruption problem, for that you must check the above link, or if you don't care about history just throw away places.sqlite and let Firefox generate a new one.

Comment 33

[Marco Bonardo [:mak]]

[Tracking Requested - why for this release]: Currently if places.sqlite is corrupt the location bar stops working completely, it's not possible to use Firefox to browse the Web and look for a solution.

Comment 34

[Marco Bonardo [:mak]]

Attachment: patch v1

Comment 35

[Lawrence Mandel [:lmandel] (use needinfo)]

It's pretty bad anytime we need to kick the user out of Firefox to solve a Firefox problem. I'm tracking for all active branches and nomming for ESR38.

mak - Do you know how far back this issue goes?

Comment 36

[Marco Bonardo [:mak]]

(In reply to Lawrence Mandel [:lmandel] (use needinfo) from comment #35)
> mak - Do you know how far back this issue goes?

I don't know how old are the corruptions and I can't know, the code that exposed them to the locationbar landed in 39, bug 1125113.
Note this patch will only workaround the problem making the location bar work, we can't fix such kind of corruptions at the moment.

Comment 37

[Lawrence Mandel [:lmandel] (use needinfo)]

(In reply to Marco Bonardo [::mak] from comment #36)
> I don't know how old are the corruptions and I can't know, the code that
> exposed them to the locationbar landed in 39, bug 1125113.

I take it that means that the issue of the location bar not functioning in this circumstance is a regression in 39.

> Note this patch will only workaround the problem making the location bar
> work, we can't fix such kind of corruptions at the moment.

Understood.

Comment 38

[billko]

Hi, everyone:

I used SQLite manager on my places.sqlite and executed a Check Integrity --> Complete Check - which, if I understand correctly, is the PRAGMA integrity_check - and it did not find anything.  However, I guess I'm the one that started the original thread in the Mozillazine forums.  Would you like a copy of the file so you can dissect it?

Comment 39

[FredMcD]

Found another user; https://support.mozilla.org/en-US/questions/1071079

Comment 40

[Ben Philipp]

(In reply to billko from comment #38)
> Hi, everyone:
> 
> I used SQLite manager on my places.sqlite and executed a Check Integrity -->
> Complete Check - which, if I understand correctly, is the PRAGMA
> integrity_check - and it did not find anything.

Hi,

what happens if you run FF after you removed / renamed your places.sqlite? Does the problem still persist?

if YES -> different problem source after all, or additional bug
if NO -> problem not down to traditional corruption

Comment 41

[Jim Jeffery not reading bug-mail 1/2/11]

I have to ask, how many of the folks reporting corruption issues use CCleeaner or some other 3rd party application to 'clean up' Firefox ?

CCleaner has been known for some time to not play nice with Firefox.

Comment 42

[Ben Philipp]

(In reply to Jim Jeffery not reading bug-mail 1/2/11 from comment #41)
> CCleeaner or some other 3rd party application to 'clean up' Firefox ?

Not me

Comment 43

[Marco Bonardo [:mak]]

(In reply to billko from comment #38)
> I used SQLite manager on my places.sqlite and executed a Check Integrity -->
> Complete Check - which, if I understand correctly, is the PRAGMA
> integrity_check - and it did not find anything.

it must return the string "ok" if it doesn't return anything, it's still a sign of corruption.

Comment 44

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/db1b6b8ed0be

Comment 45

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/db1b6b8ed0be

Comment 46

[Marco Bonardo [:mak]]

Comment on attachment 8631000 [details] [diff] [review]
patch v1

Approval Request Comment
[Feature/regressing bug #]: bug 1125113
[User impact if declined]: location bar doesn't work at all if places.sqlite is corrupt 
[Describe test coverage new/current, TreeHerder]: Nightly
[Risks and why]: very low risk, it's just a try/catch
[String/UUID change made/needed]: none

Comment 47

[Kate Glazko]

Comment on attachment 8631000 [details] [diff] [review]
patch v1

Approving uplift because low-risk and tests passed. Waiting for ESR approval because we want to make sure that it landed/baked in Aurora/Beta for a few days.

Comment 48

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/45221d7387db

Comment 49

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/5d5cb60a6d69

Comment 50

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8631000 [details] [diff] [review]
patch v1

Taking it to esr 38 too.

Comment 51

[[:philipp]]

is esr even affected? comment 46 said the regressing feature was bug 1125113 which landed in 39 and from what i noticed in support the issue only started to pop-up after the firefox 39 release as well...

Comment 52

[Marco Bonardo [:mak]]

Comment on attachment 8631000 [details] [diff] [review]
patch v1

Right, sorry for the noise, this is not needed on ESR38. My fault.

Comment 53

[Marco Bonardo [:mak]]

For some reason this appears in release alert e-mails as tracking-ESR31, but it's not, can someone check please?

Comment 54

[Liz Henry (:lizzard) (relman/hg->git project)]

I don't think we need to track this for 39, since 40 will be released in a couple of weeks.

Comment 55

[[:philipp]]

[Tracking Requested - why for this release]:
untracking requested for esr as well, since it isn't affected

Comment 56

[Kate Glazko]

Untracking for ESR 38 given Comment 52, Comment 51.

Marco, looking into it!

Comment 57

[Kate Glazko]

Untracking for ESR 38 given Comment 52, Comment 51.

Marco, looking into it!

Comment 58

[Sylvestre Ledru [:Sylvestre]]

I reported this bug about the nag issue:
https://github.com/mozilla/bztools/issues/12