Bookmark folders doesn't show even if they are in the json file
Categories
(Firefox :: Bookmarks & History, defect, P2)
Tracking
()
People
(Reporter: johannes.menzel, Assigned: standard8)
References
Details
Attachments
(2 files)
bookmaks.png
User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:64.0) Gecko/20100101 Firefox/64.0
Steps to reproduce:
- created a fresh firefox profile
- reinstalled firefox including deleting folders in %(local)appdata%
- exported and imported both html and json
- evaluated the exported files (html/json; all looking fine) tree structure (missing folders show up there)
- synced via Sync
Actual results:
Some bookmark folders doens't show in the places firefox bookmarks should show up. When I manually enter a site which was bookmarked before the properties page of this bookmark in the toolbar shows the correct path until I go one level up and down again. The exported files (html/json) show the complete bookmarks, the IDs and index properties show comprehensible values and the tree structure looks fine. The missing bookmark folders don't show in every firefox instance of all of my synced devices (win, linux, android). Searching these pages via toolbar show them as bookmarked.
Expected results:
Well, show the missing bookmarks.
|
Reporter | |
Updated•5 years ago
|
|
|
Reporter | |
Comment 1•5 years ago
|
||
Some users on the web reported similar behaviour, but nowhere a solution can be found. Most answers tell to use the bookmark tools to show the bookmark toolbar.
|
Assignee | |
Comment 2•5 years ago
|
||
Would you be able to share with us a JSON file that exhibits the issue? You can change URLs/names as long as the overall issue is still present.
|
|
Reporter | |
Comment 3•5 years ago
|
||
Is the part of the parent Folder that has missing child folders sufficient?
|
|
Reporter | |
Comment 4•5 years ago
|
||
There you are.
|
|
Reporter | |
Comment 5•5 years ago
|
||
Since it is now anonymized, for little overview see parent folder with guid "gMCBAE2Ysy0d", where children for example with guid "EYwavvvzZhuU" are shown, whilst with "io8wNBXF3uHw", "i24dUJ6vVOEu" aren't.
|
|
Assignee | |
Comment 6•5 years ago
|
||
On importing that data, I'm seeing:
Error: "Error(s) encountered during statement execution: NOT NULL constraint failed: moz_bookmarks.syncStatus"
handleCompletion resource://gre/modules/Sqlite.jsm:815
which is coming from this transaction: https://searchfox.org/mozilla-central/rev/4faab2f1b697827b93e16cb798b22b197e5235c9/toolkit/components/places/Bookmarks.jsm#1790-1833
I checked that the syncStatus for the entire items array is 1.
Marco, any ideas? I was originally thinking there may be some sort of size constraint for the executeCached, but then looking at it, I don't think there's actually significant amount of bookmarks here.
|
|
Reporter | |
Comment 7•5 years ago
|
||
I wasn't sure weather it was a problem that I anonymized the data of almost 1500 bookmarks via prettifying and using sed, so that the strings for "guid": "root________", or "guid": "menu________", also changed. But I guess you boys and girls figured that out.
|
||
Comment 8•5 years ago
|
||
This is what I see in the mozlog that Mark sent me, we are basically inserting a fully null object. I have no idea how that's possible, yet.
[Parent 51770: Main Thread]: D/mozStorage Inited async statement 'BEGIN IMMEDIATE TRANSACTION' (0x0x117a770a0)
[Parent 51770: mozStorage #1]: D/mozStorage Initialized statement 'BEGIN IMMEDIATE TRANSACTION' (0x0x126ccf5e0)
[Parent 51770: mozStorage #1]: D/mozStorage TRACE_STMT on 0x124c286b0: 'BEGIN IMMEDIATE TRANSACTION'
[Parent 51770: Main Thread]: D/mozStorage Finalizing statement 'BEGIN IMMEDIATE TRANSACTION'
[Parent 51770: mozStorage #1]: D/mozStorage TRACE_STMT on 0x124c286b0: 'INSERT OR IGNORE INTO moz_places (url, url_hash, rev_host, hidden, frecency, guid) VALUES
(NULL, hash(NULL), NULL, 0, NULL,
IFNULL((SELECT guid FROM moz_places WHERE url_hash = hash(NULL) AND url = NULL), NULL))'
[Parent 51770: mozStorage #1]: D/mozStorage TRACE_STMT on 0x124c286b0: 'DELETE FROM moz_updateoriginsinsert_temp'
[Parent 51770: mozStorage #1]: D/mozStorage TRACE_STMT on 0x124c286b0: 'INSERT INTO moz_bookmarks (fk, type, parent, position, title,
dateAdded, lastModified, guid,
syncChangeCounter, syncStatus)
VALUES (CASE WHEN NULL ISNULL THEN NULL ELSE (SELECT id FROM moz_places WHERE url_hash = hash(NULL) AND url = NULL) END, NULL,
(SELECT id FROM moz_bookmarks WHERE guid = NULL),
IFNULL(NULL, (SELECT count(*) FROM moz_bookmarks WHERE parent = NULL)),
NULLIF(NULL, ""), NULL, NULL, NULL,
NULL, NULL)'
[Parent 51770: Main Thread]: D/mozStorage Inited async statement 'ROLLBACK TRANSACTION' (0x0x117aa3940)
|
|
||
Comment 9•5 years ago
|
||
I honestly don't see where we check insertInfos length, and if that is an empty array, we may end up not binding anything.
|
|
||
Updated•5 years ago
|
|
|
||
Updated•5 years ago
|
|
|
Assignee | |
Comment 10•5 years ago
|
||
Johannes, although we've found an issue (bug 1523665), I'm not convinced it is exactly the same as your issue.
Please could you try this:
- Go to Tools -> Web Developer -> Browser Console (note: not the "Web" Console).
- That will open up a new window.
- Go to "Show All Bookmarks", and then attempt the import again.
When the import fails, there may be a message or two appear on the Browser Console. If so, could you copy/paste them here?
|
|
Assignee | |
Updated•5 years ago
|
|
|
Reporter | |
Comment 11•5 years ago
|
||
There's no message showing after importing
|
|
Reporter | |
Comment 12•5 years ago
|
||
When I do the same with HTML export and import it does:
Failed to import favicon data:[Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIFaviconService.replaceFaviconDataFromDataURL]" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: resource://gre/modules/BookmarkHTMLUtils.jsm :: insertFaviconForNode :: line 1000" data: no]
BookmarkHTMLUtils.jsm:1008
insertFaviconForNode
resource://gre/modules/BookmarkHTMLUtils.jsm:1008:7
insertFaviconsForTree
resource://gre/modules/BookmarkHTMLUtils.jsm:1036:3
insertFaviconsForTree
resource://gre/modules/BookmarkHTMLUtils.jsm:1040:7
insertFaviconsForTree
resource://gre/modules/BookmarkHTMLUtils.jsm:1040:7
insertFaviconsForTree
resource://gre/modules/BookmarkHTMLUtils.jsm:1040:7
insertFaviconsForTree
resource://gre/modules/BookmarkHTMLUtils.jsm:1040:7
insertFaviconsForTree
resource://gre/modules/BookmarkHTMLUtils.jsm:1040:7
_importBookmarks
resource://gre/modules/BookmarkHTMLUtils.jsm:777:7
InterpretGeneratorResume self-hosted:1257:8 next self-hosted:1212:9
Alternate Service Mapping found: http://www.acustica-audio.com:-1 to http://www.acustica-audio.com:443
Alternate Service Mapping found: http://www.acustica-audio.com:-1 to http://www.acustica-audio.com:443
|
|
Reporter | |
Comment 13•5 years ago
|
||
But it seems to have to do with this crazy plain stored icon color values which isn't stored by json i guess.
|
|
Assignee | |
Comment 14•5 years ago
|
||
(In reply to Johannes Menzel from comment #11)
There's no message showing after importing
Just to clarify, does it still say that it fails to import?
Would you be interested in trying a special build that has more debugging output?
|
|
Reporter | |
Comment 15•5 years ago
|
||
No, theres no related message at all, neither that it fails import.
For the second question: yes. Is it for testing, or can it be used for productive use? Is there a repo? I still could debug with adb I guess, but I'd needed some advice.
|
|
Reporter | |
Comment 16•5 years ago
|
||
Just for the sake of completeness: This is the entire output on start of the Browser console:
Could not read chrome manifest 'file:///usr/lib/firefox/chrome.manifest'.
Unkown category for SetEventRecordingEnabled.
Content Security Policy: 'x-frame-options' wird wegen 'frame-ancestors'-Direktive ignoriert.
Content Security Policy: "'unsafe-inline'" wird innerhalb von script-src oder style-src ignoriert: nonce-source oder hash-source angegeben
Firefox Monitor: no strings available for default locale. Falling back to en-US. FirefoxMonitor.jsm:158
loadStrings
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:158:7
throw self-hosted:1229:9
Firefox Monitor: malformed breach entry.
Site:
{"Name":"ElasticsearchSalesLeads","Domain":"","schema":1547745032683,"PwnCount":5788169,"AddedDate":"2018-11-17T09:04:54Z","BreachDate":"2018-10-29","id":"30af4ecc-4d08-4d51-8d47-3213e4abec18","last_modified":1547745048641,"_status":"synced"} FirefoxMonitor.jsm:180
Firefox Monitor: malformed breach entry.
Site:
{"Name":"CrimeAgencyVBulletin","Domain":"","schema":1547745057658,"PwnCount":942044,"AddedDate":"2017-03-21T03:12:40Z","BreachDate":"2017-01-19","id":"4eb74f39-32ce-4f9f-b69b-3a4aac1a6751","last_modified":1547745063355,"_status":"synced"} FirefoxMonitor.jsm:180
loadBreaches/populateSites/<
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:180:11
forEach self-hosted:262:13 populateSites
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:178:7
loadBreaches
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:204:7
InterpretGeneratorResume self-hosted:1257:8 next self-hosted:1212:9
Firefox Monitor: malformed breach entry.
Site:
{"Name":"MasterDeeds","Domain":"","schema":1547744994293,"PwnCount":2257930,"AddedDate":"2017-10-18T11:01:46Z","BreachDate":"2017-03-14","id":"6e7daef1-57f8-4171-a37e-6a705ac2f72c","last_modified":1547745000600,"_status":"synced"} FirefoxMonitor.jsm:180
loadBreaches/populateSites/<
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:180:11
forEach self-hosted:262:13 populateSites
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:178:7
loadBreaches
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:204:7
InterpretGeneratorResume self-hosted:1257:8 next self-hosted:1212:9
Firefox Monitor: malformed breach entry.
Site:
{"Name":"YouveBeenScraped","Domain":"","schema":1547745092571,"PwnCount":66147869,"AddedDate":"2018-12-06T19:11:27Z","BreachDate":"2018-10-05","id":"846aa4bf-98e9-4ac7-96c1-a6a3b62db690","last_modified":1547745316104,"_status":"synced"} FirefoxMonitor.jsm:180
loadBreaches/populateSites/<
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:180:11
forEach self-hosted:262:13 populateSites
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:178:7
loadBreaches
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:204:7
InterpretGeneratorResume self-hosted:1257:8 next self-hosted:1212:9
Firefox Monitor: malformed breach entry.
Site:
{"Name":"Estonia","Domain":"","schema":1547745025686,"PwnCount":655161,"AddedDate":"2018-06-11T09:41:17Z","BreachDate":"2018-06-07","id":"874f9748-d504-49c9-9510-c0d995fe8abe","last_modified":1547745032674,"_status":"synced"} FirefoxMonitor.jsm:180
loadBreaches/populateSites/<
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:180:11
forEach self-hosted:262:13 populateSites
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:178:7
loadBreaches
moz-extension://9dd14132-d046-445b-96c0-3845e913c4d8/privileged/FirefoxMonitor.jsm:204:7
InterpretGeneratorResume self-hosted:1257:8 next self-hosted:1212:9
Successfully compiled asm.js code (loaded from cache in 290ms) background.bundle.js
Successfully compiled asm.js code (loaded from cache in 37ms) background.bundle.js
Successfully compiled asm.js code (loaded from cache in 132ms) background.bundle.js
No such native application gpgmejson NativeMessaging.jsm:99
NativeApp</this.startupPromise<
resource://gre/modules/NativeMessaging.jsm:99:9
Successfully compiled asm.js code (loaded from cache in 3ms) openpgp.js
Successfully compiled asm.js code (loaded from cache in 17ms) openpgp.js
Successfully compiled asm.js code (loaded from cache in 23ms) openpgp.js
TypeError: Cc['@mozilla.org/updates/update-checker;1'] is undefined[Weitere Informationen] ASRouterTargeting.jsm:66:9
[Exception... "Favicon at "https://assets.xlnaudio.com/favicon.ico" failed to load: Forbidden." nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: resource:///modules/FaviconLoader.jsm :: onStopRequest :: line 156" data: no]
onStopRequest
resource:///modules/FaviconLoader.jsm:156:31
next self-hosted:1212:9
Log warning: The log 'Services.Common.RESTRequest' is configured to use the preference 'services.common.log.logger.rest.request' - you must adjust the level by setting this preference, not by using the level setter Log.jsm:22
dumpError
resource://gre/modules/Log.jsm:22:3
set level
resource://gre/modules/Log.jsm:243:7
Intl
resource://services-common/hawkrequest.js:150:31
getIntl
resource://services-common/hawkrequest.js:189:12
HawkAuthenticatedRESTRequest
resource://services-common/hawkrequest.js:64:16
newHAWKAuthenticatedRESTRequest
resource://services-common/hawkclient.js:310:12
request
resource://services-common/hawkclient.js:211:19
InterpretGeneratorResume self-hosted:1257:8 next self-hosted:1212:9 _request
resource://gre/modules/FxAccountsClient.jsm:599:24
InterpretGeneratorResume self-hosted:1257:8 next self-hosted:1212:9 signCertificate/<
resource://gre/modules/FxAccountsClient.jsm:317:18
|
|
Reporter | |
Comment 17•5 years ago
|
||
I actually can't imagine that its an error in the import method source, there must rather be an error inside the structure of the stored bookmrks (sql?) and then rather a problem with exporting/syncing. Don't give to much on my newbee words, but importing bookmrks leaves out exactly the same specific bookmarks on different platforms.
|
|
Assignee | |
Comment 18•5 years ago
|
||
Hi Johannes, I just want to check my understanding of the problem is right:
- You have bookmarks which are not displayed in the Firefox user interface.
- Some of these are folders.
- Saving as backup and restoring does not help, even though the backups show the missing bookmarks.
- There's no errors on restore.
- These bookmarks are not displayed on any of the Firefoxes that you're syncing across(?)
Please can you confirm if they are right or not.
Here's some further ideas to try and track this down:
- Go to Help -> Troubleshooting Information, Scroll down to "Places Database", select "Verify Integrity".
- Please copy and paste the output here.
- Then restart Firefox and see if the issue is still present.
- Maybe attach a screenshot of where the bookmarks are missing in the UI.
|
|
Reporter | |
Comment 19•5 years ago
|
||
Just a little answer from abroad, because I'm currently moving to another flat. Can confirm everything, although I'm not sure if any (upper level) bookmarks themselves are missing, but definitely folders (and it's containing bookmarks).
I can deliver a screenshot later, I already sent you a list of specific missing/available folders of one top level folder.
|
|
Reporter | |
Comment 20•5 years ago
|
||
This is the output of the integrity check (which I already made before and didn't saw any suspicious entries):
> Task: checkIntegrity
+ The places.sqlite database is sane
+ The favicons.sqlite database is sane
> Task: invalidateCaches
+ The caches have been invalidated
> Task: checkCoherence
+ The database is coherent
> Task: expire
+ Database cleaned up
> Task: originFrecencyStats
+ Recalculated origin frecency stats
> Task: vacuum
+ Initial database size is 25600KiB
+ The database has been vacuumed
+ Final database size is 25600KiB
> Task: stats
+ Places.sqlite size is 25600KiB
+ Favicons.sqlite size is 4992KiB
+ pragma_user_version is 52
+ pragma_page_size is 32768
+ pragma_cache_size is -2048
+ pragma_journal_mode is wal
+ pragma_synchronous is 1
+ History can store a maximum of 155115 unique pages
+ Table moz_places has 47788 records
+ Table moz_historyvisits has 78746 records
+ Table moz_inputhistory has 73 records
+ Table moz_hosts has 0 records
+ Table moz_bookmarks has 2028 records
+ Table moz_bookmarks_deleted has 0 records
+ Table moz_keywords has 1 records
+ Table sqlite_sequence has 1 records
+ Table moz_anno_attributes has 6 records
+ Table moz_annos has 59 records
+ Table moz_items_annos has 20 records
+ Table sqlite_stat1 has 20 records
+ Table moz_meta has 8 records
+ Table moz_origins has 4803 records
+ Index sqlite_autoindex_moz_inputhistory_1
+ Index sqlite_autoindex_moz_hosts_1
+ Index sqlite_autoindex_moz_bookmarks_deleted_1
+ Index sqlite_autoindex_moz_keywords_1
+ Index sqlite_autoindex_moz_anno_attributes_1
+ Index sqlite_autoindex_moz_origins_1
+ Index moz_places_url_hashindex
+ Index moz_places_hostindex
+ Index moz_places_visitcount
+ Index moz_places_frecencyindex
+ Index moz_places_lastvisitdateindex
+ Index moz_places_guid_uniqueindex
+ Index moz_historyvisits_placedateindex
+ Index moz_historyvisits_fromindex
+ Index moz_historyvisits_dateindex
+ Index moz_bookmarks_itemindex
+ Index moz_bookmarks_parentindex
+ Index moz_bookmarks_itemlastmodifiedindex
+ Index moz_bookmarks_guid_uniqueindex
+ Index moz_keywords_placepostdata_uniqueindex
+ Index moz_annos_placeattributeindex
+ Index moz_items_annos_itemattributeindex
+ Index moz_bookmarks_dateaddedindex
+ Index moz_places_originidindex
> Task: _refreshUI
See the screenshot. The folder Development has duplicate entries ("JUCE", "C++", ..), but there should be folders like "DSP", "Filters" which are missing (See post above with the corresponding GUIDs)
|
|
Assignee | |
Comment 21•5 years ago
|
||
Hi Johannes, I'm sorry this got dropped for a while.
I've just reviewed it again, and there's not enough information for me to understand what's happening. The only really way that I can see that I might be able to work it out is if I could have a copy of your places.sqlite. I would only use it for debugging what's happening, and I would delete it afterwards. You can send it securely through https://send.firefox.com/ - which encrypts it before upload - and send me directly a one-time download link (which removes the file from the server).
If you don't want to do that, I understand.
Maybe the only other way is a json file that doesn't invalidate the urls (just changes the domain/paths maybe, so that the fundamental parts of the url are the same, just the alphanumerics replaced), and keeps any non-ascii characters in the titles.
|
|
Assignee | |
Comment 22•5 years ago
|
||
Also, maybe just check in safe mode, in case this is a weird add-on issue.
|
|
Assignee | |
Comment 23•5 years ago
|
||
Per my previous comments, I don't think there's anything more I can do here without more data. Bug 1523665 may have helped a bit, but it is hard to say.
Sorry for not being able to continue this earlier. If there's more information you're willing to provide I could take another look.
Description
•