Open Bug 974080 Opened 8 years ago Updated 4 years ago

Sync always messes up bookmarks with huge logs

Categories

(Firefox :: Sync, defect, P3)

defect

Tracking

()

People

(Reporter: webmaster, Unassigned)

References

Details

(Whiteboard: [sync:bookmarks][sync:rigor][sync:scale][qa+])

Attachments

(7 files)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0 (Beta/Release)
Build ID: 20140214013806

Steps to reproduce:

Mozilla/5.0 (X11; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0
Over 8000 bookmarks that I couldn't Sync since 2011 but recently cleaned up and organized, hoping to sync them.

Profile A is the 'bookmark master' while B is the one I wanna sync. 
I sync A, then connect profile B to Sync, 
- selecting {merge,replace-with-Sync-data}
- starting with {none,not-the-same,exact-copy-of-A} bookmarks
- resetting the key
- sync an {existing,brand-new-empty} profile.


Actual results:

Big soup. 'Unsorted folder' gets full of up to thousands of folders and bookmarks moved from their initial location and stay like this, I miss all to dozen of tags, 'Last modified' is revamped, many duplicates. That is, after 30 minutes to N days keeping the profiles open.
« Sync encountered an error while syncing: Unknown error... auto retry » message appears on some (not all) profiles.

Sync error logs weight from 3 KB to 5284 KB from one profile to another that were syncrhonized at the same time. Will add the three last logs. 
Also have some good screenshots, eg. before/after Sync :-O


Expected results:

Well the bookmarks could be synchronized between the profiles at some point, wouldn't they?

- History: Tried to use it since Weaves first releases, from time to time. If my memory is correct it kinda worked arround 2010-2011 then made a soup of my bookmarks' organization, had me stop using them (I didn't have the many hours it takes to reorganize them)
- I recently processed to reorganize my bookmarks: *cleaned* with Bookmarks Checker, *Sqlite checked* with Places Maintenance after any noticeable change
- Have read various bug reports for last three weeks, tried many of the appliable advices in the hope one of them would make Sync works here. No stable success so far, lots of time spent on this.
Hi,
Very similar:
Bug 812348 - Firefox Sync duplicating and scrambling bookmarks & folders 
Bug 813479 - Firefox Sync scrambling bookmarks and folders 
Older:
Bug 617437 - Sync randomly moved dozens of bookmarks out of folders into the Unsorted Bookmarks folder 
[solved] Bug 631361 - Sync failing with: bookmarks failed: NS_ERROR_ILLEGAL_VALUE JS 

Sync-log shows hundreds:
---
Sync.Store.Bookmarks	WARN	Failed to apply incoming record xxxxx
Sync.Store.Bookmarks	WARN	Encountered exception: Component returned failure code: 0x80070057 \
 (NS_ERROR_ILLEGAL_VALUE) [nsINavBookmarksService.insertBookmark] \
 Stack trace: resource://gre/components/nsTaggingService.js:157 
---


Places Maintenance stats:
1. Before sync
----------
> Statistics
Database size is 10240 KiB
user_version is 23
page_size is 32768
cache_size is -2048
journal_mode is wal
synchronous is 1
History can store a maximum of 44978 unique pages
Table moz_places has 8037 records
Table moz_historyvisits has 1 records
Table moz_inputhistory has 0 records
Table moz_hosts has 5835 records
Table moz_bookmarks has 15337 records
Table moz_bookmarks_roots has 5 records
Table moz_keywords has 267 records
Table sqlite_sequence has 1 records
Table moz_favicons has 5 records
Table moz_anno_attributes has 17 records
Table moz_annos has 2950 records
Table moz_items_annos has 8408 records
Table sqlite_stat1 has 4 records
Index moz_places_url_uniqueindex
Index moz_places_faviconindex
Index moz_places_hostindex
Index moz_places_visitcount
Index moz_places_frecencyindex
Index moz_places_lastvisitdateindex
Index moz_places_guid_uniqueindex
Index moz_historyvisits_placedateindex
Index moz_historyvisits_fromindex
Index moz_historyvisits_dateindex
Index sqlite_autoindex_moz_inputhistory_1
Index sqlite_autoindex_moz_hosts_1
Index moz_bookmarks_itemindex
Index moz_bookmarks_parentindex
Index moz_bookmarks_itemlastmodifiedindex
Index moz_bookmarks_guid_uniqueindex
Index sqlite_autoindex_moz_bookmarks_roots_1
Index sqlite_autoindex_moz_keywords_1
Index sqlite_autoindex_moz_favicons_1
Index sqlite_autoindex_moz_anno_attributes_1
Index moz_annos_placeattributeindex
Index moz_items_annos_itemattributeindex

2. After messy sync ('Master Bookmarks' profile)
----------
> Statistics
Database size is 10240 KiB
user_version is 23
page_size is 32768
cache_size is -2048
journal_mode is wal
synchronous is 1
History can store a maximum of 9388 unique pages
Table moz_places has 8764 records
Table moz_historyvisits has 1654 records
Table moz_inputhistory has 11 records
Table moz_hosts has 5976 records
Table moz_bookmarks has 16774 records
Table moz_bookmarks_roots has 5 records
Table moz_keywords has 267 records
Table sqlite_sequence has 1 records
Table moz_favicons has 253 records
Table moz_anno_attributes has 22 records
Table moz_annos has 2975 records
Table moz_items_annos has 9849 records
Table sqlite_stat1 has 15 records
Index sqlite_autoindex_moz_inputhistory_1
Index sqlite_autoindex_moz_hosts_1
Index sqlite_autoindex_moz_bookmarks_roots_1
Index sqlite_autoindex_moz_keywords_1
Index sqlite_autoindex_moz_favicons_1
Index sqlite_autoindex_moz_anno_attributes_1
Index moz_places_faviconindex
Index moz_places_hostindex
Index moz_places_visitcount
Index moz_places_frecencyindex
Index moz_places_lastvisitdateindex
Index moz_historyvisits_placedateindex
Index moz_historyvisits_fromindex
Index moz_historyvisits_dateindex
Index moz_bookmarks_itemindex
Index moz_bookmarks_parentindex
Index moz_bookmarks_itemlastmodifiedindex
Index moz_places_url_uniqueindex
Index moz_places_guid_uniqueindex
Index moz_bookmarks_guid_uniqueindex
Index moz_annos_placeattributeindex
Index moz_items_annos_itemattributeindex


2'. After messy sync (Main profile 1)
------------
> Statistics
Database size is 10240 KiB
user_version is 23
page_size is 32768
cache_size is -2048
journal_mode is wal
synchronous is 1
History can store a maximum of 9159 unique pages
Table moz_places has 6022 records
Table moz_historyvisits has 1386 records
Table moz_inputhistory has 13 records
Table moz_hosts has 3910 records
Table moz_bookmarks has 11941 records
Table moz_bookmarks_roots has 5 records
Table moz_keywords has 127 records
Table sqlite_sequence has 1 records
Table moz_favicons has 167 records
Table moz_anno_attributes has 15 records
Table moz_annos has 83 records
Table moz_items_annos has 3437 records
Table sqlite_stat1 has 15 records
Index sqlite_autoindex_moz_inputhistory_1
Index sqlite_autoindex_moz_hosts_1
Index sqlite_autoindex_moz_bookmarks_roots_1
Index sqlite_autoindex_moz_keywords_1
Index sqlite_autoindex_moz_favicons_1
Index sqlite_autoindex_moz_anno_attributes_1
Index moz_places_faviconindex
Index moz_places_hostindex
Index moz_places_visitcount
Index moz_places_frecencyindex
Index moz_places_lastvisitdateindex
Index moz_historyvisits_placedateindex
Index moz_historyvisits_fromindex
Index moz_historyvisits_dateindex
Index moz_bookmarks_itemindex
Index moz_bookmarks_parentindex
Index moz_bookmarks_itemlastmodifiedindex
Index moz_places_url_uniqueindex
Index moz_places_guid_uniqueindex
Index moz_bookmarks_guid_uniqueindex
Index moz_annos_placeattributeindex
Index moz_items_annos_itemattributeindex

2''. After messy sync (Main profile 2)
------------
> Statistics
Database size is 10240 KiB
user_version is 23
page_size is 32768
cache_size is -2048
journal_mode is wal
synchronous is 1
History can store a maximum of 9211 unique pages
Table moz_places has 4510 records
Table moz_historyvisits has 12 records
Table moz_inputhistory has 0 records
Table moz_hosts has 3512 records
Table moz_bookmarks has 7590 records
Table moz_bookmarks_roots has 5 records
Table moz_keywords has 87 records
Table sqlite_sequence has 1 records
Table moz_favicons has 2 records
Table moz_anno_attributes has 14 records
Table moz_annos has 6 records
Table moz_items_annos has 3532 records
Table sqlite_stat1 has 14 records
Index sqlite_autoindex_moz_inputhistory_1
Index sqlite_autoindex_moz_hosts_1
Index sqlite_autoindex_moz_bookmarks_roots_1
Index sqlite_autoindex_moz_keywords_1
Index sqlite_autoindex_moz_favicons_1
Index sqlite_autoindex_moz_anno_attributes_1
Index moz_places_faviconindex
Index moz_places_hostindex
Index moz_places_visitcount
Index moz_places_frecencyindex
Index moz_places_lastvisitdateindex
Index moz_historyvisits_placedateindex
Index moz_historyvisits_fromindex
Index moz_historyvisits_dateindex
Index moz_bookmarks_itemindex
Index moz_bookmarks_parentindex
Index moz_bookmarks_itemlastmodifiedindex
Index moz_places_url_uniqueindex
Index moz_places_guid_uniqueindex
Index moz_bookmarks_guid_uniqueindex
Index moz_annos_placeattributeindex
Index moz_items_annos_itemattributeindex
Attachment #8377778 - Attachment description: sync-error - existing profile - library was empty → sync-error - existing profile - library was empty before sync
Places Maintenance Stats again but after 40 hours:

* 'Master Bookmarks' profile:
> Statistics
Database size is 10240 KiB
user_version is 23
page_size is 32768
cache_size is -2048
journal_mode is wal
synchronous is 1
History can store a maximum of 9388 unique pages
Table moz_places has 8808 records
Table moz_historyvisits has 1690 records
Table moz_inputhistory has 11 records
Table moz_hosts has 5990 records
Table moz_bookmarks has 12755 records
Table moz_bookmarks_roots has 5 records
Table moz_keywords has 262 records
<snip>

* Main profile 1
Now has: 50 tags and 9073 items of which two subfolders in the Unsorted Bookmarks' root.
> Statistics
Database size is 10240 KiB
user_version is 23
page_size is 32768
cache_size is -2048
journal_mode is wal
synchronous is 1
History can store a maximum of 9159 unique pages
Table moz_places has 9147 records
Table moz_historyvisits has 1391 records
Table moz_inputhistory has 13 records
Table moz_hosts has 5967 records
Table moz_bookmarks has 18560 records
Table moz_bookmarks_roots has 5 records
Table moz_keywords has 260 records
<snip>

* Main profile 2
Now has: 38 tags and 1125 items of which two subfolders in the Unsorted Bookmarks' root.
> Statistics
Database size is 10240 KiB
user_version is 23
page_size is 32768
cache_size is -2048
journal_mode is wal
synchronous is 1
History can store a maximum of 9211 unique pages
Table moz_places has 8030 records
Table moz_historyvisits has 13 records
Table moz_inputhistory has 0 records
Table moz_hosts has 5834 records
Table moz_bookmarks has 10427 records
Table moz_bookmarks_roots has 5 records
Table moz_keywords has 262 records
<snip>

Note that I Also have stats (and logs) for the brand new profile, in which I imported the _same_ bookmarks library than in 'Bookmarks Master' before starting to sync.

Hope this bug report has some interest. Please let me know how to improve it.
Attached log after Sync kinda worked on two profiles + source:
- with half my bookmarks library, i.e. 4997 bookmarks
- all bookmarks synchronized
- tags were synchronized for one of the two profiles
- all dates reseted for bookmarks, folders and tags
8,000 bookmarks is well outside the feasible scope of what I'd trust to Sync.

We hard-limit to 5,000 on Android.

That said, I see two problems here.

One is that you have some malformed bookmarks. For example:

---
1392752069261	Sync.Store.Bookmarks	DEBUG	Parent is undefined; reparenting to unfiled.
1392752069262	Sync.Store.Bookmarks	DEBUG	created bookmark 62185 under 5 as ASS2L http://www.ass2l.org/
1392752069263	Sync.Store.Bookmarks	WARN	Failed to apply incoming record lPCoqUfLmaCR
1392752069263	Sync.Store.Bookmarks	WARN	Encountered exception: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsINavBookmarksService.insertBookmark] Stack trace: resource://gre/components/nsTaggingService.js:157 < resource://gre/components/nsTaggingService.js:168 < TS_tagURI()@resource://gre/components/nsTaggingService.js:170 < _tagURI()@resource://gre/modules/services-sync/engines/bookmarks.js:1264 < BStore_create()@resource://gre/modules/services-sync/engines/bookmarks.js:679 < resource://gre/modules/services-sync/engines.js:329 < BStore_applyIncoming()@resource://gre/modules/services-sync/engines/bookmarks.js:555 < resource://gre/modules/services-sync/engines.js:297 < doApplyBatch()@resource://gre/modules/services-sync/engines.js:889 < resource://gre/modules/services-sync/engines.js:1000 < resource://services-sync/record.js:625 < Channel_onDataAvail()@resource://services-sync/resource.js:542 < <file:unknown>
1392752069266	Sync.Store.Bookmarks	DEBUG	created bookmark 62186 under 5 as DRM « Polaris http://polaris.wordpress.com/tag/drm/
1392752069267	Sync.Store.Bookmarks	WARN	Failed to apply incoming record lPONpDa_81Y_
1392752069267	Sync.Store.Bookmarks	WARN	Encountered exception: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsINavBookmarksService.insertBookmark] Stack trace: resource://gre/components/nsTaggingService.js:157 < resource://gre/components/nsTaggingService.js:168 < TS_tagURI()@resource://gre/components/nsTaggingService.js:170 < _tagURI()@resource://gre/modules/services-sync/engines/bookmarks.js:1264 < BStore_create()@resource://gre/modules/services-sync/engines/bookmarks.js:679 < resource://gre/modules/services-sync/engines.js:329 < BStore_applyIncoming()@resource://gre/modules/services-sync/engines/bookmarks.js:555 < resource://gre/modules/services-sync/engines.js:297 < doApplyBatch()@resource://gre/modules/services-sync/engines.js:889 < resource://gre/modules/services-sync/engines.js:1000 < resource://services-sync/record.js:625 < Channel_onDataAvail()@resource://services-sync/resource.js:542 < <file:unknown>
---

I see dozens or hundreds of these in your log, which will result in many of your bookmarks ending up in the wrong place.

1392752080405	Sync.Engine.Bookmarks	INFO	Records: 8744 applied, 8312 successfully, 432 failed to apply, 432 newly failed to apply, 0 reconciled.



This snippet in your first log:

---
1392752009838	Sync.Engine.Bookmarks	DEBUG	ExMMEOXjuSpm mapped to undefined
1392752009838	Sync.Store.Bookmarks	DEBUG	Applying record ExMMEOXjuSpm
1392752009838	Sync.Store.Bookmarks	DEBUG	Local parent is 8iO76vimzhgR
1392752009838	Sync.Store.Bookmarks	DEBUG	Record ExMMEOXjuSpm is not an orphan.
1392752009839	Sync.Store.Bookmarks	DEBUG	created bookmark 67629 under 65513 as Win XP: Dossier n'est pas accessible. accès refusé. http://www.commentcamarche.net/forum/affich-1452358-dossier-n-est-pas-accessible-acces-refuse
1392752011420	Sync.Status	INFO	Resetting Status.
1392752011420	Sync.Status	DEBUG	Status.service: success.status_ok => success.status_ok
1392752011420	Sync.Identity	INFO	Sync Key has no value. Deleting.
1392752011420	Sync.Status	DEBUG	Status.login: success.login => error.login.reason.no_recoverykey
1392752011420	Sync.Status	DEBUG	Status.service: success.status_ok => service.client_not_configured
1392752011420	Sync.Service	INFO	Logging out
1392752011421	Sync.Status	DEBUG	Status.login: error.login.reason.no_recoverykey => error.login.reason.no_recoverykey
1392752011421	Sync.Status	DEBUG	Status.service: service.client_not_configured => service.client_not_configured
1392752011421	Sync.SyncScheduler	DEBUG	Next sync in 90000 ms.
1392752011421	Sync.Status	DEBUG	Status.login: error.login.reason.no_recoverykey => error.login.reason.no_recoverykey
1392752011421	Sync.Status	DEBUG	Status.service: service.client_not_configured => service.client_not_configured
1392752011422	Sync.SyncScheduler	DEBUG	Clearing sync triggers and the global score.
1392752017264	Sync.Resource	DEBUG	mesg: DELETE success 200 https://phx-sync-15-1-9.services.mozilla.com/1.1/g5lvwotcojhmjjwhw42vdszt4cmntahg/storage/clients/eL3hk4hOjbOF
1392752017264	Sync.Resource	DEBUG	DELETE success 200 https://phx-sync-15-1-9.services.mozilla.com/1.1/g5lvwotcojhmjjwhw42vdszt4cmntahg/storage/clients/eL3hk4hOjbOF
1392752018114	Sync.Resource	DEBUG	mesg: DELETE success 200 https://phx-sync-15-1-9.services.mozilla.com/1.1/g5lvwotcojhmjjwhw42vdszt4cmntahg/storage/tabs/eL3hk4hOjbOF
1392752018114	Sync.Resource	DEBUG	DELETE success 200 https://phx-sync-15-1-9.services.mozilla.com/1.1/g5lvwotcojhmjjwhw42vdszt4cmntahg/storage/tabs/eL3hk4hOjbOF
1392752018114	Sync.Service	INFO	Service reset.
---

indicates that you disabled Sync half-way through the first download. That resulted in failures at the end of the sync:

1392752087918	Sync.Engine.Bookmarks	WARN	Error creating record: A key bundle must be supplied to encrypt. JS Stack trace: encrypt@record.js:200 < @engines.js:1354 < @engines.js:1421 < wrappedSync@bookmarks.js:214 < _sync@bookmarks.js:220 < WrappedNotify@util.js:142 < @engines.js:594 < _syncEngine@enginesync.js:192 < sync@enginesync.js:147 < onNotify@service.js:1197 < WrappedNotify@util.js:142 < WrappedLock@util.js:97 < _lockedSync@service.js:1191 < @service.js:1182 < WrappedCatch@util.js:71 < sync@service.js:1170
1392752087918	Sync.Engine.Bookmarks	WARN	Error creating record: A key bundle must be supplied to encrypt. JS Stack trace: encrypt@record.js:200 < @engines.js:1354 < @engines.js:1421 < wrappedSync@bookmarks.js:214 < _sync@bookmarks.js:220 < WrappedNotify@util.js:142 < @engines.js:594 < _syncEngine@enginesync.js:192 < sync@enginesync.js:147 < onNotify@service.js:1197 < WrappedNotify@util.js:142 < WrappedLock@util.js:97 < _lockedSync@service.js:1191 < @service.js:1182 < WrappedCatch@util.js:71 < sync@service.js:1170
1392752087919	Sync.Engine.Bookmarks	INFO	Uploading all of 2 records
1392752087976	Sync.Status	DEBUG	Status for engine bookmarks: error.engine.reason.unknown_fail
1392752087976	Sync.Status	DEBUG	Status.service: error.sync.failed_partial => error.sync.failed_partial
1392752087976	Sync.ErrorHandler	DEBUG	bookmarks failed: Component returned failure code: 0x804b000a (NS_ERROR_MALFORMED_URI) [nsIIOService2.newChannel] JS Stack trace: Res_post@resource.js:427 < @engines.js:1322 < innerBind@util.js:554 < @engines.js:1370 < @engines.js:1421 < wrappedSync@bookmarks.js:214 < _sync@bookmarks.js:220 < WrappedNotify@util.js:142 < @engines.js:594 < _syncEngine@enginesync.js:192 < sync@enginesync.js:147 < onNotify@service.js:1197 < WrappedNotify@util.js:142 < WrappedLock@util.js:97 < _lockedSync@service.js:1191 < @service.js:1182 < WrappedCatch@util.js:71 < sync@service.js:1170
1392752087976	Sync.Synchronizer	DEBUG	Aborting sync, no cluster URL: not uploading new meta/global.
So: if you're really interested in getting this amount of data to sync, the first step is to figure out why the tagging service is failing.

Marco?
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(mak77)
Whiteboard: [sync:bookmarks][sync:rigor][sync:scale]
@Richard

> 8,000 bookmarks is well outside the feasible scope of what I'd trust to Sync.
> We hard-limit to 5,000 on Android.

So it's the occasion to further clean this library, allright. Please do you know where is this "barrier" information available? 
As for Android I'd care when Nightly stops martyrizing the poor Galaxy ACE Qualcomm single core (or I change mobile ;)


On the two problems here:

1. Any way to locate/know what are the "malformed bookmarks" like these please?
> Encountered exception: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE)

2. I disabled Sync half-way through the first download which resulted in failures at the end of the sync:
Sorry i had to delete some "sensitive" items from the library then restart and post quickly. Have got 3 weeks of _uninterrupted_ (I swear) logs before that one if needed. I waited that time, searching bugzilla and making all kind of tests, in the hope to find out same issue, and its solution.

Problem for me is my total lack of knowledge about the method to debug here. e.g. start with nn bookmarks? check for non-ASCII or non-Latin strings? tags? date? and so on. Maybe I should go for Delicious or Pinterest (what's $27 compared to tens of hours if it works?). But I feel Mozilla Firefox deserves time from us, users.
either uri is null or the tag folder doesn't exist.. it's strange, to add a tag we first check if we should create the tag folder, create it, then create a bookmark into it (yes this is crazy). We create everything we need so this should not fail.
I wonder if the tags root is corrupt, and causing all of the other errors, it may have bogus children, it's hard to tell... are you willing to share your places.sqlite with me through mail?
I guess restoring bookmarks from scratch from a json profile may help (delete places.sqlite and a new one will be rebuilt, history gets lost though) to build a clean bookmarks database.
Flags: needinfo?(mak77)
Hey Marco
> either uri is null or the tag folder doesn't exist..
wooo! means unusable/unvisible in the library UI? please correct me if I'm wrong.

Have deleted places.sqlite no more than a couple of time, then restored a previous backup, then tried Sync again.
Runned Places Maintenance dozen of times though.
Remember Sync's been broken for me ever since 2011 which was a pain in the heeer organization (travelling).

Yes no problem; sending you an email with the "light" library that was used for this last test (5K bookmarks and 500 tags).
_PS if someone knows how to edit bugzilla profile when logged through Persona please let me know, so I can put a name in place of my email in clear. Asks me a password which I don't have since logged via Persona :-o_
the only particular thing I see in the json you sent me is that some tags have names containing particular characters, but I can't tell if there's an issue with that. Restoring this json into a new profile doesn't seem to show anything broken. Off-hand I don't think there's a relation.

Did you already try to restore this backup into a new profile, then reset Sync remote bookmarks to the contents of that clean profile?
Thanks a lot for getting your hands on it Marco.

> some tags have names containing particular characters
That's something you can be certain happens a lot out of the English world ;)
If 'particular characters' in tags might cause problem then Mozilla c'mon, say it and display the information somewhere! (same for the bookmarks size or number of items' limit).

> Did you already try to restore this backup into a new profile, then reset Sync remote bookmarks to the contents of that clean profile?
Yes, results as per Comment 7.
Re-trying again towards a new profile; looks good after 5 minutes.
If it was to keep synchronizing succesfuly, I will follow Richard "common sense" advice and divide my library in two smaller ones, i.e. two Sync's profiles. 

But these two new informations let me think there's still a problem (that may explode on my noze anytime when I don't need it, as bugs frequently do):

1. By chance this week end I put a finger on a rather precise issue (yeah) in my <5K shortened Bookmarks Library. It mixed at least one folder with an item:
I use Visibo's extension TooManyTabs, which as you may know uses a bookmark folder in Uncategorized Bookmarks*. And this folder's name had been changed from default 'TooManyTabs (Do Not Delete)' which _is_ present with full content, to 'OpenLDAP: serveur sous Debian (rapide), qlqs scripts et commandes utiles | otasc.org' (not by me) which I bookmarked a long time ago. Actually the bookmark _is_ present in the library with no other sign of Ubiquity illness ;)
Edited TooManyTabs folder's name in about:config and all went back to normal with this extension
* http://blog.visibotech.com/2010/01/toomanytabs-advanced-configuration.html

2. Then, Diigo won't import the library, though it's 1 MB under the maximum size they handle.
> Oops. Something wrong. Please check the file size and format. Don't hesitate to contact us (with your diigo user name) if you still experience problems. 
is what Diigo Import says.
Update on the new sync started after I restored this backup:

- New profile syncs fine; 

- Existing profile 1 does not sync a single bookmark (well it does but in the "wrong" direction i.e. from itself to Sync)

- Existing profile 2 sync... making a bechamel of my bookmarks

On the existing profile I a) deleted places.sqlite, b) deleted bookmarks, and c) added to Sync.
Will join the sync-log for all three (zipped as they're up to 4.6 MB)
Update: after a couple of hours. Number of bookmarks goes from 7792 to 8150 depending the profile. Here are their stats:

- New profile:
> Statistics
Database size is 10240 KiB
user_version is 23
page_size is 32768
cache_size is -2048
journal_mode is wal
synchronous is 1
History can store a maximum of 7504 unique pages
Table moz_places has 6932 records
Table moz_historyvisits has 1 records
Table moz_inputhistory has 0 records
Table moz_hosts has 5019 records
Table moz_bookmarks has 7792 records
Table moz_bookmarks_roots has 5 records
Table moz_keywords has 261 records
Table sqlite_sequence has 1 records
Table moz_favicons has 2 records
Table moz_anno_attributes has 10 records
Table moz_annos has 0 records
Table moz_items_annos has 540 records
Table sqlite_stat1 has 14 records
Index moz_places_url_uniqueindex
Index moz_places_faviconindex
Index moz_places_hostindex
Index moz_places_visitcount
Index moz_places_frecencyindex
Index moz_places_lastvisitdateindex
Index moz_places_guid_uniqueindex
Index moz_historyvisits_placedateindex
Index moz_historyvisits_fromindex
Index moz_historyvisits_dateindex
Index sqlite_autoindex_moz_inputhistory_1
Index sqlite_autoindex_moz_hosts_1
Index moz_bookmarks_itemindex
Index moz_bookmarks_parentindex
Index moz_bookmarks_itemlastmodifiedindex
Index moz_bookmarks_guid_uniqueindex
Index sqlite_autoindex_moz_bookmarks_roots_1
Index sqlite_autoindex_moz_keywords_1
Index sqlite_autoindex_moz_favicons_1
Index sqlite_autoindex_moz_anno_attributes_1
Index moz_annos_placeattributeindex
Index moz_items_annos_itemattributeindex

- Existing profile 1:
> Statistics
Database size is 10240 KiB
user_version is 23
page_size is 32768
cache_size is -2048
journal_mode is wal
synchronous is 1
History can store a maximum of 7759 unique pages
Table moz_places has 7372 records
Table moz_historyvisits has 97 records
Table moz_inputhistory has 0 records
Table moz_hosts has 5264 records
Table moz_bookmarks has 8150 records
Table moz_bookmarks_roots has 5 records
Table moz_keywords has 263 records
Table sqlite_sequence has 1 records
Table moz_favicons has 26 records
Table moz_anno_attributes has 19 records
Table moz_annos has 2767 records
Table moz_items_annos has 7529 records
Table sqlite_stat1 has 11 records
Index sqlite_autoindex_moz_inputhistory_1
Index sqlite_autoindex_moz_hosts_1
Index sqlite_autoindex_moz_bookmarks_roots_1
Index sqlite_autoindex_moz_keywords_1
Index sqlite_autoindex_moz_favicons_1
Index sqlite_autoindex_moz_anno_attributes_1
Index moz_places_faviconindex
Index moz_places_hostindex
Index moz_places_visitcount
Index moz_places_frecencyindex
Index moz_places_lastvisitdateindex
Index moz_historyvisits_placedateindex
Index moz_historyvisits_fromindex
Index moz_historyvisits_dateindex
Index moz_bookmarks_itemindex
Index moz_bookmarks_parentindex
Index moz_bookmarks_itemlastmodifiedindex
Index moz_places_url_uniqueindex
Index moz_places_guid_uniqueindex
Index moz_bookmarks_guid_uniqueindex
Index moz_annos_placeattributeindex
Index moz_items_annos_itemattributeindex

- Existing profile 2:
> Statistics
Database size is 10240 KiB
user_version is 23
page_size is 32768
cache_size is -2048
journal_mode is wal
synchronous is 1
History can store a maximum of 7687 unique pages
Table moz_places has 7190 records
Table moz_historyvisits has 5 records
Table moz_inputhistory has 0 records
Table moz_hosts has 5166 records
Table moz_bookmarks has 7989 records
Table moz_bookmarks_roots has 5 records
Table moz_keywords has 262 records
Table sqlite_sequence has 1 records
Table moz_favicons has 3 records
Table moz_anno_attributes has 12 records
Table moz_annos has 1700 records
Table moz_items_annos has 571 records
Table sqlite_stat1 has 14 records
Index moz_places_url_uniqueindex
Index moz_places_faviconindex
Index moz_places_hostindex
Index moz_places_visitcount
Index moz_places_frecencyindex
Index moz_places_lastvisitdateindex
Index moz_places_guid_uniqueindex
Index moz_historyvisits_placedateindex
Index moz_historyvisits_fromindex
Index moz_historyvisits_dateindex
Index sqlite_autoindex_moz_inputhistory_1
Index sqlite_autoindex_moz_hosts_1
Index moz_bookmarks_itemindex
Index moz_bookmarks_parentindex
Index moz_bookmarks_itemlastmodifiedindex
Index moz_bookmarks_guid_uniqueindex
Index sqlite_autoindex_moz_bookmarks_roots_1
Index sqlite_autoindex_moz_keywords_1
Index sqlite_autoindex_moz_favicons_1
Index sqlite_autoindex_moz_anno_attributes_1
Index moz_annos_placeattributeindex
Index moz_items_annos_itemattributeindex
Hi, 
Here what happened after one more day changing nothing but a few bookmarks:
a) 80 to 90% of my tags disapeared (varies between profiles)
b) Uncategorized bookmarks (of which TooManyTabs and my latests bookmarked items) totaly disapeared from at least one profile.
c) The New profile was exempt of these two issues yesterday, but it was quickly auto-synchronized following the others --not the opposite unfortunately.

Unless someone manages to find the issue, which I couldn't myself, I guess Bookmarks on Firefox are just _over_ for me :/
The only thing I could suggest is to create a completely new Sync account from a completely fresh Firefox profile, restore json to this new profile. Only attach new profiles to this new account, don't try to attach old ones or you may restart with this nightmare. I don't know why Sync is messing with bookmarks like that, so far I can exclude issues in the local storage, so there must be something broken in the remote management or in the process. At this point I think your remote profile is quite messed up.
Hi 
@Marco Yesterday I have gone the radical way that you kindly suggested. So far so good (amazingly good compared to the previous "sync" functions lol)
Implies rebuilding a few carefully personnalized profiles. So God bless Chuck Baker, FEBE - CLEO developper :))
Bookmarks were messed up after I connected a third profile.
I.e. a bit more than slightly messed up. 500 items moved to 'Uncategorized' and staying there; a few dozen tags removed / disapeared
All three profiles were _created_ for the occasion and as empty as is my bank account.

Not too surprised (see comment 15). But I'm done with waisting time on FF's Sync without usable result.
Logs are available if needed.
Priority: -- → P2
Whiteboard: [sync:bookmarks][sync:rigor][sync:scale] → [sync:bookmarks][sync:rigor][sync:scale][qa+]
Quick follow-up. 

Initial situation: 3 Firefox profiles, with same bookmarks and tags, that I manualy restored.
- Change 1 bookmark on a profile
- Open another profile. Sync _deletes_ then resyncs _all_ bookmarks and their tags, giving them all the same/present date and time.
- On the other profiles, bookmarks still have their correct "added time".

This is the best scenario I've encoutered (starting with same databases). Others scénarios all resulted in loosing bookmarks, tags, and folders.
Adding this to our larger list of bookmark dataloss issues.
Blocks: 621584
Priority: P2 → --
Priority: -- → P3
Component: Firefox Sync: Backend → Sync
Product: Cloud Services → Firefox
You need to log in before you can comment on or make changes to this bug.