731273 - Mobile Bookmarks are sync'd in a "mobile" directory instead of Desktop Mobile Bookmarks directory

Status: VERIFIED FIXED

(Firefox for Android Graveyard :: Android Sync, defect, P2)

Description

[Cristian Nicolae (:xti)]

When I Sync my device, some mobile pages are listed in Mobile Bookmarks category like about:home or AMO mobile version, CNN mobile version. However, those pages are not listed on Desktop side in Mobile Bookmarks directory, even if I tapped on Sync Now button. To be more exactly, the Mobile Bookmarks is empty on Desktop Side.
I am able to reproduce this on the latest Nightly build on a clean profile. Profile migration is not involved here.

--
Firefox 13.0a1 (2012-02-28)
20120228031102
http://hg.mozilla.org/mozilla-central/rev/7ce4d9b55863
Device: Samsung Galaxy S2
OS: Android 2.3.4

Comment 1

[Richard Newman [:rnewman]]

Several questions.

1. Do they autocomplete in the awesomebar on desktop? (That is, they're in your bookmarks *somewhere*...)

2. Have you actually had a sync occur on the mobile device? Take a look in Settings > Accounts and sync. You can force a sync from here, then on the desktop.

3. Can you provide logs from:

  Android: http://160.twinql.com/how-to-file-a-good-android-sync-bug
  Desktop: https://philikon.wordpress.com/2011/06/13/how-to-file-a-good-sync-bug/

Comment 2

[Allison Naaktgeboren :ally]

waiting on more information

Comment 3

[Cristian Nicolae (:xti)]

(In reply to Richard Newman [:rnewman] from comment #1)
> Several questions.
> 
> 1. Do they autocomplete in the awesomebar on desktop? (That is, they're in
> your bookmarks *somewhere*...)
> 
> 2. Have you actually had a sync occur on the mobile device? Take a look in
> Settings > Accounts and sync. You can force a sync from here, then on the
> desktop.
> 
> 3. Can you provide logs from:
> 
>   Android: http://160.twinql.com/how-to-file-a-good-android-sync-bug
>   Desktop:
> https://philikon.wordpress.com/2011/06/13/how-to-file-a-good-sync-bug/

1. Those bookmarks are listed in awesomebar and seems to be somewhere in a "mobile" directory. However I cannot find that category in Desktop Library.

2. I did that, but nothing was changed on Desktop side.

3. I will attach logs from Desktop and Mobile side.

Comment 4

[Cristian Nicolae (:xti)]

Attachment: desktop sync log

Comment 5

[Cristian Nicolae (:xti)]

Attachment: alogcat - Nightly

Comment 6

[Cristian Nicolae (:xti)]

Those webpages were bookmarked a long time ago on a XUL Nightly build, but I remember that they were listed in Mobile Bookmarks in Desktop Library. Let me know if I could provide more useful information about this issue.

Comment 7

[Allison Naaktgeboren :ally]

results of triage: We think this is a result of the old schema, which did not have GUIDs. If it is not too much of a hassle, would you please try resetting sync? (we have a theory)

Rnewman, what are your thoughts?

Comment 8

[Richard Newman [:rnewman]]

(In reply to :Ally Naaktgeboren from comment #7)
> results of triage: We think this is a result of the old schema, which did
> not have GUIDs. If it is not too much of a hassle, would you please try
> resetting sync? (we have a theory)
> 
> Rnewman, what are your thoughts?

Most likely some curious interaction between profile migration and database migration results in the bookmarks now being in the right folder on the device, but Sync doesn't know they've changed since it uploaded them to where they are on the desktop.

Either that or you've got two Mobile Bookmarks folders. There have been a lot of bugs stamped out of migration and bookmarks over the past few weeks.

There is no way to force Android Sync to do a fresh upload (Bug 730636). Resetting the desktop won't help. The only thing you can do is fix your bookmarks on the desktop and sync; if that doesn't work, uninstall Fennec and start over.

Comment 9

[Cristian Nicolae (:xti)]

I've done some more investigation about this issue.

www.apple.com was one of those bookmarks from the "mobile" list. I opened this page on Desktop side and I tried to remove bookmark and the button said "Remove 2 bookmarks". I noticed that it happens the same for any page that is bookmarked on Desktop Firefox.

On mobile side I remove the bookmark for that webpage and I re-bookmarked it back (hoping that it will be listed in the right place). Even if the toaster notification confirmed that the bookmark was added, the Star remains black. Also there were 2 entries for apple.com. Unfortunately, the bookmark was still in "mobile" directory.

I removed the sync account and I cleared the Nightly profile. I opened www.android.com and I bookmarked it and then I sync'd my device with Desktop side. I thought that this should be the right way to add a bookmark into Mobile Bookmarks category but when I opened www.android.com on Desktop Firefox, it was saved in "mobile" category and it cannot be found in Desktop Library. How could it be possible?

Comment 10

[Richard Newman [:rnewman]]

(In reply to Cristian Nicolae (:xti) from comment #9)

> I removed the sync account and I cleared the Nightly profile.

Those two operations are not enough. When I said "uninstall Fennec", I really meant it!

> when I opened www.android.com on Desktop
> Firefox, it was saved in "mobile" category and it cannot be found in Desktop
> Library. How could it be possible?

You have two mobile folders. Whichever one has the GUID "mobile" on the phone is the one that Fennec will use for storing mobile bookmarks. If that's not the one at the top level in the UI, you'll see what you're seeing.

If you don't mind sharing your folder hierarchy:

  * Go to your profile directory
  * sqlite3 places.sqlite
  * Enter this:

.width 6 14 6 32 20
select b.id, b.guid, b.parent, b.title, b.lastModified, a.content from moz_bookmarks b left outer join moz_items_annos a on a.item_id = b.fk where b.type = 2;

and paste the output here.

Thanks!

Comment 11

[Richard Newman [:rnewman]]

(In reply to Cristian Nicolae (:xti) from comment #3)

> 1. Those bookmarks are listed in awesomebar and seems to be somewhere in a
> "mobile" directory. However I cannot find that category in Desktop Library.

Did you have a look in Unsorted Bookmarks?

Comment 12

[Cristian Nicolae (:xti)]

(In reply to Richard Newman [:rnewman] from comment #11)
> (In reply to Cristian Nicolae (:xti) from comment #3)
> 
> > 1. Those bookmarks are listed in awesomebar and seems to be somewhere in a
> > "mobile" directory. However I cannot find that category in Desktop Library.
> 
> Did you have a look in Unsorted Bookmarks?

I did, but it's not there. Actually I searched everywhere in the Library and still I cannot find it.

Also I will follow your suggestion from comment #10 and I will update this bug with the new results

Comment 13

[Cristian Nicolae (:xti)]

(In reply to Richard Newman [:rnewman] from comment #10)
> (In reply to Cristian Nicolae (:xti) from comment #9)
> 
> > I removed the sync account and I cleared the Nightly profile.
> 
> Those two operations are not enough. When I said "uninstall Fennec", I
> really meant it!
> 
> > when I opened www.android.com on Desktop
> > Firefox, it was saved in "mobile" category and it cannot be found in Desktop
> > Library. How could it be possible?
> 
> You have two mobile folders. Whichever one has the GUID "mobile" on the
> phone is the one that Fennec will use for storing mobile bookmarks. If
> that's not the one at the top level in the UI, you'll see what you're seeing.
> 
> If you don't mind sharing your folder hierarchy:
> 
>   * Go to your profile directory
>   * sqlite3 places.sqlite
>   * Enter this:
> 
> .width 6 14 6 32 20
> select b.id, b.guid, b.parent, b.title, b.lastModified, a.content from
> moz_bookmarks b left outer join moz_items_annos a on a.item_id = b.fk where
> b.type = 2;
> 
> and paste the output here.
> 
> Thanks!

I hope that this will be helpful: http://pastebin.com/Fv9W839B

Comment 14

[Cristian Nicolae (:xti)]

I have no idea what happened, but my bookmarks hierarchy was changed somehow: http://pastebin.mozilla.org/1505126

I ran select id, title from moz_bookmarks where parent = 1 and here is it the output: http://pastebin.mozilla.org/1505127

Comment 15

[Richard Newman [:rnewman]]

(In reply to Cristian Nicolae (:xti) from comment #14)
> I have no idea what happened, but my bookmarks hierarchy was changed
> somehow: http://pastebin.mozilla.org/1505126
> 
> I ran select id, title from moz_bookmarks where parent = 1 and here is it
> the output: http://pastebin.mozilla.org/1505127

Are you able to go to Settings > Apps > Fennec and choose Move to SD Card?

If so, I'd be interested in grabbing your browser DB.

Comment 16

[Richard Newman [:rnewman]]

Android browser DB looks perfect. If one were able to do a Reset Sync from mobile to desktop, I think all would be well. Alas, that's not the case.

What probably happened is that an earlier version of Fennec resulted in dodgy records reaching your desktop, and now there's nothing to tell the desktop to remove them.

If you find your empty Mobile Bookmarks folder on desktop, and remove it, that might help. If that doesn't work, you can edit the DB…

Comment 17

[Cristian Nicolae (:xti)]

I had a new idea today and I tried some things around this issue. I modify the parent id for "mobile" folder and I set it to "2" (instead of "1"). Because of this change, I was able to see it and its content in the Library and I erased it.

Then I reinstalled the latest Nightly build and I bookmarked www.android.com. After that I synced my device with Desktop Firefox and for one minute or less, the "Mobile Bookmarks" folder disappeared from the Library, but it was listed back and contains the android.com as it was expected. However, the "Mobile Bookmarks" icon was different now, like a regular "folder" icon. Also I was able to add new bookmarks and folders in it. I ran the command that you mentioned in comment #10 and another "mobile" entry was added in the list with this id: 42427.

On Nightly, besides those 4 entries, another one was added now: "mobile", but it's always empty. I bookmarked another page (cnn.com) to check where it will be saved in this case and everything works as expected both on mobile and desktop side. The new bookmark was listed in the right folder: "Mobile Bookmarks". At least one issue was resolved.

I had an idea to change the id for the new "mobile" folder from 42427 to "6" (I guess that this is the default value for "Mobile Bookmarks"). Now the "Mobile Bookmarks" icon looks as expected and also I was not able any more to add folders or new bookmarks in it. 

However still remains the additional "mobile" entry in Fennec Bookmarks tab after the Sync is performed (which is always empty).

Comment 18

[Cristian Nicolae (:xti)]

Oh, it seems that after I changed the "mobile" id to "6", the "Mobile Bookmarks" is empty...again... I checked it after I wrote the previous comment.

Comment 19

[Cristian Nicolae (:xti)]

I did some exploratory testing regarding this issue and here is it what I found so far:

Case 1: Desktop Beta build + XUL + Native

I've installed a fresh Desktop beta build and the latest XUL Nightly build (2012-03-08). I saved a couple of bookmarks both on Desktop and Mobile Firefox and after that I did Sync. After the Sync was completed, I uninstalled the XUL build and I installed the latest regular Native Nightly build (2012-03-08). It seems that a new folder was created on Desktop side named Mobile Bookmarks (I was able to add/remove bookmarks and its icon was like for a regular folder - it should be similar like for other 3 main categories imo).
I did some bookmarks on the mobile side and I Synced it with Desktop side. The Desktop Mobile Bookmarks entry was not changed but the new synced mobile bookmarks from the Native build were not listed there. It seems that if a XUL build was synced before, after that any Native bookmarks will be not listed on the Desktop side.
Also besides the 4 bookmark categories on mobile, another one was listed with a blank name.
Sometimes, the existing Mobile Bookmarks folder from Desktop Library could be hidden at this point (I am not sure when and why does it happen).



Case 2: Desktop Beta build + Native

In this case everything works as expected, but the Mobile Bookmarks entry is listed like a regular folder in the Desktop Library.


More details like desktop sync-logs (logOnSucces was enabled for both cases), changes regarding the bookmarks hierarchy in places.sqlite or screenshots can be found in attached zip file

Because of the 1st case, I will nom this bug as a betablocker. The first case will occur for all existing XUL users when the Native will be released in the Google Play.

Comment 20

[Cristian Nicolae (:xti)]

Attachment: cases.zip

Comment 21

[Richard Newman [:rnewman]]

Sync on desktop ensures that there's a Mobile Bookmarks folder. Unfortunately, the code that does so is insane: Bug 733507. That could result in the presence of two Mobile Bookmarks folders, and that's probably what you're seeing in Case 2.

Could you clarify: did you completely uninstall the XUL build, or did you upgrade it to a native client?

The former implies that (XUL + desktop) results in an inconsistent state; the latter adds profile migration to the mix.

Hypothesis:

* Desktop creates a Mobile Bookmarks folder with some dodgy root. It does this immediately prior to the first sync.
* XUL Fennec really uses this Mobile Bookmarks folder, and its own operations "fix" it.
* If you've used XUL Fennec, your bookmarks go in the "fixed" folder, and you can't see them (because it's under a different root!). Desktop creates another Mobile Bookmarks folder. If you haven't, they go in the Mobile Bookmarks folder, and you can see them.

Comment 22

[Mark Finkle (:mfinkle) (use needinfo?)]

Richard - Should this be blocking? What's your take on the severity?

Comment 23

[Richard Newman [:rnewman]]

(In reply to Mark Finkle (:mfinkle) from comment #22)
> Richard - Should this be blocking? What's your take on the severity?

Getting to the bottom of it is certainly blocking; if there's a common upgrade path that puts your bookmarks somewhere you can't ever get them them, we should fix it.

Comment 24

[Cristian Nicolae (:xti)]

(In reply to Richard Newman [:rnewman] from comment #21)
> 
> Could you clarify: did you completely uninstall the XUL build, or did you
> upgrade it to a native client?

I completely uninstalled the XUL build and I did a fresh and clean install of the latest Native build.

Comment 25

[Richard Newman [:rnewman]]

Hmm, I might be misleading myself.

There's the root, which is under the places root, and there's a query, which is under the library. (I think.)

The query does not seem to be synced; the mobile folder is.

More thinking.

Comment 26

[Richard Newman [:rnewman]]

(Darn flags and refreshing!)

Here's what I did.

* New desktop profile. Sync.
* New XUL Fennec profile. Sync. Mobile Bookmarks folder is created on desktop.
* Add bookmark on mobile. Sync.
* Sync on desktop. Mobile bookmark appears in Mobile Bookmarks folder.
* Uninstall XUL Fennec.
* Install Native Fennec. Sync.
* Empty folder, containing empty folder (loops back to top) and "All Bookmarks" appears on device.
  -- my hypothesis is that this is related to the mobile root.
* Add bookmark in Native Fennec. Sync.
* Sync on desktop. New mobile bookmark appears in Mobile Bookmarks folder as expected.

I have places.sqlite snapshots of each stage, so I will examine those now.

The only confusion, then, is about the blank folder. But I'll continue to investigate.

Comment 27

[Richard Newman [:rnewman]]

Observed oddities:

* Every bookmark created in XUL Fennec is annotated:

30    YJZxfV25o0VN      24    Slashdot: News for n  1   PlacesOrganizer/OrganizerQuery          
30    YJZxfV25o0VN      24    Slashdot: News for n  1   places/excludeFromBackup                

This is not the case with Android Sync. I don't know why this is.


* Something is uploading the All Bookmarks entry…

1331367960.25: {id:"KvA1pFYqLRqM", children:[], parentid:"kR2q-yWTBopJ", title:"All Bookmarks", type:"folder", parentName:""}

… and its parent:

1331367960.25: {id:"kR2q-yWTBopJ", children:["KvA1pFYqLRqM"], parentid:"places", title:"", type:"folder", parentName:""}

This is created on the first sync, and apparently uploaded on the second sync. (It has the second-oldest timestamp on the server.)

Those two records explain the untitled folder and its "All Bookmarks" in Native Fennec.

All server records: https://gist.github.com/2010895



So, we have two problems:

* Android Sync is applying the "mobile" title when it shouldn't. Margaret just fixed this on the display side, and I'll probably close Bug 733481 to allow for easier reconciling.

* Something is uploading the Library (I guess) and All Bookmarks. (My guess is XUL Fennec.) Android Sync isn't handling this well. Most likely it should ignore any records that are anchored in a non-standard root… and no other client should upload them, either.


Cristian: I am not able to reproduce your finding of Native bookmarks failing to appear on desktop. Did you make sure to force a sync on each device, in the right order?

Comment 28

[Cristian Nicolae (:xti)]

(In reply to Richard Newman [:rnewman] from comment #27)
> 
> Cristian: I am not able to reproduce your finding of Native bookmarks
> failing to appear on desktop. Did you make sure to force a sync on each
> device, in the right order?

I tried to reproduce this steps again using the latest XUL and Native builds and now I am able to see the Native bookmarks too. Perhaps something was fixed meanwhile.

Comment 29

[Richard Newman [:rnewman]]

OK, so Bug 731267 will handle the empty/All Bookmarks folders from XUL Fennec users, and everything else is take care of. Marking this as resolved.

Thanks for the debugging, xti!

Comment 30

[Andreea Pod]

There are no other folders in the Bookmarks tab, besides the expected ones after sync.
Verified fixed on build: Firefox 14.0a1 (2012-03-23)
Device: LG Optimus 2X (Android 2.2)