1437153 - sync duplicates bookmark/folder

Status: RESOLVED INVALID

(Firefox :: Sync, defect)

Description

[Kanchan Kumari QA]

Attachment: duplicate.PNG

Prerequisite:
1. About:sync addon installed
2. services.sync.engine.bookmarks.buffer is set to true

STR:
1. Set up sync on two devices
2. Run the backup script https://gist.github.com/kitcambridge/b08dae1f4f1b5a93f2d30425cdef09d5 and save tree-with-new-guids.json file
3. Restore same json file on both device and sync 
(Note: no duplicated items were observed here)

4. import same HTML bookmark(exported from same profile)file on both device
5. Sync both device
6. Run this script ( https://gist.github.com/kitcambridge/ae61d4e8ce54db74d18379c5d8393258 )in browser context

Observed:infinite sync bug 1436573. No new bookmark got added for step 4.

5. Quit the browser
6. Sync

Observed: Duplicate items in bookmark library. No new bookmarks got added due to step 4. Sync works fine and there is no error logged in about:sync

Comment 1

[Mark Hammond [:markh] [:mhammond]]

I'm afraid I can't reproduce this. One other strange thing I noticed from your screenshot is that the duplicated "Recent Tags" is typically in the "Bookmarks Menu", but all the dupes are in your "Other Bookmarks" - but I expect that's a different symptom of the same problem.

I also had a little trouble following the STR as written as it seems to ask for certain things to be done simultaneously on 2 different devices/profiles - I read that as "do it on the first device, then on the second" - which might explain why I had trouble.

Kanchan, can you please try to repro again, and attach "Trace" logs from both the devices?

Comment 2

[Kanchan Kumari QA]

Attachment: logsOndevice1

More clear STR:
1. Set up sync on two devices
2. Run the backup script https://gist.github.com/kitcambridge/b08dae1f4f1b5a93f2d30425cdef09d5 and save tree-with-new-guids.json file on device 1.
3. Restore json file on device 1
4. Restore same json file on device 2 

Things worked as expected till here.

5. Export bookmarks to HTML file on device 1.
6. Export bookmarks to HTML file on device 2.

I encountered bug 1443343. After couple minutes all bookmarks reappeared in library.

7. Import same (exported in step 5) HTML bookmarks file on device 1.
8. Import same (exported in step 6) HTML bookmarks file on device 2.
7. Trigger sync on both devices.

Bookmarks synced as expected.

8. Run this script ( https://gist.github.com/kitcambridge/ae61d4e8ce54db74d18379c5d8393258 )in browser context on device 1.

Observed the bug 1437153

Attached  are trace logged on both devices.

Comment 3

[Kanchan Kumari QA]

Attachment: TraceLoggedOndevice2

Comment 4

[Mark Hammond [:markh] [:mhammond]]

Thanks Kanchan. I'll dig into this a little later, but I think it's going to be WONTFIX/INVALID - doing an import (as opposed to a restore) is known to cause dupes. The new bookmark engine probably changes some of the semantics a little as the new general rule is "only de-dupe items from this device which have never synced". However, I'll ni? myself to check that what you are reporting here is actually doing what we expect.

Comment 5

[Lina Butler [:lina]]

Hi Kanchan, thanks for posting the steps! 

(In reply to Kanchan Kumari QA from comment #2)
> 2. Run the backup script
> https://gist.github.com/kitcambridge/b08dae1f4f1b5a93f2d30425cdef09d5 and
> save tree-with-new-guids.json file on device 1.
> 3. Restore json file on device 1

Just to make sure, are you exporting the tree on device 1, then immediately restoring on device 1? That won't trigger Sync's deduping on device 2, because you're importing bookmarks with the same GUIDs (in `tree-with-new-guids.json`) on both devices. IOW, the end state of (4) is the same as (1). A good way to exercise deduping is:

1. Set up sync with existing bookmarks on device 1.
2. Run the backup script to save `tree-with-new-guids.json` on device 1.
3. Restore `tree-with-new-guids.json` on device 2.
4. Set up sync on device 2.

After step 4, device 2 shouldn't have duplicates.

> 7. Import same (exported in step 5) HTML bookmarks file on device 1.

As Mark said in comment 4, this will create duplicates on device 1, because importing from HTML doesn't erase existing bookmarks before import.

> 8. Import same (exported in step 6) HTML bookmarks file on device 2.

This will also create duplicates on device 2, for the same reason.

> 7. Trigger sync on both devices.

So, after this step, both devices should have two copies of each bookmark. (If deduping failed, you'd see 3 or 4, which is wrong). Is that what you're seeing?

Comment 6

[Kanchan Kumari QA]

(In reply to Kit Cambridge (they/them) [:kitcambridge] from comment #5)

> Just to make sure, are you exporting the tree on device 1, then immediately
> restoring on device 1? 

Yes, here I was doing this.

>A good way to exercise deduping is:
> 1. Set up sync with existing bookmarks on device 1.
> 2. Run the backup script to save `tree-with-new-guids.json` on device 1.
> 3. Restore `tree-with-new-guids.json` on device 2.
> 4. Set up sync on device 2.
> 
> After step 4, device 2 shouldn't have duplicates.

I tested this way too and it worked as expected/no duplicate bookmarks.


> > 7. Import same (exported in step 5) HTML bookmarks file on device 1.
> 
> As Mark said in comment 4, this will create duplicates on device 1, because
> importing from HTML doesn't erase existing bookmarks before import.
> 
> > 8. Import same (exported in step 6) HTML bookmarks file on device 2.
> 
> This will also create duplicates on device 2, for the same reason.
> 
> > 7. Trigger sync on both devices.
> 
> So, after this step, both devices should have two copies of each bookmark.

Seems things are working fine then.

Closing this bug as resolved:Invalid