1332751 - Validation error when syncing bookmarks to and from Firefox account

Status: RESOLVED DUPLICATE of bug 1323333

(Firefox :: Sync, defect)

Description

[Kanchan Kumari QA]

Attachment: ValidatioError20thJan

Pre-requisite: About sync addon is installed

STR:
1. Create a fresh profile and add few thousand bookmarks to it
2. Sign in to existing sync account
3. Trigger the sync
4. Go to Tools > About Sync.Under the “bookmarks” section, click the      “Validation” tab

Expected: There should be  “No validation problems found \o/”. 

Actual: Validation error like "Server record references child 5Ep2rTveyrf5 that doesn't exist on the server"

Note:Seeing same kind of validation error with same Fx account on different profiles and platforms.

Comment 1

[Thom Chiovoloni [:tcsc] (ex-moco)]

1. Is this reliable? E.g. whenever you create a fresh profile with a large # of bookmarks, and sync, after the sync completes you see a validation error in about:sync?

2. Are there any error logs in your about:sync-log?

3. Were you on windows? If so, did you restart the browser between 1 and 2? On windows there are substantial known performance issues in this case, and they *might* cause the sync to fail (although I don't know for sure about that?). But it's a case that only happens during testing, it's not something we prioritize -- This was mostly in bug 1296560, which was resolved as WORKSFORME since it should work in every realistic case.

On the bright side, even if this is reliable (which I hope it isn't...) this is an error that will be caught by our repair requester, once it's completed.

Comment 2

[Thom Chiovoloni [:tcsc] (ex-moco)]

Whoops, didn't mean to resolve this D:

Comment 3

[Kanchan Kumari QA]

(In reply to Thom Chiovoloni [:tcsc] from comment #1)
> 1. Is this reliable? E.g. whenever you create a fresh profile with a large #
> of bookmarks, and sync, after the sync completes you see a validation error
> in about:sync?
This issue is not reproducible if I create fresh profile and sign in to new sync account. I could reproduce this  issue today too with fresh profile (with large # of bookmarks) and one specific (existing) sync account. I saw same validation error in about:sync every time. I attached JSON export (validation result) to bug report.
But at the end after triggering sync multiple time at some interval, I don't see validation error anymore. Seems that there was some problem with syncing before. 

> 2. Are there any error logs in your about:sync-log?
No I don't see any error log
 
> 3. Were you on windows? If so, did you restart the browser between 1 and 2?
I had tested this issue on Windows and Linux. I created profile, synced to Fx account on Windows and verified the sync on Linux with same Fx account. I was getting same kind of validation error on Linux. No I didn't restart browser between 1 and 2.

> On the bright side, even if this is reliable (which I hope it isn't...)
Seems, this issue is not reliable :)

Comment 4

[Thom Chiovoloni [:tcsc] (ex-moco)]

Can you try a bit more to reproduce this? If it is repeatable, even if rare, we definitely want to fix it.

Also, it is probably beneficial if you enable logging on success in your profiles (just in general this is a good idea if you're going to be doing QA for sync). This is controlled by the "services.sync.log.appender.file.logOnSuccess" pref, but about sync can change this for you.

It's also probably worth enabling debug or trace (note that trace can contain PII) logging, and the easiest way to do that is by using about sync, since it's several prefs.

If you can reproduce this and get a copy of the success (or error) logs with debug or trace enabled, that would at least get us to the point where we can track down what happened.

Comment 5

[Kanchan Kumari QA]

Attachment: ValidationError

I already set About Sync pref “Create log files even on success?” and “Remember these values when Sync is reconfigured?” checked, “Level of messages written by Sync engines” and “Level of messages written to about:sync-logs log files”  to “Trace”. 

I am again getting validation error but of different kind "Server-side parent/child mismatch for parent mobile (first) and child Hf13ZPOq64ld (second)", there is no error log, only success log. Attached is Jason export of Validation result and Sync success log.

I am also attaching zip file of logs created during(From comment 3:
>I could reproduce this  issue today too with fresh profile (with large # of bookmarks) and one specific (existing) sync account. I saw same validation error in about:sync every time. But at the end after triggering sync multiple time at some interval, I don't see validation error anymore.)

Hope this helps. Let me know if you have any question regarding this. I will try reproducing same reported validation error.

Comment 6

[Kanchan Kumari QA]

Attachment: aboutsync-logfiles.zip

Comment 7

[Kanchan Kumari QA]

Attachment: success-sync-1485214023422.txt

Comment 8

[Mark Hammond [:markh] [:mhammond]]

That's very interesting, thanks! The issue appear to be around how we de-dupe existing bookmarks and we are having some discussions in bug 1323333. To help give us some context and judge how frequently people might hit this, could you please try and give us some history of these profiles? It *looks* like you used a script to pre-populate a number of bookmarks, and used this script independently on multiple profiles, then synced these profiles. Does that sound roughly correct?

Comment 9

[Kanchan Kumari QA]

(In reply to Mark Hammond [:markh] from comment #8)
> It *looks* like you used a script to pre-populate a number of bookmarks, and
> used this script independently on multiple profiles, then synced these
> profiles. Does that sound roughly correct?

Yes, that's absolutely correct. I used following steps:
1. Created a profile and used the script to add a large number of bookmarks to it.
2. Synced these bookmarks to Fx account
3. Created another profile on another OS/device and and synced with same Fx account (for bookmark verification)
4. I created multiple profiles on different OS/devices and used same script to populate bookmarks to it (for testing purpose) and used same sync account through all these profiles to sync bookmarks. 
5. Sometimes I also used another existing Fx account with these profiles. 

So, yes a number of syncing (client to server and server to client) were done with multiple profiles, sync accounts, and devices.

Comment 10

[Mark Hammond [:markh] [:mhammond]]

(In reply to Kanchan Kumari QA from comment #9)
> Yes, that's absolutely correct. I used following steps:

Thanks for clarifying! Unfortunately, this has hit the worse-case scenario for Sync - these multiple profiles have very similar bookmark structures, but each profile has a different unique identifier for the bookmarks. When Sync runs, it tries to de-duplicate these items and, basically, screws it up. This has helped us with new test cases, but the work to fix this isn't trivial and is probably going to take a number of months. It's even possible that the end result will be that we *don't* de-dupe at all, leaving these multiple bookmark trees across all profiles (ie, so each profile would end up with multiple copies of the bookmarks) and rely on the user manually deleting the ones they want deleted (at which point we *will* do the right thing).

So, long story short, this kind of testing is valuable, but probably only if there are *different* trees across the profiles. For example, either:

* Run the script on one profile, but not the others. When connecting the other profiles, ensure that the bookmarks generated on that first profile have correctly been synced to the other "clean" profiles and no validation errors exist.

* Similarly to the above: have a subtly different script for each of the profiles - eg, on one profile have the test folder "test 1", with many bookmarks in that folder named, say "test bookmark 1 in profile 1", "test bookmark 2 in profile 1". On the other profiles replace "test 1" and "profile 1" with "test 2" and "profile 2" etc - this way they will not be seen as duplicates to be resolved, and all those separate bookmark trees should end up on every profile, and none of the profiles should report validation errors.

I hope that makes sense - feel free to ping me if it doesn't - and thanks very much for your help here, I find it very valuable (even if it does sometimes highlight problems we can't always fix!)

Comment 11

[Lina Butler [:lina]]

Let's roll this up into bug 1323333.