965910 - Sync intermittently failing with prod token server

Status: VERIFIED FIXED

(Firefox :: Sync, defect, P3)

Description

[Edwin Wong [:edwong]]

1.  Using fx-team-<platform>/1390969090/ build:
2. set prefs:
identity.fxaccounts.enabled=true
services.sync.tokenServerURI=https://token.services.mozilla.com//1.0/sync/1.5
3. sign in with an acct, add bookmark, and select sync now
4. open a new profile with same build, prefs, and sign in with same acct
5. select - sync now

actual: bookmark is not synced
I've also tried restarting both profiles.  Sync now from either profile.  Karl also confirmed this.

Comment 1

[Edwin Wong [:edwong]]

Attachment: sync-log.txt

Comment 2

[Toby Elliott [:telliott]]

Gonna ask the obvious question here - looks like it's pointing at a sync1.1 instance? Is this just a URL oversight, or are we accessing the wrong URL?

Comment 3

[Edwin Wong [:edwong]]

:telliot is this not the correct pref to set?
services.sync.tokenServerURI=https://token.services.mozilla.com//1.0/sync/1.5

Comment 4

[Toby Elliott [:telliott]]

Yes, it's the resulting sync node that the client is pointing at (https://sync-1-us-east-1.sync.services.mozilla.com/1.1/8988/info/collections) that I'm concerned about.

Comment 5

[Chris Karlof [:ckarlof]]

This is related to how (on desktop) we construct the storage endpoint from the token server response. Nightly currently does it the way legacy sync does it by "building" it like: 

> this.service.clusterURL + SYNC_API_VERSION + "/" + this.identity.username + "/"

where SYNC_API_VERSION = "1.1" and this.service.clusterURL is the token server response with the path stripped down to "/". This happens to work because it matches the current token server response, modulo 1.1/1.5 versions in the path. 

Bug 960887 fixes this process to use the token server response in FxA Sync and do it old school for Legacy Sync. This Bug was backed out last night due to test failure garbage.

But :rfk says using 1.1 in the new storage URLs will work just fine for now.

Comment 6

[Chris Karlof [:ckarlof]]

FWIW, the test failures were related to something else, which we've tracked down this morning, so hopefully this detail will get cleared up in tomorrow nightly, but I don't think this is related to edwin's issue.

Comment 7

[Toby Elliott [:telliott]]

OK, good to know. That's the main thing that stood out from the logs for me.

Comment 8

[Ryan Kelly [:rfkelly]]

Hmm, that log doesn't seem to show much bookmark-related activity, but this is a little suspicious:

   1391108334090	Sync.Engine.History	DEBUG	Records that will be uploaded again because the server couldn't store them: hmDoOdoTZDmE, XumM6hGPdQE6, 2BFYI3rleCOZ, Wzd_fdDeCl

I'll try to reproduce.

Comment 9

[Ryan Kelly [:rfkelly]]

This worked OK for me on Windows, bookmark synced successfully from one profile to another and back again.  Edwin is this the only sync log from your system?

Comment 10

[Ryan Kelly [:rfkelly]]

Ah, so I was using Nightly, I'll try with a fxa-team build

Comment 11

[Ryan Kelly [:rfkelly]]

Unable to reproduce with that specific build either (again on Windows).  Platform specific bug maybe?

Comment 12

[Ryan Kelly [:rfkelly]]

I see some not-properly-syncing behaviour if I try throwing some folders into the mix, will work through that in more detail

Comment 13

[Edwin Wong [:edwong]]

This is generally fixed - but intermittent.  I'll update the title.

Comment 14

[Edwin Wong [:edwong]]

syncing generally works however, it looks like:
The token server for production is https://token.services.mozilla.com/1.0/sync/1.5 which means ‘give me a token for sync 1.5’. But the endpoint that it returns is https://sync-1-us-east-1.sync.services.mozilla.com/1.1/$UID. Note the 1.5 vs 1.1.

the client is building that URL - thus we're waiting for a patch to land to resolve.

Comment 15

[Chris Karlof [:ckarlof]]

This should be resolved now, meaning we landed the prod token server url in Desktop, and Desktop should be using 1.5 storage URLs.

Comment 16

[Chris Karlof [:ckarlof]]

It landed early this morning in Bug 960887. Desktop team is spinning up new Nightly builds, so I'm hoping you can see that in a couple hours at the latest.

Comment 17

[Edwin Wong [:edwong]]

With the Nightly build here:
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/firefox-29.0a1.en-US.mac.dmg
31-Jan-2014 19:03
	 
I'm seeing this in my logs:
https://sync-1-us-east-1.sync.services.mozilla.com/1.5/...

Comment 18

[James Bonacci [:jbonacci]]

We need to have someone look into this...

Comment 19

[Chris Karlof [:ckarlof]]

Edwin, are you still having problems? That URL looks correct.

Comment 20

[Edwin Wong [:edwong]]

This issue is fixed with 2/3 nightly.

Comment 21

[James Bonacci [:jbonacci]]

good.