765064 - HttpClient in use by Sync and other services doesn't support SNI

Status: RESOLVED DUPLICATE of bug 1412650

(Firefox for Android Graveyard :: Android Sync, defect, P3)

Description

[Patrick]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20100101 Firefox/13.0
Build ID: 20120609111901

Steps to reproduce:

1) Installed Firefox Beta from Google Play Store
2) Set up sync with my own server, which worked in non-native Fennec, and works in other Firefox instances on Linux and Windows. I made sure I have only one instance (no old Firefox, no Aurora, ...)



Actual results:

Nothing. No syncing is happening (although the sync logo shows up for some time in the status bar). I checked the server logs - at no time did my Android phone connect to the sync server. I read about SSL certificate issues elsewhere, but there shouldn't be a problem with mine (StartSSL cert.), and it seems Firefox / Sync does not even get that far. No error message anywhere, it just does not work. I spent half an hour trying to scroll through logcat messages but also did not find anything useful.


Expected results:

At least there should have been some error message of what has gone wrong. Ideally of course the syncing should have worked, as it had before, for many months.

Comment 1

[Richard Newman [:rnewman]]

Please attach your logs from the device.

Comment 2

[Nick Alexander :nalexander [he/him]]

Dev investigation pending.  QA (tracy) reports minimal testing against custom servers, so it's possible we're missing something obvious.

Comment 3

[Patrick]

Thanks for all the reactions so far. Today I had more time to investigate, and I think I got some useful information. After clearing the app's data and removing the Sync account, I started logging to my PC and set up my sync account from the fresh Firefox Beta on my phone.

There is actually a connection to my server (https://sync.patrick-nagel.net), but something is wrong with the SSL certificate verification: "06-16 15:22:08.737 12650 12696 W GlobalSession: javax.net.ssl.SSLException: hostname in certificate didn't match: <sync.patrick-nagel.net> != <c3po.patrick-nagel.net> OR <c3po.patrick-nagel.net> OR <patrick-nagel.net>" indicates that the certificate hostname does not match - but that's not true (see http://p173.de/s/1339832075.png and http://p173.de/s/1339832117.png). It rather looks like as if it's checking the certificate on https://patrick-nagel.net.

Maybe the new syncing method does no longer support SNI (http://en.wikipedia.org/wiki/Server_Name_Indication)?

06-16 15:21:28.748 12650 12650 I SetupSync: SetupSyncActivity constructor called.
06-16 15:21:28.748 12650 12650 I SetupSync: Called SetupSyncActivity.onCreate.
06-16 15:21:28.748 12650 12650 I SetupSync: Called SetupSyncActivity.onResume.
06-16 15:21:29.159  1322  1354 I ActivityManager: Displayed org.mozilla.firefox_beta/org.mozilla.gecko.sync.setup.activities.SetupSyncActivity: +423ms
06-16 15:21:29.159 12650 12650 I GeckoApp: stop
06-16 15:21:29.629 12650 12652 D dalvikvm: GC_CONCURRENT freed 419K, 52% free 3423K/7111K, external 3563K/4138K, paused 2ms+2ms
06-16 15:21:30.500 12650 12696 E JPakeCrypto: myhash: a58f080c17b9115c30f403dceea476[...]
06-16 15:21:30.560 12650 12696 E JPakeCrypto: myhash: 51e0c7e767c828f5f32e7e26e38f5d[...]
06-16 15:21:31.641  1400  3472 D dalvikvm: GC_EXTERNAL_ALLOC freed 128K, 51% free 3372K/6791K, external 2580K/2598K, paused 41ms
06-16 15:21:39.879  1322  5154 D dalvikvm: GC_EXTERNAL_ALLOC freed 288K, 39% free 8086K/13191K, external 3610K/3616K, paused 87ms
06-16 15:21:43.773 11831 11839 D KEEPSCREEN IS: [45] org.mozilla.firefox_beta
06-16 15:21:48.197  1799  1801 D dalvikvm: GC_CONCURRENT freed 396K, 51% free 3055K/6215K, external 667K/1179K, paused 2ms+2ms
06-16 15:21:52.281  1400  7980 D dalvikvm: GC_EXTERNAL_ALLOC freed 167K, 51% free 3372K/6791K, external 2584K/2598K, paused 42ms
06-16 15:21:55.114 12650 12706 D JPakeCrypto: round2 started.
06-16 15:21:55.244 12650 12706 D JPakeCrypto: *** ZKP SUCCESS ***
06-16 15:21:55.354 12650 12706 D JPakeCrypto: *** ZKP SUCCESS ***
06-16 15:21:55.434 12650 12706 E JPakeCrypto: myhash: 8fc893c37177d8154785d4773f5b253[...]
06-16 15:21:55.444 12650 12706 D JPakeCrypto: round2 finished.
06-16 15:21:55.494 12650 12652 D dalvikvm: GC_CONCURRENT freed 1142K, 50% free 3688K/7303K, external 1528K/3563K, paused 2ms+3ms
06-16 15:21:59.528 12650 12694 D JPakeCrypto: Final round started.
06-16 15:21:59.649 12650 12694 D JPakeCrypto: *** ZKP SUCCESS ***
06-16 15:21:59.729 12650 12694 D JPakeCrypto: Final round finished; returning key.
06-16 15:22:03.412 12650 12695 I SyncAccounts: Setting explicit server URL: https://sync.patrick-nagel.net/
06-16 15:22:03.462  1322  1472 D SyncManager: setSyncAutomatically: , provider org.mozilla.firefox_beta.db.browser -> true
06-16 15:22:03.552  1322  1328 D SyncManager: setSyncAutomatically: , provider subscribedfeeds -> true
06-16 15:22:03.763 11850 11861 D Volley  : [13] DiskBasedCache.clear: Cache cleared.
06-16 15:22:03.823  1322  9067 D SyncManager: setIsSyncable: Account {name=me, type=org.mozilla.firefox_sync}, provider org.mozilla.firefox_beta.db.browser -> 1
06-16 15:22:03.873 11850 11865 D Volley  : [17] DiskBasedCache.clear: Cache cleared.
06-16 15:22:03.893 11874 11874 I PicasaSyncManager: account change detect - update database
06-16 15:22:03.893  1322  1328 D SyncManager: setSyncAutomatically: already set to true, doing nothing
06-16 15:22:03.893 12650 12695 I SyncAccounts: Clearing preferences for this account.
06-16 15:22:03.903 11874 12538 D PicasaSync: sync account database
06-16 15:22:04.133 11874 12538 D PicasaSync: accounts in DB=1                                                                         
06-16 15:22:04.163  1322  4346 I ActivityManager: Starting: Intent { flg=0x30000 cmp=org.mozilla.firefox_beta/org.mozilla.gecko.sync.setup.activities.SetupSuccessActivity (has extras) } from pid 12650                                                                    
06-16 15:22:04.203  1322  1329 I ActivityManager: Start proc com.google.android.youtube for broadcast com.google.android.youtube/.core.async.UserAuthorizer$AccountsChangedReceiver: pid=12713 uid=10102 gids={3003, 1015}                                                  
06-16 15:22:04.293  1322  1466 D SyncManager: setIsSyncable: Account {name=[...]@googlemail.com, type=com.google}, provider com.google.android.apps.books -> 1                                                                                                      
06-16 15:22:04.293  1322  1466 D SyncManager: setIsSyncable: already set to 1, doing nothing                                          
06-16 15:22:04.423  1322  1354 I ActivityManager: Displayed org.mozilla.firefox_beta/org.mozilla.gecko.sync.setup.activities.SetupSuccessActivity: +252ms                                                                                                                   
06-16 15:22:04.483 12713 12713 D szipinf : Initializing inflate state
06-16 15:22:04.523 12713 12713 I ActivityThread: Pub com.google.android.youtube.SuggestionProvider: com.google.android.youtube.core.suggest.SuggestionProvider                                                                                                              
06-16 15:22:04.984 12713 12715 D dalvikvm: GC_CONCURRENT freed 236K, 49% free 2903K/5639K, external 0K/0K, paused 2ms+2ms
06-16 15:22:05.334 12713 12739 I Database: sqlite returned: error code = 0, msg = Recovered 5 frames from WAL file /data/data/com.google.android.youtube/databases/downloads.db-wal                                                                                         
06-16 15:22:05.374 12650 12650 D SyncAdapter: AccountManager.get(org.mozilla.gecko.GeckoApplication@40533d80)
06-16 15:22:05.374 12650 12741 I SyncAdapter: Got onPerformSync. Extras bundle is Bundle[{}]
06-16 15:22:05.374 12650 12741 I SyncAdapter: Account name: me                                                                    
06-16 15:22:05.374 12650 12741 D SyncAdapter: Invalidating auth token.
06-16 15:22:05.384 12650 12656 I SyncAuthService: AccountManager.get(org.mozilla.gecko.sync.setup.SyncAuthenticatorService@40590cd0)
06-16 15:22:05.414 12650 12656 I SyncAuthService: Setting username. Null? false                                                       
06-16 15:22:05.414 12650 12656 I SyncAuthService: Setting Sync Key. Null? false                                                       
06-16 15:22:05.724 12650 12657 I SyncAuthService: AccountManager.get(org.mozilla.gecko.sync.setup.SyncAuthenticatorService@405bb670)  
06-16 15:22:05.724 12650 12657 I SyncAuthService: Setting username. Null? false                                                       
06-16 15:22:05.724 12650 12657 I SyncAuthService: Setting Sync Key. Null? false                                                       
06-16 15:22:05.734 12650 12650 I SyncAdapter: AccountManagerCallback invoked.                                                         
06-16 15:22:05.734 12650 12650 D SyncAdapter: Username: me
06-16 15:22:05.734 12650 12650 D SyncAdapter: Server:   https://sync.patrick-nagel.net/                                               
06-16 15:22:05.734 12650 12650 D SyncAdapter: Password? true                                                                          
06-16 15:22:05.734 12650 12650 D SyncAdapter: Key?      true                                                                          
06-16 15:22:05.734 12650 12650 I SyncAdapter: Performing sync.
06-16 15:22:05.744 12650 12650 I GlobalSession: GlobalSession initialized with bundle Bundle[{}]                                      
06-16 15:22:05.754 12650 12650 I SyncAdapter: Stage completed: idle                                                                   
06-16 15:22:05.754 12650 12650 I GlobalSession: Running next stage checkPreconditions (org.mozilla.gecko.sync.stage.CheckPreconditionsStage@406d1660)...                                                                                                                    
06-16 15:22:05.754 12650 12650 I SyncAdapter: Stage completed: checkPreconditions                                                     
06-16 15:22:05.754 12650 12650 I GlobalSession: Running next stage ensureClusterURL (org.mozilla.gecko.sync.stage.EnsureClusterURLStage@407269b0)...                                                                                                                        
06-16 15:22:05.754 12650 12650 I EnsureClusterURLStage: Fetching cluster URL.                                                         
06-16 15:22:05.754 12650 12696 D EnsureClusterURLStage: In fetchClusterURL: node/weave is https://sync.patrick-nagel.net/user/1.0/me/node/weave                                                                                                                         
06-16 15:22:05.775 12650 12741 I SyncAdapter: Waiting on sync monitor.
06-16 15:22:05.895 11850 11862 I ElegantRequestDirector: I/O exception (org.apache.http.NoHttpResponseException) caught when processing request: The target server failed to respond                                                                                        
06-16 15:22:05.895 11850 11862 I ElegantRequestDirector: Retrying request                                                             
06-16 15:22:06.455 11850 11852 D dalvikvm: GC_CONCURRENT freed 2156K, 48% free 5157K/9863K, external 4401K/5326K, paused 20ms+101ms
06-16 15:22:06.996 12650 12652 D dalvikvm: GC_CONCURRENT freed 761K, 49% free 3780K/7303K, external 1528K/3563K, paused 6ms+3ms       
06-16 15:22:07.366 11850 11863 I global  : Loaded time zone names for en_US in 856ms.
06-16 15:22:07.626 12650 12696 D class ch.boye.httpclientandroidlib.impl.conn.DefaultClientConnection: Connection closed
06-16 15:22:07.626 12650 12696 D class ch.boye.httpclientandroidlib.impl.conn.DefaultClientConnection: Connection shut down           
06-16 15:22:07.696 11850 11850 D Finsky  : [1] DetailsDataBasedFragment.rebindViews: Page [class=DetailsFragment] loaded in [1076897 ms] (partial? true)                                                                                                                    
06-16 15:22:07.766 11831 11839 D KEEPSCREEN IS: [46] org.mozilla.firefox_beta                                                         
06-16 15:22:07.877 11850 11852 D dalvikvm: GC_CONCURRENT freed 1856K, 46% free 5349K/9863K, external 4431K/5326K, paused 3ms+6ms      
06-16 15:22:07.917 11850 11855 I dalvikvm: Jit: resizing JitTable from 8192 to 16384
06-16 15:22:08.667  1322  1472 D dalvikvm: GC_EXTERNAL_ALLOC freed 831K, 39% free 8151K/13191K, external 3610K/3616K, paused 81ms
06-16 15:22:08.717 12650 12696 D class ch.boye.httpclientandroidlib.impl.conn.DefaultClientConnection: Connection closed              
06-16 15:22:08.717 12650 12696 D class ch.boye.httpclientandroidlib.impl.conn.DefaultClientConnection: Connection shut down           
06-16 15:22:08.737 12650 12696 W GlobalSession: Aborting sync: Got exception fetching cluster URL.
06-16 15:22:08.737 12650 12696 W GlobalSession: javax.net.ssl.SSLException: hostname in certificate didn't match: <sync.patrick-nagel.net> != <c3po.patrick-nagel.net> OR <c3po.patrick-nagel.net> OR <patrick-nagel.net>                                                   
06-16 15:22:08.737 12650 12696 W GlobalSession:         at ch.boye.httpclientandroidlib.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:228)                                                                                                                         
06-16 15:22:08.737 12650 12696 W GlobalSession:         at ch.boye.httpclientandroidlib.conn.ssl.BrowserCompatHostnameVerifier.verify(BrowserCompatHostnameVerifier.java:54)                                                                                                
06-16 15:22:08.737 12650 12696 W GlobalSession:         at ch.boye.httpclientandroidlib.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:149)                                                                                                                         
06-16 15:22:08.737 12650 12696 W GlobalSession:         at ch.boye.httpclientandroidlib.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:130)                                                                                                                         
06-16 15:22:08.737 12650 12696 W GlobalSession:         at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:397)                                                                                                                  
06-16 15:22:08.737 12650 12696 W GlobalSession:         at ch.boye.httpclientandroidlib.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)                                                                                  
06-16 15:22:08.737 12650 12696 W GlobalSession:         at ch.boye.httpclientandroidlib.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)                                                                                                                        
06-16 15:22:08.737 12650 12696 W GlobalSession:         at ch.boye.httpclientandroidlib.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)                                                                                                        
06-16 15:22:08.737 12650 12696 W GlobalSession:         at ch.boye.httpclientandroidlib.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:573)                                                                                                      
06-16 15:22:08.737 12650 12696 W GlobalSession:         at ch.boye.httpclientandroidlib.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)                                                                                                         
06-16 15:22:08.737 12650 12696 W GlobalSession:         at ch.boye.httpclientandroidlib.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:818)                                                                                                                 
06-16 15:22:08.737 12650 12696 W GlobalSession:         at ch.boye.httpclientandroidlib.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:752)                                                                                                                 
06-16 15:22:08.737 12650 12696 W GlobalSession:         at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:244)     
06-16 15:22:08.737 12650 12696 W GlobalSession:         at org.mozilla.gecko.sync.net.BaseResource.retryRequest(BaseResource.java:275)
06-16 15:22:08.737 12650 12696 W GlobalSession:         at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:258)     
06-16 15:22:08.737 12650 12696 W GlobalSession:         at org.mozilla.gecko.sync.net.BaseResource.go(BaseResource.java:299)          
06-16 15:22:08.737 12650 12696 W GlobalSession:         at org.mozilla.gecko.sync.net.BaseResource.get(BaseResource.java:305)         
06-16 15:22:08.737 12650 12696 W GlobalSession:         at org.mozilla.gecko.sync.stage.EnsureClusterURLStage.fetchClusterURL(EnsureClusterURLStage.java:176)                                                                                                               
06-16 15:22:08.737 12650 12696 W GlobalSession:         at org.mozilla.gecko.sync.stage.EnsureClusterURLStage$3.run(EnsureClusterURLStage.java:249)                                                                                                                         
06-16 15:22:08.737 12650 12696 W GlobalSession:         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:444)    
06-16 15:22:08.737 12650 12696 W GlobalSession:         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:306)         
06-16 15:22:08.737 12650 12696 W GlobalSession:         at java.util.concurrent.FutureTask.run(FutureTask.java:138)                   
06-16 15:22:08.737 12650 12696 W GlobalSession:         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1088)                                                                                                                                  
06-16 15:22:08.737 12650 12696 W GlobalSession:         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:581)                                                                                                                                  
06-16 15:22:08.737 12650 12696 W GlobalSession:         at java.lang.Thread.run(Thread.java:1019)                                     
06-16 15:22:08.737 12650 12696 I GlobalSession: Not uploading updated meta/global record since there are no engines requesting upload.
06-16 15:22:08.737 12650 12696 I SyncAdapter: GlobalSession indicated error. Flagging auth token as invalid, just in case.            
06-16 15:22:08.787 12650 12696 I SyncAdapter: Notifying sync monitor.                                                                 
06-16 15:22:08.797 12650 12741 I SyncAdapter: Setting minimum next sync time to 1339917728791                                         
06-16 15:22:08.827 12650 12743 D class ch.boye.httpclientandroidlib.impl.conn.tsccm.ThreadSafeClientConnManager: Closing expired connections                                                                                                                                
06-16 15:22:08.827 12650 12743 D class ch.boye.httpclientandroidlib.impl.conn.tsccm.ConnPoolByRoute: Closing expired connections      
06-16 15:22:08.827 12650 12743 D class ch.boye.httpclientandroidlib.impl.conn.tsccm.ThreadSafeClientConnManager: Shutting down        
06-16 15:22:08.827 12650 12743 D class ch.boye.httpclientandroidlib.impl.conn.DefaultClientConnection: Connection closed

Comment 4

[Richard Newman [:rnewman]]

Apache HttpClient (which Sync uses on Android) doesn't support SNI. (Neither does the Android stock browser, AFAICS.)

There'll be a workaround for this if Bug 756763 gets fixed -- accepting the wrong cert -- but otherwise this isn't something we're going to have time to address. You'll have to find a setup that does not use SNI in order to use Sync on Android.

This is Apache issue #1119. If that gets fixed, it will eventually trickle into our codebase.

https://issues.apache.org/jira/browse/HTTPCLIENT-1119

Marking this bug as depending on Bug 756763, so you'll get bugmail if that changes, and resolving this as WONTFIX because we have no plans to directly address this ourselves.

Comment 5

[Patrick]

Thanks for connecting all the dots for me, Richard. It's a pity that Apache's HTTP client does not support SNI, but their famous httpd does.

I think I have a spare IP address which I can assign as a workaround on my server side.

Comment 6

[Andi Mayer]

Recent Apache HTTP client versions supports SNI, but not the version shipped with Android.

However, there's a way to use SNI with Android's Apache HttpClient, too (Android 4.2+ only):
http://blog.dev001.net/post/67082904181/android-using-sni-with-apache-httpclient-library

Comment 7

[Richard Newman [:rnewman]]

(In reply to Andi Mayer from comment #6)
> Recent Apache HTTP client versions supports SNI, but not the version shipped
> with Android.

We ship our own, but even so the fix for #1119 requires Java 7 features, which means it ain't a fix.


> However, there's a way to use SNI with Android's Apache HttpClient, too
> (Android 4.2+ only):
> http://blog.dev001.net/post/67082904181/android-using-sni-with-apache-
> httpclient-library

Urgh, messy. But let's reopen this, see if I feel like fixing it on an airplane sometime.

Comment 8

[michael.heuberger]

Mhhhh, I am not sure but probably it is the same bug mentioned in
https://github.com/owncloud/mozilla_sync/issues/85

Unable to sync to my own onwcloud server from my Firefox on Android 27.

Comment 9

[MS]

I'm using "Android 4.4", "Firefox 36.0", "fxa-custom-server-addon 0.3" and SNI is still not working.

When I disabled "SSLStrictSNIVHostCheck On" in Apache2 for testing purposes everything works fine, but when I enable it again (and thats our normal setting) I receive the following errors:

[Apache2 Log]
> [ssl:error] [pid 24245] AH02033: No hostname was provided via SNI for a name based virtual host

[adb logcat]
> W/GeckoLogger( 7752): firefox :: FxAccountSetupTask :: Got failure.
> W/FxAccounts( 7752): firefox :: FxAccountAbstractSetupActivity :: Got exception; showing error message: Es trat ein Problem auf.
> W/FxAccounts( 7752): <FxAccountClientRemoteException 0 [999]: Response malformed>
> W/FxAccounts( 7752): 	at org.mozilla.gecko.background.fxa.FxAccountClient10.validateResponse(FxAccountClient10.java:329)
> W/FxAccounts( 7752): 	at org.mozilla.gecko.background.fxa.FxAccountClient10$ResourceDelegate.handleHttpResponse(FxAccountClient10.java:220)
> W/FxAccounts( 7752): 	at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:280)
> W/FxAccounts( 7752): 	at org.mozilla.gecko.sync.net.BaseResource.go(BaseResource.java:311)
> W/FxAccounts( 7752): 	at org.mozilla.gecko.sync.net.BaseResource.post(BaseResource.java:341)
> W/FxAccounts( 7752): 	at org.mozilla.gecko.sync.net.BaseResource.post(BaseResource.java:464)
> W/FxAccounts( 7752): 	at org.mozilla.gecko.background.fxa.FxAccountClient10.post(FxAccountClient10.java:286)
> W/FxAccounts( 7752): 	at org.mozilla.gecko.background.fxa.FxAccountClient20.login(FxAccountClient20.java:96)
> W/FxAccounts( 7752): 	at org.mozilla.gecko.background.fxa.FxAccountClient20.loginAndGetKeys(FxAccountClient20.java:177)
> W/FxAccounts( 7752): 	at org.mozilla.gecko.fxa.tasks.FxAccountSignInTask.doInBackground$64801092(FxAccountSignInTask.java:32)
> W/FxAccounts( 7752): 	at org.mozilla.gecko.fxa.tasks.FxAccountSignInTask.doInBackground(FxAccountSignInTask.java:17)
> W/FxAccounts( 7752): 	at android.os.AsyncTask$2.call(AsyncTask.java:288)
> W/FxAccounts( 7752): 	at java.util.concurrent.FutureTask.run(FutureTask.java:237)
> W/FxAccounts( 7752): 	at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
> W/FxAccounts( 7752): 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
> W/FxAccounts( 7752): 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
> W/FxAccounts( 7752): 	at java.lang.Thread.run(Thread.java:841)
> W/FxAccounts( 7752): Caused by: <HTTPFailureException 403 :: ([unknown error message])>
> W/FxAccounts( 7752): 	... 17 more

Comment 10

[Richard Newman [:rnewman]]

(In reply to MS from comment #9)
> I'm using "Android 4.4", "Firefox 36.0", "fxa-custom-server-addon 0.3" and
> SNI is still not working.

It's not expected to.

Comment 11

[github78]

I too spent many hours trying to understand why my Linux desktop Firefox 31.6.0 could sync with my self hosted server and why Firefox 40.0 and fxa-custom-server-addon 0.3 on Android 5.1.1 couldn't.  I ended up using wireshark to see that my Linux Firefox did SNI and that Android Firefox didn't.  Nothing in the adb log messages indicated that Firefox sync had received a certificate for the wrong host.  It just said that the certificate was bad.  When I got the first block of log messages below, the default certificate on my server was self signed and unknown to Android.  The certificate for the host that Firefox sync is configured to use is valid (checked with the default Android browser). 

In the second block of messages below, the adb messages are much clearer because (after a configuration change on my web server) the certificate for the default host on my web server is acceptable to Android.

I got Firefox sync to work by making my sync server the default host on my web server.

D/GeckoLogger(12755): Thread with tag and thread id acquiring lock: FxAccountSyncAdapter, 868 ...
D/GeckoLogger(12755): Thread with tag and thread id acquiring lock: FxAccountSyncAdapter, 868 ... ACQUIRED
I/FxAccounts(12755): fennec_fdroid :: LoginStateMachineDelegate :: handleFinal: in Married
I/FxAccounts(12755): fennec_fdroid :: AndroidFxAccount :: Moving account named like [e-mail] to state Married
I/FxAccounts(12755): fennec_fdroid :: FxAccountNotificationManager :: State Married needs no action; cancelling any existing notification.
I/FxAccounts(12755): fennec_fdroid :: LoginStateMachineDelegate :: handleMarried: in Married
E/FxAccounts(12755): fennec_fdroid :: FxAccountSyncAdapter :: Failed to get token.
E/FxAccounts(12755): javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
E/FxAccounts(12755): 	at com.android.org.conscrypt.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:322)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.connectSocket$4028dcbe(SSLSocketFactory.java:535)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:403)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:144)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:131)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:825)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:275)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.retryRequest(BaseResource.java:316)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:285)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.go(BaseResource.java:340)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.get(BaseResource.java:346)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter.syncWithAssertion$1a55e242(FxAccountSyncAdapter.java:368)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter$3.handleMarried(FxAccountSyncAdapter.java:534)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.authenticator.FxADefaultLoginStateMachineDelegate.handleFinal(FxADefaultLoginStateMachineDelegate.java:81)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.login.FxAccountLoginStateMachine.advance(FxAccountLoginStateMachine.java:78)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter.onPerformSync(FxAccountSyncAdapter.java:484)
E/FxAccounts(12755): 	at android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:259)
E/FxAccounts(12755): Caused by: java.security.cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
E/FxAccounts(12755): 	at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:318)
E/FxAccounts(12755): 	at com.android.org.conscrypt.TrustManagerImpl.checkServerTrusted(TrustManagerImpl.java:219)
E/FxAccounts(12755): 	at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:114)
E/FxAccounts(12755): 	at com.android.org.conscrypt.OpenSSLSocketImpl.verifyCertificateChain(OpenSSLSocketImpl.java:550)
E/FxAccounts(12755): 	at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method)
E/FxAccounts(12755): 	at com.android.org.conscrypt.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:318)
E/FxAccounts(12755): 	... 20 more
E/FxAccounts(12755): Caused by: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
E/FxAccounts(12755): 	... 26 more
D/GeckoLogger(12755): Thread with tag and thread id releasing lock: FxAccountSyncAdapter, 794 ...
D/GeckoLogger(12755): Thread with tag and thread id releasing lock: FxAccountSyncAdapter, 794 ... RELEASED
W/FxAccounts(12755): fennec_fdroid :: FxAccountSyncAdapter :: Global session failed.
E/FxAccounts(12755): fennec_fdroid :: FxAccountSyncAdapter :: Got exception syncing.
E/FxAccounts(12755): javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
E/FxAccounts(12755): 	at com.android.org.conscrypt.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:322)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.connectSocket$4028dcbe(SSLSocketFactory.java:535)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:403)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:144)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:131)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:825)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:275)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.retryRequest(BaseResource.java:316)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:285)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.go(BaseResource.java:340)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.get(BaseResource.java:346)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter.syncWithAssertion$1a55e242(FxAccountSyncAdapter.java:368)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter$3.handleMarried(FxAccountSyncAdapter.java:534)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.authenticator.FxADefaultLoginStateMachineDelegate.handleFinal(FxADefaultLoginStateMachineDelegate.java:81)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.login.FxAccountLoginStateMachine.advance(FxAccountLoginStateMachine.java:78)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter.onPerformSync(FxAccountSyncAdapter.java:484)
E/FxAccounts(12755): 	at android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:259)
E/FxAccounts(12755): Caused by: java.security.cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
E/FxAccounts(12755): 	at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:318)
E/FxAccounts(12755): 	at com.android.org.conscrypt.TrustManagerImpl.checkServerTrusted(TrustManagerImpl.java:219)
E/FxAccounts(12755): 	at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:114)
E/FxAccounts(12755): 	at com.android.org.conscrypt.OpenSSLSocketImpl.verifyCertificateChain(OpenSSLSocketImpl.java:550)
E/FxAccounts(12755): 	at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method)
E/FxAccounts(12755): 	at com.android.org.conscrypt.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:318)
E/FxAccounts(12755): 	... 20 more
E/FxAccounts(12755): Caused by: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
E/FxAccounts(12755): 	... 26 more
D/GeckoLogger(12755): Thread with tag and thread id releasing lock: FxAccountSyncAdapter, 868 ...
D/GeckoLogger(12755): Thread with tag and thread id releasing lock: FxAccountSyncAdapter, 868 ... NOT LOCKED
I/FxAccounts(12755): fennec_fdroid :: FxAccountSyncAdapter :: Syncing done.
D/SyncManager(18680): failed sync operation [e-mail] u0 (org.mozilla.fennec_fdroid_fxaccount), org.mozilla.fennec_fdroid.db.browser, LOCAL, currentRunTime 265022065, reason: 10067, SyncResult: stats [ numIoExceptions: 1 numUpdates: 1]

I/GeckoLogger(12755): fennec_fdroid :: FxAccountStatusFragment :: Got sync started message; refreshing.
I/FxAccounts(12755): fennec_fdroid :: FxAccountSyncAdapter :: Syncing FxAccount account named like [e-mail] for authority org.mozilla.fennec_fdroid.db.browser with instance org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter@9d5cab.
I/FxAccounts(12755): fennec_fdroid :: FxAccountSyncAdapter :: Account last synced at: 1440874686426
I/FxAccounts(12755): fennec_fdroid :: FirefoxAccounts :: Sync hints; scheduling now: true; ignoring local rate limit: true; ignoring remote server backoff: true.
D/GeckoLogger(12755): Thread with tag and thread id acquiring lock: FxAccountSyncAdapter, 882 ...
D/GeckoLogger(12755): Thread with tag and thread id acquiring lock: FxAccountSyncAdapter, 882 ... ACQUIRED
I/FxAccounts(12755): fennec_fdroid :: LoginStateMachineDelegate :: handleFinal: in Married
I/FxAccounts(12755): fennec_fdroid :: AndroidFxAccount :: Moving account named like [e-mail] to state Married
I/FxAccounts(12755): fennec_fdroid :: FxAccountNotificationManager :: State Married needs no action; cancelling any existing notification.
I/FxAccounts(12755): fennec_fdroid :: LoginStateMachineDelegate :: handleMarried: in Married
E/FxAccounts(12755): fennec_fdroid :: FxAccountSyncAdapter :: Failed to get token.
E/FxAccounts(12755): javax.net.ssl.SSLException: hostname in certificate didn't match: <iwsync.pook.it> != <self.pook.it> OR <self.pook.it> OR <self.local>
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:236)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.BrowserCompatHostnameVerifier.verify(BrowserCompatHostnameVerifier.java:54)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:157)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:138)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.verifyHostname(SSLSocketFactory.java:561)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.connectSocket$4028dcbe(SSLSocketFactory.java:536)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:403)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:144)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:131)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:825)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:275)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.retryRequest(BaseResource.java:316)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:285)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.go(BaseResource.java:340)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.get(BaseResource.java:346)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter.syncWithAssertion$1a55e242(FxAccountSyncAdapter.java:368)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter$3.handleMarried(FxAccountSyncAdapter.java:534)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.authenticator.FxADefaultLoginStateMachineDelegate.handleFinal(FxADefaultLoginStateMachineDelegate.java:81)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.login.FxAccountLoginStateMachine.advance(FxAccountLoginStateMachine.java:78)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter.onPerformSync(FxAccountSyncAdapter.java:484)
E/FxAccounts(12755): 	at android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:259)
D/GeckoLogger(12755): Thread with tag and thread id releasing lock: FxAccountSyncAdapter, 794 ...
D/GeckoLogger(12755): Thread with tag and thread id releasing lock: FxAccountSyncAdapter, 794 ... RELEASED
W/FxAccounts(12755): fennec_fdroid :: FxAccountSyncAdapter :: Global session failed.
E/FxAccounts(12755): fennec_fdroid :: FxAccountSyncAdapter :: Got exception syncing.
E/FxAccounts(12755): javax.net.ssl.SSLException: hostname in certificate didn't match: <iwsync.pook.it> != <self.pook.it> OR <self.pook.it> OR <self.local>
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:236)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.BrowserCompatHostnameVerifier.verify(BrowserCompatHostnameVerifier.java:54)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:157)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:138)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.verifyHostname(SSLSocketFactory.java:561)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.connectSocket$4028dcbe(SSLSocketFactory.java:536)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:403)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:144)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:131)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:825)
E/FxAccounts(12755): 	at ch.boye.httpclientandroidlib.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:275)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.retryRequest(BaseResource.java:316)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:285)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.go(BaseResource.java:340)
E/FxAccounts(12755): 	at org.mozilla.gecko.sync.net.BaseResource.get(BaseResource.java:346)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter.syncWithAssertion$1a55e242(FxAccountSyncAdapter.java:368)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter$3.handleMarried(FxAccountSyncAdapter.java:534)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.authenticator.FxADefaultLoginStateMachineDelegate.handleFinal(FxADefaultLoginStateMachineDelegate.java:81)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.login.FxAccountLoginStateMachine.advance(FxAccountLoginStateMachine.java:78)
E/FxAccounts(12755): 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter.onPerformSync(FxAccountSyncAdapter.java:484)
E/FxAccounts(12755): 	at android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:259)
D/GeckoLogger(12755): Thread with tag and thread id releasing lock: FxAccountSyncAdapter, 882 ...
D/GeckoLogger(12755): Thread with tag and thread id releasing lock: FxAccountSyncAdapter, 882 ... NOT LOCKED
I/FxAccounts(12755): fennec_fdroid :: FxAccountSyncAdapter :: Syncing done.
I/GeckoLogger(12755): fennec_fdroid :: FxAccountStatusFragment :: Got sync finished message; refreshing.
D/SyncManager(18680): failed sync operation [e-mail] u0 (org.mozilla.fennec_fdroid_fxaccount), org.mozilla.fennec_fdroid.db.browser, USER, currentRunTime 266003652, reason: 10067, SyncResult: stats [ numIoExceptions: 1 numUpdates: 1]

Comment 12

[Nick Alexander :nalexander [he/him]]

(In reply to github78 from comment #11)
> I too spent many hours trying to understand why my Linux desktop Firefox
> 31.6.0 could sync with my self hosted server and why Firefox 40.0 and
> fxa-custom-server-addon 0.3 on Android 5.1.1 couldn't.  I ended up using
> wireshark to see that my Linux Firefox did SNI and that Android Firefox
> didn't.  Nothing in the adb log messages indicated that Firefox sync had
> received a certificate for the wrong host.  It just said that the
> certificate was bad.  When I got the first block of log messages below, the
> default certificate on my server was self signed and unknown to Android. 
> The certificate for the host that Firefox sync is configured to use is valid
> (checked with the default Android browser). 
> 
> In the second block of messages below, the adb messages are much clearer
> because (after a configuration change on my web server) the certificate for
> the default host on my web server is acceptable to Android.

All you say is correct, but is there a request here?  I suppose we could try to add commentary to the error message we get from the httpclientlib stack, but it would be rather generic ("If you're using SNI, ...").

The lack of SNI in the httpclientlib stack we use is known and not something we (Fennec team) will ever address.

Comment 13

[github78]

I added the error message for Google searches and yes it would be great if you did add something to the httpclientlib error message: "If your http server relies on TLS SNI (Server Name Indication) to send the correct certificate then Android Firefox sync will not work".

Comment 14

[Richard Newman [:rnewman]]

Now that we use HttpClient (in the form of httpclientandroidlib) in a bunch of places in Fennec (see Bug 1169421), this doesn't just affect Sync.

Comment 15

[Robert Buchholz]

Is there a way to merge the upstream fix for SNI support? I encountered this bug today and was rather surprised to see SNI not working (especially in a browser).

Comment 16

[Richard Newman [:rnewman]]

There are a few ways forward:

* Upgrade our HttpClient implementation to a more recent one that supports SNI. Given that they seem to break API compatibility on pretty much every release, last time I tried this (<https://github.com/mozilla-services/android-sync/pull/554>) it was a huge pain in the ass, which is why I landed something simpler for Bug 1061273.

* Try to merge or reimplement the fix for <https://issues.apache.org/jira/browse/HTTPCLIENT-1119> on top of the version of HttpClient that we use. This sounds suspiciously like "maintain a fork of HttpClient", so I don't want to go there.

* Implement a hacky workaround as mentioned in Comment 6. This might not work with all Android versions that Firefox supports, and it's a hack.


A patch from a motivated and skilled contributor would be welcome, but this isn't a trivial fix and it involves a fair amount of testing, so I wouldn't hold your breath.

Comment 17

[Richard Newman [:rnewman]]

One way of assessing the priority of this work is to determine its impact in places where workarounds aren't possible. For Mozilla-hosted properties we can avoid SNI, and self-hosted Sync setups are (a) rare and (b) can be reconfigured to work around this limitation.

That leaves favicon fetching. If we could measure the rate of favicon fetch failures due to SNI -- or get that metric indirectly, e.g., via measuring the rate of SNI usage in desktop favicon fetches -- and it's high, then we have a motivation to do this work.

Comment 18

[Robert Buchholz]

Richard, thanks for your response. I didn't realize apache's httpclient had such a horrible record of API changes. What is the problem with using Firefox's own HTTP implementation to perform these tasks?

Regarding this specific issue: I worked around it by using a custom port for the sync-server. Since it's not a user-facing service, it doesn't matter much if the about:config value contains an additional ":1234". Might break synchronization in heavily firewalled networks.

Comment 19

[Richard Newman [:rnewman]]

(In reply to Robert Buchholz from comment #18)
> Richard, thanks for your response. I didn't realize apache's httpclient had
> such a horrible record of API changes. What is the problem with using
> Firefox's own HTTP implementation to perform these tasks?

Sync, FxA, favicon fetching, the updater, and various other parts of Firefox on Android are implemented in Android-native Java. They're services that are invoked by the OS, run early in startup, etc.; we can't rely on Gecko running.

Necko isn't well-separated, needs a profile, and a bunch of other things that make it inconvenient to use directly from Java services. See Bug 507641 for extensive discussion.

Another way of looking at this: on Android we embed Gecko to render web pages, and that's about it.

Comment 20

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Richard Newman [:rnewman] from comment #16)
> There are a few ways forward:
> 
> * Upgrade our HttpClient implementation to a more recent one that supports
> SNI. Given that they seem to break API compatibility on pretty much every
> release, last time I tried this
> (<https://github.com/mozilla-services/android-sync/pull/554>) it was a huge
> pain in the ass, which is why I landed something simpler for Bug 1061273.

Likely non-trivial work, but maybe something we should consider.

> * Try to merge or reimplement the fix for
> <https://issues.apache.org/jira/browse/HTTPCLIENT-1119> on top of the
> version of HttpClient that we use. This sounds suspiciously like "maintain a
> fork of HttpClient", so I don't want to go there.

This could be a decent quick fix. Keeping an outdated version of the library in our tree is already like maintaining a fork.

Comment 22

[:Margaret Leibovic]

Let's be optimistic and try to get a fix for this. Bug 1250997 is what makes me want to prioritize this.

Comment 23

[Sebastian Kaspari (:sebastian; :pocmo)]

I'm interested in picking this up.

Comment 24

[Nick Alexander :nalexander [he/him]]

Based on sebastian's patch for Bug 1254089, I wonder if we should just /replace/ ch.boye.* with simpler equivalents.  I feel like a case-by-case transition wouldn't be that tricky.  I haven't really considered all that's involved here, but I'd want significant testing to ensure we work well with Mozilla Services configurations (and partner endpoints).

Comment 25

[Sebastian Kaspari (:sebastian; :pocmo)]

Not going to happen in 48.

Comment 27

[CharlyDelta]

I am also suffering because of this issue and I would really appreciate if this would get more attention. Sadly, I cannot change to other IPs or open more ports. 

I did not expect Firefox (at least the sync-part) to be unable to handle SNI, I assumed that it is a well established standard nowadays.

Comment 28

[MS]

Firefox 50.1.0, tried to sync with custom sync server.

> 01-23 12:46:22.764 31941 32348 E FxAccounts: javax.net.ssl.SSLException: hostname in certificate didn't match: <fxsync.mydomain.de> != <mydomain.de> OR <mydomain.de> OR <www.mydomain.de>
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:236)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.conn.ssl.BrowserCompatHostnameVerifier.verify(BrowserCompatHostnameVerifier.java:54)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:157)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:138)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.verifyHostname(SSLSocketFactory.java:561)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.connectSocket$4028dcbe(SSLSocketFactory.java:536)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:403)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:174)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:144)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:131)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:825)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at ch.boye.httpclientandroidlib.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:308)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at org.mozilla.gecko.sync.net.BaseResource.retryRequest(BaseResource.java:349)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at org.mozilla.gecko.sync.net.BaseResource.execute(BaseResource.java:318)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at org.mozilla.gecko.sync.net.BaseResource.go(BaseResource.java:373)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at org.mozilla.gecko.sync.net.BaseResource.get(BaseResource.java:379)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter.syncWithAssertion$1a55e242(FxAccountSyncAdapter.java:371)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter$3.handleMarried(FxAccountSyncAdapter.java:539)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at org.mozilla.gecko.fxa.authenticator.FxADefaultLoginStateMachineDelegate.handleFinal(FxADefaultLoginStateMachineDelegate.java:81)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at org.mozilla.gecko.fxa.login.FxAccountLoginStateMachine.advance(FxAccountLoginStateMachine.java:78)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter.onPerformSync(FxAccountSyncAdapter.java:489)
> 01-23 12:46:22.764 31941 32348 E FxAccounts: 	at android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:272)

History:
Until now I was able to resolve that problem by having a self-signed certificate, importing the root certificate in android and provide a wildcard certificate serverside (therefore covering fxsync.mydomain.de).
Recently, I switchted to LetsEncrypt and therefore cannot provide a wildcard certificate as fallback anymore.
Following that, I ran in the SNI problem once again.

Comment 29

[CharlyDelta]

I will switch to LetsEncrypt soon, too. Didn't think of the fact that they don't provide wildcard-certs... You may have the chance to open the ports so that you can use them instead of specific domains - this would work I think. I don't have the option so I could try that unfortunately

Comment 30

[lars]

Hello,

i try to sync my android deive with my own Syncserver. Windows Desktop works good, but Android fails. (latest Playstore Version and Beta)

I have this log-entry:

`
02-26 17:58:55.766 I/FxAccounts(14952): firefox :: FxAccountSyncAdapter :: Syncing FxAccount account named like [e-mail] for authority org.mozilla.firefox.db.browser with instance org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter@ba389c0.
02-26 17:58:55.769 I/FxAccounts(14952): firefox :: FxAccountSyncAdapter :: Account last synced at: -1
02-26 17:58:55.770 I/FxAccounts(14952): firefox :: FirefoxAccounts :: Sync options -- scheduling now: true
02-26 17:58:55.774 I/FxAccounts(14952): firefox :: FxAccountSyncAdapter :: Forced sync (rate): overruling remaining backoff of 89774ms.
02-26 17:58:55.794 D/GeckoLogger(14952): Thread with tag and thread id acquiring lock: FxAccountSyncAdapter, 1816 ...
02-26 17:58:55.794 D/GeckoLogger(14952): Thread with tag and thread id acquiring lock: FxAccountSyncAdapter, 1816 ... ACQUIRED
02-26 17:58:55.797 I/FxAccounts(14952): firefox :: LoginStateMachineDelegate :: handleTransition: LogMessage('Upgraded Firefox clients might know what to do here.') to Doghouse
02-26 17:58:55.797 I/FxAccounts(14952): firefox :: LoginStateMachineDelegate :: handleFinal: in Doghouse
02-26 17:58:55.798 I/FxAccounts(14952): firefox :: AndroidFxAccount :: Moving account named like [e-mail] to state Doghouse
02-26 17:58:55.817 I/FxAccounts(14952): firefox :: FxAccountNotificationManager :: State Doghouse needs action; offering notification with title: Sync ist nicht verbunden
02-26 17:58:55.829 I/FxAccounts(14952): firefox :: LoginStateMachineDelegate :: handleNotMarried: in Doghouse
02-26 17:58:55.829 I/FxAccounts(14952): firefox :: FxAccountSchedulePolicy :: Scheduling periodic sync for 86400.
02-26 17:58:55.847 I/Xposed  (4638): fsbi >> icon >> notify >> ic_status_logo            > id: 107769471 >> StatusBarIcon(icon=Icon(typ=RESOURCE pkg=org.mozilla.firefox id=0x7f0200e8) visible user=0 ) >> StatusBarIconView(slot=org.mozilla.firefox/0xfa8c2859 icon=StatusBarIcon(icon=Icon(typ=RESOURCE pkg=org.mozilla.firefox id=0x7f0200e8) visible user=0 ) notification=Notification(pri=0 contentView=org.mozilla.firefox/0x1090087 vibrate=null sound=null defaults=0x0 flags=0x10 color=0x00000000 vis=PRIVATE))
02-26 17:58:55.857 W/FxAccounts(14952): firefox :: FxAccountSyncAdapter :: Cannot sync from state: Doghouse
02-26 17:58:55.858 D/GeckoLogger(14952): Thread with tag and thread id releasing lock: FxAccountSyncAdapter, 1816 ...
02-26 17:58:55.858 D/GeckoLogger(14952): Thread with tag and thread id releasing lock: FxAccountSyncAdapter, 1816 ... RELEASED
02-26 17:58:55.858 I/FxAccounts(14952): firefox :: FxAccountSyncAdapter :: Syncing done.
02-26 17:58:55.869 D/SyncManager(3428): failed sync operation [e-mail] u0 (org.mozilla.firefox_fxaccount), org.mozilla.firefox.db.browser, SERVER, currentRunTime 27133344, EXPEDITED, reason: 10182, SyncResult: stats [ numIoExceptions: 1 numUpdates: 1]
02-26 17:58:55.870 I/FxAccounts(14952): firefox :: FxAccountStatusFragment :: Got sync finished message; refreshing.
02-26 17:58:55.871 I/FxAccounts(14952): firefox :: FxAccountStatusFragment :: AvatarURI is empty, skipping profile image fetch.
02-26 17:58:55.981 W/ctxmgr  (5730): [AclManager]No 2 for (accnt=account#-149203989#, com.google.android.gms(10027):UserLocationProducer, vrsn=10298000, 0, 3pPkg = null ,  3pMdlId = null). Was: 2 for 1, account#-149203989#
02-26 17:58:56.673 I/FxAccounts(14952): firefox :: FirefoxAccounts :: Requesting sync.
02-26 17:58:56.673 I/FxAccounts(14952): firefox :: FirefoxAccounts :: Sync options -- scheduling now: true
02-26 17:58:56.685 D/audio_hw_primary(871): out_set_parameters: enter: usecase(1: low-latency-playback) kvpairs: routing=2
02-26 17:58:56.740 I/FxAccounts(14952): firefox :: FxAccountStatusFragment :: Got sync started message; refreshing.
02-26 17:58:56.741 I/FxAccounts(14952): firefox :: FxAccountStatusFragment :: AvatarURI is empty, skipping profile image fetch.
02-26 17:58:56.785 I/FxAccounts(14952): firefox :: FxAccountSyncAdapter :: Syncing FxAccount account named like [e-mail] for authority org.mozilla.firefox.db.browser with instance org.mozilla.gecko.fxa.sync.FxAccountSyncAdapter@ba389c0.
02-26 17:58:56.788 I/FxAccounts(14952): firefox :: FxAccountSyncAdapter :: Account last synced at: -1
02-26 17:58:56.788 I/FxAccounts(14952): firefox :: FirefoxAccounts :: Sync options -- scheduling now: true
02-26 17:58:56.794 I/FxAccounts(14952): firefox :: FxAccountSyncAdapter :: Forced sync (rate): overruling remaining backoff of 88980ms.
02-26 17:58:56.815 D/GeckoLogger(14952): Thread with tag and thread id acquiring lock: FxAccountSyncAdapter, 1817 ...
02-26 17:58:56.815 D/GeckoLogger(14952): Thread with tag and thread id acquiring lock: FxAccountSyncAdapter, 1817 ... ACQUIRED
02-26 17:58:56.817 I/FxAccounts(14952): firefox :: LoginStateMachineDelegate :: handleTransition: LogMessage('Upgraded Firefox clients might know what to do here.') to Doghouse
02-26 17:58:56.817 I/FxAccounts(14952): firefox :: LoginStateMachineDelegate :: handleFinal: in Doghouse
02-26 17:58:56.817 I/FxAccounts(14952): firefox :: AndroidFxAccount :: Moving account named like [e-mail] to state Doghouse
02-26 17:58:56.833 I/FxAccounts(14952): firefox :: FxAccountNotificationManager :: State Doghouse needs action; offering notification with title: Sync ist nicht verbunden
02-26 17:58:56.840 I/FxAccounts(14952): firefox :: LoginStateMachineDelegate :: handleNotMarried: in Doghouse
02-26 17:58:56.840 I/FxAccounts(14952): firefox :: FxAccountSchedulePolicy :: Scheduling periodic sync for 86400.`

Comment 31

[Nick Alexander :nalexander [he/him]]

(In reply to lars from comment #30)
> Hello,
> 
> i try to sync my android deive with my own Syncserver. Windows Desktop works
> good, but Android fails. (latest Playstore Version and Beta)
> 
> I have this log-entry:

This doesn't look like an SNI issue...

> 02-26 17:58:56.817 I/FxAccounts(14952): firefox :: LoginStateMachineDelegate
> :: handleTransition: LogMessage('Upgraded Firefox clients might know what to
> do here.') to Doghouse

this looks like a configuration issue.  It looks like you have incorrect URLs for the various custom server things you need to configure.  You can check by opening the menu in Firefox, tapping Settings, and then tapping your Account.  You should get a screen with various options and diagnostics; if you're running a custom server, the URLs will be printed for you.  If I recall correctly, you can't edit the URLs (sorry!), so you'll probably need to delete the Android Account and start again (with the correct URLs).

This is documented at https://docs.services.mozilla.com/howtos/run-sync-1.5.html and https://docs.services.mozilla.com/howtos/run-fxa.html.

A screenshot of the Firefox > Settings > Account screen showing the URLs (if they're not private) might help.  Feel free to delete/blur your email -- it's translated to [e-mail] in the logs above to maintain your privacy.

Comment 32

[MS]

The still exists in Firefox for Android 55.0.2.

Comment 34

[[e-mail]]

As mentioned in the above duplicate bug I also encountered the SNI non-support issue when I switched to a Let's Encrypt cert and spent a few hours trying to figure out what was going on.

As a software developer I completely understand that this is not a trivial fix and that it would touch lots of parts of a complex code base and I really appreciate the thoughts and different options you Mozillans have put into this issue and all the other amazing work you do - thank you!

For whatever it's worth I agree with @mfinkle above when he says that porting the Apache patch into the version of HTTPClient you guys already have in your repo seems like a quick fix for a 5 year old bug, which could later be replaced with some of the more robust fixes discussed. Here's the patch again for reference (84 lines added to two files): https://issues.apache.org/jira/browse/HTTPCLIENT-1119

Thanks again for your amazing work - I looove this browser!