1238218 - Some experiences regarding building, confiuration and installation of syncserver and FxA components

Status: RESOLVED WONTFIX

(Cloud Services :: General, defect)

Description

[Kay Peters]

Attachment: Configuration files and more

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0
Build ID: 2015122500

Steps to reproduce:

Hello Ryan!

As promised in IRC, I wrote down some documentation hints as seen by a non FxA developer who tried to compile / configure / deploy the current sync / FxA stack.

I don't claim that it is comprehensive, even correct at all . Maybe it's just chance that it works for me!

These additional documentation hints are not meant to replace the existing documentation - it is more of an extension or an example which should be extended or even fixed.


First of all: Why did I want to install a private sync? What was my motivation?

My goal was to keep all data private. This means the sync data per se including all of the other necessary data like account data and the always ongoing data flow during daily use should be held in LAN.



Actual results:

Therefore, all necessary components must be run locally. This minimally includes not just an auth-server and content-server, but some more servers.

The necessary stack is:

- syncserver-1.5.2
- content-server-0.52
- auth-server-1.51.1

- profile-server-0.53.0
- oauth-server-0.53.0
- auth-db-mysql-0.51
- browserid-verifier-0.3.0


Ok - if you just want to run sync w/o ever using the account management functionality, you don't need the last four pieces of the stack. But I don't think, it is realistic, that you never ever will have the need to remove an account or maybe just want to change a password.

I deployed the servers in a VM (-> no docker).



Which ports / addresses are minimally used?
===========================================

I created this table of some necessary ports provided by the servers:

Port	Who 			publ. IF 	DNS name
--------------------------------------------------------------------
9000	fxa-auth-server 	yes 		fxaccount-api.my.tld
3030	fxa-content-server 	yes 		fxaccount.my.tld
8800	fxa-auth-db-mysql 	no
9010	fxa-oauth-server 	yes 		fxaoauth.my.tld
5000	syncserver 		yes 		fxsync.my.tld
5050	browserid-verifier	no
1111	fxa-profile-server	yes		fxaprofile.my.tld


The necessary configuration for Firefox (43.0.3) is part of the attached config.tar.gz (user.js).



Use of self signed certificates
===============================

One big problem I faced, was the fact, that I'm using self signed certificates with an own CA. It would be good to have a big fat notice, that the own ca.crt must be put to syncserver/local/lib/python2.7/site-packages/requests/cacert.pem to get it working.

Another self signed certificate problem: the browserid-verifier connects via public(!) IP to fxaccount-api.my.tld and therefore via https - and has to check as client the ca.crt - which it can't do because it doesn't know the CA. Therefore this patch (or better: workaround) was necessary (or is there a better possibility?):

--- a/browserid-verifier/node_modules/browserid-local-verify/lib/lookup.js     2016-01-08 07:44:45.387239322 +0100
+++ b/browserid-verifier/node_modules/browserid-local-verify/lib/lookup.js    2016-01-06 00:52:05.132198569 +0100
@@ -130,7 +130,8 @@
       host: host,
       port: port,
       path: pathToWellKnown,
-      rejectUnauthorized: !args.insecureSSL,
+      // rejectUnauthorized: !args.insecureSSL,
+      rejectUnauthorized: false,
       agent: false
     }, function(res) {
       var body = "";



How to create the necessary certs:
===========================
The attached config.tar.gz contains the subdirectory ca/ with a small script, which creates a CA and a self signed SAN certificate. You have to configure cert.conf, ext.conf and mkca.sh before you can use it.



Providing public URLs
================
I provide the public interfaces via reverse proxy. Two possibilities (each one example):

1. apache (2.4.6):
-------------------------
<VirtualHost [public ip address]:443>
ServerName fxaccount-api.my.tld
DocumentRoot /dev/null

SSLEngine on
SSLStrictSNIVHostCheck on
SSLCertificateFile      /etc/ssl-fxa/fxa.crt
SSLCertificateKeyFile   /etc/ssl-fxa/fxa.key

CustomLog /var/log/apache2/fxaccount-api.my.tld-access.log combined
ErrorLog  /var/log/apache2/fxaccount-api.my.tld-error.log

ProxyPreserveHost On
ProxyRequests off
ProxyPass / http://127.0.0.1:9000/
ProxyPassReverse / http://127.0.0.1:9000/

</VirtualHost>
-------------------------

The syncserver via wsgi:
-------------------------
<IfDefine SSL>
<IfDefine !NOSSL>

<VirtualHost [IP Address]:443>
  ServerAdmin [e-mail]

  SSLEngine on
  #  SSL protocols
  #  Supporting TLS only is adequate nowadays
  SSLProtocol all -SSLv2

  #   SSL Cipher Suite:
  #   List the ciphers that the client is permitted to negotiate.
  #   See the mod_ssl documentation for a complete list.
  SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5

  SSLCertificateFile    /etc/ssl-fxa/fxa.crt
  SSLCertificateKeyFile /etc/ssl-fxa/fxa.key

  <Directory /home/sync/syncserver-1.5.2>
     Order deny,allow
     Allow from all
  </Directory>

  ServerName fxsync.my.tld
  DocumentRoot /home/sync/syncserver-1.5.2

  HostnameLookups Off

  WSGIProcessGroup sync
  WSGIDaemonProcess sync user=sync group=sync processes=2 threads=25 python-path=/home/sync/syncserver-1.5.2/local/lib/python2.7/site-packages/
  WSGIPassAuthorization On
  WSGIScriptAlias / /home/sync/syncserver-1.5.2/syncserver.wsgi

  CustomLog /var/log/apache2/fxsync.my.tld-access.log combined
  # ErrorLog  /var/log/apache2/fxsync.my.tld-error.log
</VirtualHost>
-------------------------

2. nginx
-------------------------
    server {
    listen [public IP]:443 ssl;

    server_name fxaccount-api.my.tld;

    ssl_certificate /etc/ssl-fxa/fxa.crt;
    ssl_certificate_key /etc/ssl-fxa/fxa.key;

    location / {
    proxy_pass http://127.0.0.1:9000/;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_redirect off;
    proxy_read_timeout 120;
    proxy_connect_timeout 10;
    }
}
-------------------------

syncserver via wsgi:
-------------------------
    server {
    listen [IP address]:443 ssl;
    server_name fxsync.my.tld;

    ssl_certificate /etc/ssl-fxa/fxa.crt;
    ssl_certificate_key /etc/ssl-fxa/fxa.key;

    location / {
    proxy_pass http://127.0.0.1:5001/;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_redirect off;
    proxy_read_timeout 120;
    proxy_connect_timeout 10;
    }
}


At syncserver, I wasn't able to distinguish between access.log and error.log in apache, because all of the normal messages go to error.log to - therefore I switched off error.log for the syncserver.



Configuration of the servers
============================

A really hard bit of work  was the configuration each of the servers - I'm pretty sure I didn't do the best possible in this case. I attached all the config files as I use them.



Starting the servers
====================

The start of each server directly depends on the configuration. I realized it w/ systemd. These are the service-files for each server in /etc/systemd/system:


fxasync.service
---------------
[Unit]
Description=Firefox Sync 1.5 Sync Server
After=network.target

[Service]
WorkingDirectory=/home/sync/syncserver-1.5.2
ExecStart=/usr/bin/make serve
Type=simple
User=sync
TimeoutStopSec=190

[Install]
WantedBy=multi-user.target
-------------------


fxaprofile.service
------------------
[Unit]
Description=Firefox Account Profile Server
After=network.target fxadb.service mysql.service

[Service]
WorkingDirectory=/home/sync/fxa-profile-server
Environment=HOST=127.0.0.1
Environment=HTTPDDB_URL=127.0.0.1
ExecStart=/usr/bin/npm start
Type=simple
User=sync
TimeoutStopSec=190

[Install]
WantedBy=multi-user.target
-------------------


fxaoauth.service
-------------------
[Unit]
Description=Firefox oauth Server
After=network.target fxadb.service mysql.service

[Service]
WorkingDirectory=/home/sync/fxa-oauth-server-0.53.0
Environment=HOST=127.0.0.1
Environment=HTTPDDB_URL=127.0.0.1
Environment=NODE_ENV=prod
ExecStart=/usr/bin/npm start
Type=simple
User=sync
TimeoutStopSec=190

[Install]
WantedBy=multi-user.target
--------------------


fxadb.service
--------------------
[Unit]
Description=Firefox Account DB Server
After=network.target mysql.service

[Service]
WorkingDirectory=/home/sync/fxa-auth-db-mysql-0.51
Environment=HOST=127.0.0.1
Environment=NODE_ENV=prod
ExecStart=/usr/bin/npm start
Type=simple
User=sync
TimeoutStopSec=190

[Install]
WantedBy=multi-user.target
---------------------


fxacontent.service
---------------------
[Unit]
Description=Firefox Account Content Server
After=network.target mysql.service fxadb.service fxaauth.service

[Service]
WorkingDirectory=/home/sync/fxa-content-server-0.52
ExecStart=/usr/bin/npm start
Type=simple
User=sync
TimeoutStopSec=190

[Install]
WantedBy=multi-user.target
-----------------------


fxabidverifier.service
-----------------------
[Unit]
Description=Firefox Account Browser ID verifier
After=network.target fxadb.service mysql.service

[Service]
WorkingDirectory=/home/sync/browserid-verifier
Environment=HOST=127.0.0.1
Environment=HTTPDDB_URL=127.0.0.1
ExecStart=/usr/bin/npm start
Type=simple
User=sync
TimeoutStopSec=190

[Install]
WantedBy=multi-user.target
-------------------------


fxaauth.service
-------------------------
[Unit]
Description=Firefox Account Authentication Server
After=network.target fxadb.service mysql.service

[Service]
WorkingDirectory=/home/sync/fxa-auth-server-1.51.1
Environment=HOST=127.0.0.1
Environment=HTTPDDB_URL=127.0.0.1
EnvironMent=LOG_LEVEL=warn
ExecStart=/usr/bin/npm run start-mysql
Type=simple
User=sync
TimeoutStopSec=190

[Install]
WantedBy=multi-user.target
---------------------------


Installation of the software / platform issues
==============================================

A few words to nodejs and npm (on openSUSE 13.1). Finally I used nodejs 5.3.0 and npm 5.3.0. The behavior of npm is, let me say, somewhat "special". If it doesn't work the first time, it will work the nth time. Never give up trying! And you need a really fast machine.

But I never (except one time) was able to compile bigint - but suddenly somewhere down the road the module went through without an error (don't know any more which component required it).

Installing bigint normally always produces this error:
> bigint@0.4.2 install /home/sync/node_modules/bigint
> node-gyp configure build

make: Entering directory `/home/sync/node_modules/bigint/build'
  CXX(target) Release/obj.target/bigint/bigint.o
../bigint.cc:57:27: error: expected class-name before ‘{’ token
 class BigInt : ObjectWrap {
                           ^
../bigint.cc:74:34: error: ‘Arguments’ does not name a type
   static Handle<Value> New(const Arguments& args);
                                  ^
../bigint.cc:74:45: error: ISO C++ forbids declaration of ‘args’ with no
type [-fpermissive]
   static Handle<Value> New(const Arguments& args);
....

Do you know how to fix this problem by chance?



If you are behind a SSL intercepting proxy, you will get a problem with bower.herokuapp.com, which doesn't respect the system provided own CA during installation. I temporary added it to https_exclusions as workaround.

Comment 1

[Chris Kolosiwsky [:ckolos] (ckolos has left the building)]

WONTFIX'ing for cleanup. Re-open if this is still (somehow) an issue after all this time...

Comment 2

[Ryan Kelly [:rfkelly]]

Hello Ryan!

Oof, I am disappointed to discover (some 4 years later!) that this bug was filed and I didn't see or respond to it :-(

Still, thank you for taking the time to write this up!