Closed Bug 999628 Opened 10 years ago Closed 9 years ago

[Metrics Story] Estimate of # people hosting their own sync server

Categories

(Cloud Services :: Operations: Metrics/Monitoring, task)

x86
macOS
task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: kparlante, Unassigned)

References

Details

(Whiteboard: [qa+])

User story:

As a dev/product lead, I want to know how many people are hosting their own sync servers, so that I know how much to invest in self hosted options.

Acceptance criteria: - not yet defined. We don't have to have perfect count (and won't get one), but some metric that helps us estimate could help inform decisions.

:telliot has some ideas on how we might tackle this
Whiteboard: [qa+]
(Note that this is for Sync 1.1 at present. We'll be interested in 1.5 at some point.)

The single easiest way to know someone is using their own server is if their services.sync.clusterURL value is set to a non-Mozilla domain. I don't know if this is something we get in telemetry data.

It's also worth noting that any count we get from telemetry is still going to be underreporting. The subset as a whole are very privacy-conscious, which makes them less likely to have opted into any data gathering. 

Any number we're likely to get here is going to be educated guesswork. But, even knowing whether we're dealing in 10s, 100s or 1000s would be valuable.
I think the first one might be a commentary on how hard it currently is. Different ips hitting that might be an interesting proxy for interest.

The problem is that we're interested in total overall installs, and many of those will entirely predate the existence of docs.services. Things are further complicated due to the existence of the minimal server (http://tobyelliott.wordpress.com/2011/03/25/updating-and-deprecating-the-weave-minimal-server/). For a long time, possibly even now, this was our most popular server, because it's super-easy to install.
I don't know how reliable the download counts are, but the third-party "weave minimal server" had about 1500 downloads in the last month:

  https://pypi.python.org/pypi/weave-minimal/

This is not our official server, just an API compatible re-implementation done by a community member.
Wow.

That's an order of magnitude higher than my guess would have been, which suggests that there are more installs out there than we think.
It's also higher than the download counts for some of my smaller pypi-hosted projects - which suggests it's actual installs and not just e.g. mirroring software that downloads all the things.
Depends on: 1014957
A graph of unique rps hitting the verifier per day here: https://heka.fxa.us-west-2.prod.mozaws.net/#sandboxes/FxaBrowserIdRelyingParties/outputs/FxaBrowserIdRelyingParties.RelyingParties.cbuf

We have only a few mozilla rps, the others are likely sync self-hosters who make use of the verifier.
Unassigning myself from bugs that I'm not actively working on, & needsinfo rfkelly to triage. Set component to Metrics: Pipeline for any work requested from the data pipeline team (e.g. heka filters).
Flags: needinfo?(rfkelly)
Interesting, the linked "unique rps on the verifier per dats" has been steadily climbing and is now up around 550.  This suggests that people are slowly but surely succeeding in setting up their own FxA stack.  (Adding :ckarlof because it's kinda interesting to peek at.)

IMO further estimation here will have to depend on client-side telemetry, so let's close out this bug.  Thanks Katie.
Status: NEW → RESOLVED
Closed: 9 years ago
Flags: needinfo?(rfkelly)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.