1462107 - Federated Learning server

Status: RESOLVED FIXED

(Data Platform and Tools :: General, enhancement, P1)

Description

[Florian Hartmann]

A web server that coordinates the Federated Learning process should be created.
It waits for weight updates from users and updates the model periodically.
The exact protocol that this is based on still needs to be decided on.

Comment 1

[Florian Hartmann]

Arkadiusz Komarzewski and Sunah Suh ported my Python code to Scala to make it run on the server-side.
They also added additional parts for loading the data using Spark and writing it to s3.
The resulting Spark job lives in the telemetry-streaming repository [1].

[1] https://github.com/mozilla/telemetry-streaming/pull/154

Comment 2

[Arkadiusz Komarzewski [:akomar]]

Attachment: pr 154: Bug 1462107 - Add frecency aggregator

Comment 3

[bugzilla]

:fistbump:

The server ran, did its thing, and did it right the first time! And now we have a nice codebase to work off of for future projects.