1462105 - Federated Learning simulations

Status: RESOLVED FIXED

(Data Platform and Tools :: General, enhancement, P1)

Description

[Florian Hartmann]

To be able to quickly iterate on a prototype implementation of Federated Learning, a simulation should be created.
The data is artificially assigned to simulated users here.
Because no real network connection is required and we do not need to wait until users generate data, this will allow us to iterate much quicker and to test out ideas more easily.

The simulation code should be abstract enough to support various datasets.
Once we see that the idea works, the server part of the simulation should be used to build the real server-side part of the implementation.

Comment 1

[Florian Hartmann]

The current experimental code can be found on GitHub [1, 2].
A fake dataset based on the frecency algorithm [3] was created.
This is mostly to prototype the optimization algorithm for a pointwise ranking problem.
The federated learning part is not implemented yet.


[1] https://github.com/florian/federated-learning
[2] https://github.com/florian/federated-learning/blob/7bd0e41975c04384c0ab9f1cb4f9823a98948445/Federated-Learning.ipynb
[3] https://developer.mozilla.org/en-US/docs/Mozilla/Tech/Places/Frecency_algorithm

Comment 2

[Florian Hartmann]

The simulation code is now largely refactored and some concepts were abstracted to allow for more general simulations.
The next steps are making the simulation more realistic (e.g. by using different distributions for users) and to experiment with discrete optimization.
If we can optimize frecency scores in a discrete way, this will make the actual implementation easier and we could optimize time buckets which have to be discrete (number of days since the last visit).