1391664 - Validate the SHIELD study addon

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P1)

Description

[Alessio Placitelli [:Dexter]]

Once bug 1383757 is fixed and the SHIELD study addon for RAPPOR is complete, we should take time to verify that the data is working as expected.

One possible approach for this:

- Write a scratchpad script to call into the SHIELD study functions.
- Make it draw samples from a set distribution of homepage values.
- Serialize the RAPPOR output/pings to disk.
- Feed the data into the simulator.

Comment 2

[Alessio Placitelli [:Dexter]]

Attachment: Addon to Simulator PR

Comment 3

[Alejandro Rodriguez Salamanca [:alejandro]]

RAPPOR is an algorithm with a strong random component that make it difficult to test the correctness of the implementation using unit testing. For this reason, some simulations must be run. 

To validate the implementation, the addon can be set to work in simulation mode. This mode uses the RAPPOR simulator used in bug 1379195[1]. In the configuration file Config.jsm, the flag isSimulation must be set to true along with the location of the RAPPOR simulator in the variable rapporSimulatorPath. The instructions of how to run a simulations are located in [2].

The results has been compared with the ones obtained in [1], showing that our implementation is correct and it is possible to recover the data encoded by RAPPOR.

[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1379195
[2]: https://github.com/Alexrs95/shield-study-rappor#simulations

Comment 4

[Alessio Placitelli [:Dexter]]

Thanks Alejandro!

Closing this bug as the addon to simulator code was merged and a simulation was run to make sure it worked.