[Shield] Add-On Study: Federated Learning v2
Categories
(Shield :: Shield Study, enhancement)
Tracking
(Not tracked)
People
(Reporter: experimenter, Assigned: isegall)
Details
Attachments
(3 files, 7 obsolete files)
2.72 KB,
text/plain
|
rrayborn
:
data-review+
|
Details |
173.24 KB,
application/zip
|
Details | |
178.94 KB,
application/x-xpinstall
|
Details |
Federated Learning v2
We seek to replicate the federated learning study performed last year (PHD: https://docs.google.com/document/d/1DuZ1nQ2ve7k-98BKUKCZegtVzAxnJBEgYJYv6IUqZck/edit?ts=5b0885a3) with an updated architecture and additional probes.
More information: https://experimenter.services.mozilla.com/experiments/federated-learning-v2/
For data steward review: https://github.com/mozilla/federated-learning-v2-study-addon/blob/master/docs/TELEMETRY.md#shield-study-addon-pings-specific-to-this-study
Attached v2.1.2 for signing
Attached 2.1.3 for signing
The add-on (version 2.1.3) is now ready for signing.
Comment 6•6 years ago
|
||
Updated•6 years ago
|
Comment 7•6 years ago
|
||
Hi Ilana, can you please submit the following filled in request.md as an attachment for review?
The TELEMETRY.md itself is fantastic, and I don't see anything concerning so this should be a direct approval after the request is attached for review.
Comment 8•6 years ago
|
||
Comment 9•6 years ago
|
||
Thanks Rob - I'm going to have :fwollsen review my draft of the data-review before submitting it.
Comment 10•6 years ago
|
||
Comment 11•6 years ago
|
||
Comment 12•6 years ago
|
||
There appears to be some issue with the signing of the add-on. Attempting to install federated-learning-v2@shield.mozilla.org-2.1.3-signed-testing.xpi fails and the browser log reports Add-on federated-learning-v2@shield.mozilla.org is not correctly signed.
Using about:debugging, it installs but the add-on runtime fails with TypeError: browser.study is undefined
. This was tested on beta and dev edition.
Comment 13•6 years ago
|
||
This is expected, since this is a testing signature. In order to test these add-ons, you must enable the dev-root for add-on signature verification. To do that, you must use a Nightly or un-signed build. Instructions for how to that are maintained by QA here.
Comment 14•6 years ago
|
||
(In reply to Michael Cooper [:mythmon] from comment #13)
This is expected, since this is a testing signature. In order to test these add-ons, you must enable the dev-root for add-on signature verification. To do that, you must use a Nightly or un-signed build. Instructions for how to that are maintained by QA here.
Thanks mythmon, that clarifies it. I was afraid that the testing of "Not showing in about:addons
" was not possible outside of the branded builds, but it turns it that it is perfectly testable in the Unbranded builds.
Comment 16•6 years ago
|
||
Updated•6 years ago
|
Comment 17•6 years ago
|
||
Add-On Study: Federated Learning v2
Targeted: Firefox Release 66.0
We have finished testing the Add-On Study: Federated Learning v2 experiment.
QA’s recommendation: GREEN - SHIP IT
Reasoning:
- All the logged issues were either fixed or labeled as won't fix by the development team. None of the won't fix issues affect the users since most of them are telemetry data related.
- Because of the high amount of test cases and the complexity of the telemetry probes, we were unable to fully test the experiment on Linux (76% with no new issues found). However, we did manage to cover 100% of Mac and Windows OSes.
Testing Summary:
- Test Suite: Test Rail
Tested Platforms:
- Windows 10 x64
- Mac 10.14
- Ubuntu 16.04 x64
Tested Firefox versions:
- Firefox 66.0b14 Beta Unbranded build (en-US)
- Firefox 66.0b14 Dev Edition build (en-US)
Updated•6 years ago
|
Reporter | ||
Comment 18•6 years ago
|
||
Comment 19•6 years ago
|
||
Reporter | ||
Comment 20•6 years ago
|
||
Comment 21•6 years ago
|
||
Attached v2.2.0 for signing
Comment 22•6 years ago
|
||
Requesting QA sign-off for minor changes reflected in v2.2.0
Comment 23•6 years ago
|
||
Updated•6 years ago
|
Comment 24•6 years ago
|
||
Note for QA: The changes in v2.2.0 were necessary to be compatible with the updated remote streaming ETL job and does not affect any user-facing behavior.
Comment 25•6 years ago
|
||
Hi Martin, Carmen is returning tomorrow from PTO. Since experiments are launching only on Monday, I assume that we can wait for her return to validate the new build, right?
Comment 26•6 years ago
|
||
@Paul Oiegas, Yes, no problem.
As fwollsen mentions these are minor changes to improve compatibility with the downstream data processing jobs. It should be straightforward if you/Carmen are comfortable reviewing the diff. We aim to relaunch with the new xpi on Monday April 1st.
Comment 27•6 years ago
|
||
I've just posted the QA sign off for 2.2.0 build on bug 1539034.
Comment 28•6 years ago
|
||
ended in May
Description
•