1594357 - Add basic Telemetry to remote agent

Status: NEW

(Remote Protocol :: Agent, enhancement, P3)

Description

[Andreas Tolfsen ❲:ato❳]

We want to measure when the remote agent is used, so a basic ping
when it is listening would suffice.

This is another action that came out of the security review.

Comment 1

[David Burns :automatedtester]

Historically companies have been against automation frameworks having telemetry pings. This is partly the reason why we haven't added it to Marionette in the past. One of their concerns is that Mozilla would be aware of their super secret projects when there is automation in play.

Currently my understanding is that puppeteer doesn't have this so we would need to double check that. If they don't we would need to approach some users to see if this would be a deal breaker to using puppeteer.

Comment 2

[Andreas Tolfsen ❲:ato❳]

As I understand it telemetry is anonymised so it hides the exact
location, company, application-under-test, or other data that could
be used for identification. This is in any case not the sort of
information we’d be interested in by having telemetry the remote
agent. I’m sure there are guidelines for what kinds of data it is
ethical to gather, but I don’t know where these can be found.

From my understanding, the reason we didn’t gather telemetry in
Marionette had more to do with concerns it would skew the results.
For example, the telemetry submitted by a browser automation tool
is not from real users, so there was concern that this would be
included in the same bucket as real-world users. Since then the
Telemetry team has made some changes to which “bucket” data is
gathered in, and I believe enabled Firefox telemetry when using
Marionette. This telemetry is often disabled in automation (i.e.
by geckoriver) but not fundamentally unavailable.

Anyway, this bug was filed because the security team felt it having
some data on how often the feature is used would be worth adding.
They would, and I am liberally quoting dveditz here, expect fairly
low constant numbers (actual test labs probably turn off telemetry)
until scammers discovers the feature, at which point we would get
some notice that someone malicious has found some way to exploit
it (until they figure out they should turn off telemetry).

In any case, the initial sec-review did not require that we add
telemetry to the remote agent before enabling it in the Nightly
channel.