Disable telemetry for mozbase dependent test harnesses

RESOLVED FIXED

Status

Testing
Mozbase
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: MattN, Assigned: whimboo)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [mozbase][mozmill-1.5.21+])

Attachments

(2 attachments)

Bug 819732 and its dependencies enabled telemetry by default on Nightly and Aurora.  We should disable telemetry in Mozmill to avoid skewing the data.

I think setting toolkit.telemetry.enabled and toolkit.telemetry.enabledPreRelease to false will do this but Taras can correct me if there is a better way.
Although I'm not 100% sure that we want to disable Telemetry (I don't know all of the details here), Theo wrote the enabled-by-default code and should be able to advise on the best way to disable Telemetry for Mozmill.
(In reply to Lawrence Mandel [:lmandel] from comment #1)
> Although I'm not 100% sure that we want to disable Telemetry (I don't know
> all of the details here), 

Ignore this part of my response. I just read bug 841967.
Blocks: 841967
This is kind of a big deal. The probe noted in 841967 is overwhelmed with bogus-looking data, and the conjecture(?) is that it's due to Mozmill machines reporting. That rather spoils the usefulness of Telemerty data. If Mozmill runs are actually causing telemetry data to be reported, that needs to stop ASAP.
If you run Mozmill on "default" channel, for instance, you’re safe.

If you run Mozmill on "nightly" channel, as you said you will have to set toolkit.telemetry.enablePreRelease to false. (toolkit.telemetry.enabled is already false)
Is Mozmill typically installed as an extension? In other words, can we filter out the Mozmill Telemetry reports at the server by checking the list of installed extensions in the report?
(Assignee)

Comment 6

5 years ago
As I have read on another bug we send telemetry data after about 24h of usage. Is that correct? We do not re-use profiles when running Mozmill tests and we do not exceed 60 minutes of test runs for a single profile.

Do you have machine names or IP addresses you think of that those are from Mozmill runs?
(In reply to Henrik Skupin (:whimboo) from comment #6)
> As I have read on another bug we send telemetry data after about 24h of
> usage. Is that correct? We do not re-use profiles when running Mozmill tests
> and we do not exceed 60 minutes of test runs for a single profile.

Vladan probably knows.

> Do you have machine names or IP addresses you think of that those are from
> Mozmill runs?

No, but we can get that info in bug 841967.  I suspect we would rather not rely on the above behaviour to prevent telemetry from being sent so we should disable telemetry in Mozmill regardless of whether it caused bug 841967.
Flags: needinfo?(vdjeric)
(In reply to Matthew N. [:MattN] from comment #7)
> (In reply to Henrik Skupin (:whimboo) from comment #6)
> > As I have read on another bug we send telemetry data after about 24h of
> > usage. Is that correct? We do not re-use profiles when running Mozmill tests
> > and we do not exceed 60 minutes of test runs for a single profile.
> 
> Vladan probably knows.

Firefox submits Telemetry at most once every 24 hours. However, on first run in a new profile, it will submit Telemetry after 5 minutes of system-wide user inactivity.

> I suspect we would rather not
> rely on the above behaviour to prevent telemetry from being sent so we
> should disable telemetry in Mozmill regardless of whether it caused bug
> 841967.

Sounds like a plan. You can disable Telemetry by setting the toolkit.telemetry.enabled and toolkit.telemetry.enabledPreRelease prefs to false.
Flags: needinfo?(vdjeric)
(Assignee)

Updated

5 years ago
Blocks: 842939
(Assignee)

Comment 9

5 years ago
Preferences are getting set via mozprofile (mozbase). That means more harnesses than Mozmill can be stopped from reporting telemetry information. I will work on a patch. Thanks for the information Vladan.
Assignee: nobody → hskupin
Status: NEW → ASSIGNED
Component: Mozmill → Mozbase
Summary: Disable telemetry in Mozmill → Disable telemetry for mozbase dependent test harnesses
(Assignee)

Comment 10

5 years ago
Created attachment 715961 [details] [diff] [review]
Patch v1 (master)
Attachment #715961 - Flags: review?(jhammel)
(Assignee)

Comment 11

5 years ago
Created attachment 715965 [details] [diff] [review]
Patch v1 (mozmill 1.5)
Attachment #715965 - Flags: review?(jhammel)

Updated

5 years ago
Attachment #715965 - Flags: review?(jhammel) → review+

Comment 12

5 years ago
Ultimately preferences will be more distributed, see bug 830430, but in the interim this approach is fine
See Also: → bug 830430
Whiteboard: [mozbase]

Comment 13

5 years ago
Comment on attachment 715961 [details] [diff] [review]
Patch v1 (master)

WFM, but see previous comment
(Assignee)

Comment 14

5 years ago
Comment on attachment 715965 [details] [diff] [review]
Patch v1 (mozmill 1.5)

Review of attachment 715965 [details] [diff] [review]:
-----------------------------------------------------------------

https://github.com/mozilla/mozmill/commit/cd219b0b75ece5b1927f63a8036a0d0413c620e7 (mozmill-1.5)

Updated

5 years ago
Attachment #715961 - Flags: review?(jhammel) → review+
(Assignee)

Updated

5 years ago
Whiteboard: [mozbase] → [mozbase][mozmill-1.5.21+]
(Assignee)

Comment 15

5 years ago
Comment on attachment 715961 [details] [diff] [review]
Patch v1 (master)

Review of attachment 715961 [details] [diff] [review]:
-----------------------------------------------------------------

https://github.com/mozilla/mozbase/commit/0a8ab18098bcde4fabe486fd39c716fd463f5dd8 (master)
(Assignee)

Updated

5 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Currently, Telemetry only checks its prefs on startup & doesn't react to prefs changed during the current session.

I took a quick look at the git repo, but I just want to make sure: these patches will set the toolkit.telemetry.[..]enabled prefs when the profile is created, right?
(Assignee)

Comment 17

5 years ago
Yes, that's exactly what the patches are doing. Those prefs are setup before the application gets started the first time.
You need to log in before you can comment on or make changes to this bug.