Closed Bug 841978 Opened 7 years ago Closed 7 years ago

Disable telemetry for mozbase dependent test harnesses

Categories

(Testing :: Mozbase, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: MattN, Assigned: whimboo)

References

Details

(Whiteboard: [mozbase][mozmill-1.5.21+])

Attachments

(2 files)

Bug 819732 and its dependencies enabled telemetry by default on Nightly and Aurora.  We should disable telemetry in Mozmill to avoid skewing the data.

I think setting toolkit.telemetry.enabled and toolkit.telemetry.enabledPreRelease to false will do this but Taras can correct me if there is a better way.
Although I'm not 100% sure that we want to disable Telemetry (I don't know all of the details here), Theo wrote the enabled-by-default code and should be able to advise on the best way to disable Telemetry for Mozmill.
(In reply to Lawrence Mandel [:lmandel] from comment #1)
> Although I'm not 100% sure that we want to disable Telemetry (I don't know
> all of the details here), 

Ignore this part of my response. I just read bug 841967.
Blocks: 841967
This is kind of a big deal. The probe noted in 841967 is overwhelmed with bogus-looking data, and the conjecture(?) is that it's due to Mozmill machines reporting. That rather spoils the usefulness of Telemerty data. If Mozmill runs are actually causing telemetry data to be reported, that needs to stop ASAP.
If you run Mozmill on "default" channel, for instance, you’re safe.

If you run Mozmill on "nightly" channel, as you said you will have to set toolkit.telemetry.enablePreRelease to false. (toolkit.telemetry.enabled is already false)
Is Mozmill typically installed as an extension? In other words, can we filter out the Mozmill Telemetry reports at the server by checking the list of installed extensions in the report?
As I have read on another bug we send telemetry data after about 24h of usage. Is that correct? We do not re-use profiles when running Mozmill tests and we do not exceed 60 minutes of test runs for a single profile.

Do you have machine names or IP addresses you think of that those are from Mozmill runs?
(In reply to Henrik Skupin (:whimboo) from comment #6)
> As I have read on another bug we send telemetry data after about 24h of
> usage. Is that correct? We do not re-use profiles when running Mozmill tests
> and we do not exceed 60 minutes of test runs for a single profile.

Vladan probably knows.

> Do you have machine names or IP addresses you think of that those are from
> Mozmill runs?

No, but we can get that info in bug 841967.  I suspect we would rather not rely on the above behaviour to prevent telemetry from being sent so we should disable telemetry in Mozmill regardless of whether it caused bug 841967.
Flags: needinfo?(vdjeric)
(In reply to Matthew N. [:MattN] from comment #7)
> (In reply to Henrik Skupin (:whimboo) from comment #6)
> > As I have read on another bug we send telemetry data after about 24h of
> > usage. Is that correct? We do not re-use profiles when running Mozmill tests
> > and we do not exceed 60 minutes of test runs for a single profile.
> 
> Vladan probably knows.

Firefox submits Telemetry at most once every 24 hours. However, on first run in a new profile, it will submit Telemetry after 5 minutes of system-wide user inactivity.

> I suspect we would rather not
> rely on the above behaviour to prevent telemetry from being sent so we
> should disable telemetry in Mozmill regardless of whether it caused bug
> 841967.

Sounds like a plan. You can disable Telemetry by setting the toolkit.telemetry.enabled and toolkit.telemetry.enabledPreRelease prefs to false.
Flags: needinfo?(vdjeric)
Blocks: 842939
Preferences are getting set via mozprofile (mozbase). That means more harnesses than Mozmill can be stopped from reporting telemetry information. I will work on a patch. Thanks for the information Vladan.
Assignee: nobody → hskupin
Status: NEW → ASSIGNED
Component: Mozmill → Mozbase
Summary: Disable telemetry in Mozmill → Disable telemetry for mozbase dependent test harnesses
Attachment #715961 - Flags: review?(jhammel)
Attachment #715965 - Flags: review?(jhammel)
Attachment #715965 - Flags: review?(jhammel) → review+
Ultimately preferences will be more distributed, see bug 830430, but in the interim this approach is fine
See Also: → 830430
Whiteboard: [mozbase]
Comment on attachment 715961 [details] [diff] [review]
Patch v1 (master)

WFM, but see previous comment
Comment on attachment 715965 [details] [diff] [review]
Patch v1 (mozmill 1.5)

Review of attachment 715965 [details] [diff] [review]:
-----------------------------------------------------------------

https://github.com/mozilla/mozmill/commit/cd219b0b75ece5b1927f63a8036a0d0413c620e7 (mozmill-1.5)
Attachment #715961 - Flags: review?(jhammel) → review+
Whiteboard: [mozbase] → [mozbase][mozmill-1.5.21+]
Comment on attachment 715961 [details] [diff] [review]
Patch v1 (master)

Review of attachment 715961 [details] [diff] [review]:
-----------------------------------------------------------------

https://github.com/mozilla/mozbase/commit/0a8ab18098bcde4fabe486fd39c716fd463f5dd8 (master)
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Currently, Telemetry only checks its prefs on startup & doesn't react to prefs changed during the current session.

I took a quick look at the git repo, but I just want to make sure: these patches will set the toolkit.telemetry.[..]enabled prefs when the profile is created, right?
Yes, that's exactly what the patches are doing. Those prefs are setup before the application gets started the first time.
You need to log in before you can comment on or make changes to this bug.