1233687 - [meta] Ubuntu Firefox builds don't send Telemetry

Status: RESOLVED FIXED

(Toolkit :: Telemetry, defect, P3)

Description

[Alessio Placitelli [:Dexter]]

Attachment: Ubuntu_43.PNG

I just noticed that on my Ubuntu 14.04.3 (trusty) 64 bit there's no entry in the options panel to enable extended telemetry.

I verified that this is also happening on Nightly.

Comment 1

[Alessio Placitelli [:Dexter]]

[Tracking Requested - why for this release]:

What's Broken: there is no way to turn on extended telemetry data collection on FF 43 on Ubuntu from the UI.
Why is this important: potentially, no data extended/opt-in measurements will be collected on that platform, affecting our ability to perform analysis.

Comment 2

[Sylvestre Ledru [:Sylvestre]]

Note that it is not the official Firefox build, options might differ from the official build.

Comment 3

[Georg Fritzsche [:gfritzsche]]

* Does it actually record & submit Telemetry?
* If not, should it?

Comment 4

[Georg Fritzsche [:gfritzsche]]

ni?dexter for checking and following up with Benjamin & Sylvestre.

Comment 5

[Alessio Placitelli [:Dexter]]

It turns out that Firefox on Ubuntu:

- records base telemetry but not extended telemetry;
- is not sending out any ping (not base, nor extended).
- used to have FHR reporting (at least it wasn't disabled using preprocessor defines), before we turned it off.

Apparently, Firefox on Ubuntu is not defining:

- MOZILLA_OFFICIAL, which prevents pings to be sent [0];
- MOZ_TELEMETRY_REPORTING, which also disables the "Share additional data" UI in the Preferences page [1].

@Benjamin and @Sylvestre: should Telemetry be reporting on those platforms?

[0] - https://dxr.mozilla.org/mozilla-central/rev/c5cb194cc9cb56d742fb3a7a826f0080b0404edc/toolkit/components/telemetry/TelemetrySend.jsm#1035
[1] - https://dxr.mozilla.org/mozilla-central/rev/c5cb194cc9cb56d742fb3a7a826f0080b0404edc/browser/components/preferences/in-content/advanced.xul#203

Comment 6

[Sylvestre Ledru [:Sylvestre]]

To me, it should have the same behavior as the official build.

Comment 7

[Benjamin Smedberg]

Made an introduction via email.

Comment 8

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

This seems like a basic scenario that we need working properly on Unbuntu if we want usage data. I'd be happy to take a patch (if not too risky) in Beta44.

Comment 9

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Benjamin, is this something we plan to fix in FF44 time frame?

Comment 10

[Georg Fritzsche [:gfritzsche]]

(In reply to Alessio Placitelli [:Dexter] from comment #5)
> Apparently, Firefox on Ubuntu is not defining:
> 
> - MOZILLA_OFFICIAL, which prevents pings to be sent [0];
> - MOZ_TELEMETRY_REPORTING, which also disables the "Share additional data"
> UI in the Preferences page [1].

From what i gather, Firefox on Ubuntu has our official branding and we want to treat it like other official builds here.

Do we know why MOZILLA_OFFICIAL is not set for this build?
Because we don't built it ourselves?

MOZ_TELEMETRY_REPORTING not being on is probably just a side-effect of MOZILLA_OFFICIAL not being set.

Once we figure out the _OFFICIAL setting, we also need to check on how we can tell this build apart (is there partner info or other things?).

Comment 11

[Georg Fritzsche [:gfritzsche]]

(In reply to Ritu Kothari (:ritu) from comment #9)
> Benjamin, is this something we plan to fix in FF44 time frame?

FF44 seems unlikely at this point.

Comment 12

[Sylvestre Ledru [:Sylvestre]]

Mike, could you help here? (comment #10) thanks!

Tracked because I care about that!

Comment 13

[Liz Henry (:lizzard) (relman/hg->git project)]

Benjamin can you help us find an owner for this bug? It will likely be wontfixed for 44 at this point.

Comment 14

[Georg Fritzsche [:gfritzsche]]

We have are talking with Chris from Ubuntu and are currently waiting on an update.

Comment 15

[Alessio Placitelli [:Dexter]]

After a conversation with Chris Coulson (Ubuntu), we identified a clear course of action to resolve this problem:

- Define both MOZ_TELEMETRY_REPORTING and MOZILLA_OFFICIAL (or work around MOZILLA_OFFICIAL in case they should not define it).
- Allow to externally specify an Histogram.json revision when not using the mercurial repository (bug 1241111 takes care of that).

They used to define MOZILLA_OFFICIAL, but they no longer do. On a quick look, there doesn't seem to be a technical reason for them not to define MOZILLA_OFFICIAL again. The question is: should they use this? Benjamin, do you have any opinion on this?

Comment 16

[Benjamin Smedberg]

Doesn't that depend on what MOZILLA_OFFICIAL actually does? It's a preprocessor constant that affects a number of things. A quick MXR search shows these at least:

http://mxr.mozilla.org/mozilla-central/source/browser/modules/AboutHome.jsm#55
http://mxr.mozilla.org/mozilla-central/source/browser/app/profile/firefox.js#967
http://mxr.mozilla.org/mozilla-central/source/devtools/client/framework/ToolboxProcess.jsm#213

I couldn't say which of those should be in Ubuntu official builds or not.

Comment 17

[Alessio Placitelli [:Dexter]]

The FHR-dev thread: https://mail.mozilla.org/pipermail/fhr-dev/2016-January/000764.html

Comment 18

[Mike Connor [:mconnor]]

Ubuntu builds should behave like our builds.  I think they should be setting MOZ_OFFICIAL, and reporting any issues arising from use of that flag to Mozilla for consideration.

Comment 19

[Chris Coulson]

(In reply to Mike Connor [:mconnor] from comment #18)
> Ubuntu builds should behave like our builds.  I think they should be setting
> MOZ_OFFICIAL, and reporting any issues arising from use of that flag to
> Mozilla for consideration.

We used to build with MOZILLA_OFFICIAL - we defined it when we started sending crash reports, and then removed it again sometime after http://hg.mozilla.org/mozilla-central/rev/8a6ed3ef085a landed.

Comment 20

[Georg Fritzsche [:gfritzsche]]

Great, sounds like we can move forward on that path then.

From what i understand the potential issues here are (please correct me):

* SOURCE_REV_URL should be set properly so we can identify the correct Histograms.json for Telemetry submissions (handled in bug 1241111)

* enabling MOZILLA_OFFICIAL enables MOZ_INCLUDE_SOURCE_INFO, we need to make sure that doesn't break anything (bug 1241883)

* Make sure that enabling MOZILLA_OFFICIAL doesn't trigger undesired behavior on Ubuntu.
  Alessio & Chris checked through what it enables and it seemed ok.
  The about:rights notification should not show on Ubuntu, but can be disabled by a pref.

Comment 21

[Alessio Placitelli [:Dexter]]

(In reply to Georg Fritzsche [:gfritzsche] from comment #20)
> Great, sounds like we can move forward on that path then.
> 
> From what i understand the potential issues here are (please correct me):
> 
> * SOURCE_REV_URL should be set properly so we can identify the correct
> Histograms.json for Telemetry submissions (handled in bug 1241111)
> 
> * enabling MOZILLA_OFFICIAL enables MOZ_INCLUDE_SOURCE_INFO, we need to make
> sure that doesn't break anything (bug 1241883)
> 
> * Make sure that enabling MOZILLA_OFFICIAL doesn't trigger undesired
> behavior on Ubuntu.
>   Alessio & Chris checked through what it enables and it seemed ok.
>   The about:rights notification should not show on Ubuntu, but can be
> disabled by a pref.

Yep, this is correct.

Comment 22

[Chris Coulson]

(In reply to Georg Fritzsche [:gfritzsche] from comment #20)
> Great, sounds like we can move forward on that path then.
> 
> From what i understand the potential issues here are (please correct me):
> 
> * SOURCE_REV_URL should be set properly so we can identify the correct
> Histograms.json for Telemetry submissions (handled in bug 1241111)
> 
> * enabling MOZILLA_OFFICIAL enables MOZ_INCLUDE_SOURCE_INFO, we need to make
> sure that doesn't break anything (bug 1241883)
> 
> * Make sure that enabling MOZILLA_OFFICIAL doesn't trigger undesired
> behavior on Ubuntu.
>   Alessio & Chris checked through what it enables and it seemed ok.
>   The about:rights notification should not show on Ubuntu, but can be
> disabled by a pref.

On this last point - I believe that the about:rights notification is actually meant to be shown in Ubuntu, and it was shown at one point (IIRC it was displayed via a button in an infobar). But having looked at this over the last couple of days, I discovered that it got moved to the start page in https://hg.mozilla.org/mozilla-central/rev/c950d2ea4c5b. As we use our own start page in Ubuntu, we obviously don't show it anymore and this has gone unnoticed for nearly 3 years :/

Comment 23

[Georg Fritzsche [:gfritzsche]]

(In reply to Chris Coulson from comment #22)
> (In reply to Georg Fritzsche [:gfritzsche] from comment #20)
> > Great, sounds like we can move forward on that path then.
> > 
> > From what i understand the potential issues here are (please correct me):
> > 
> > * SOURCE_REV_URL should be set properly so we can identify the correct
> > Histograms.json for Telemetry submissions (handled in bug 1241111)
> > 
> > * enabling MOZILLA_OFFICIAL enables MOZ_INCLUDE_SOURCE_INFO, we need to make
> > sure that doesn't break anything (bug 1241883)
> > 
> > * Make sure that enabling MOZILLA_OFFICIAL doesn't trigger undesired
> > behavior on Ubuntu.
> >   Alessio & Chris checked through what it enables and it seemed ok.
> >   The about:rights notification should not show on Ubuntu, but can be
> > disabled by a pref.
> 
> On this last point - I believe that the about:rights notification is
> actually meant to be shown in Ubuntu, and it was shown at one point (IIRC it
> was displayed via a button in an infobar). But having looked at this over
> the last couple of days, I discovered that it got moved to the start page in
> https://hg.mozilla.org/mozilla-central/rev/c950d2ea4c5b. As we use our own
> start page in Ubuntu, we obviously don't show it anymore and this has gone
> unnoticed for nearly 3 years :/

Ah, fun - i moved that to bug 1242566 (it's already broken, so it doesn't block the steps for enabling Telemetry).

Comment 24

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Too late for Fx44.

Comment 25

[Alessio Placitelli [:Dexter]]

Telemetry will be enabled on all channels for Ubuntu Firefox: their channels match our own channels (nightly, aurora, beta, release).

They upload Beta every time there's a new Release. Nightlies are uploaded automatically every day - although this doesn't always happen (eg, there's an issue with or some of their patches don't apply cleanly). This is good to know and to take in consideration when we will be analysing this chunk of data, in the future.

We're aiming for getting Telemetry from Ubuntu Firefox 45 Beta. Kudos to Chris Coulson for his help and patience!

Comment 26

[Alessio Placitelli [:Dexter]]

The Ubuntu Team has started packing Ubuntu Firefox builds shipping with Unified Telemetry support. The support was added to all channels (nightly, aurora, beta, release).

FHR-dev announcement: https://mail.mozilla.org/pipermail/fhr-dev/2016-February/000819.html

Comment 27

[Alessio Placitelli [:Dexter]]

As mentioned in comment 26, Ubuntu Firefox builds are sending telemetry pings. The data was validated both locally (bug 1243772) and server-side (beta channel, bug 1246631). We're waiting for Ubuntu Firefox 45 Release to be packed and released in order to validate release data.

Is there anything else I need to do on this bug since it's tracked for Firefox 45?

Comment 28

[Georg Fritzsche [:gfritzsche]]

The client work here is done, but this tracking bug still depends on the validation bug 1246631.
I think we should track the client-side bug 1244688 instead and keep this one open.

Comment 29

[Alessio Placitelli [:Dexter]]

All the dependencies have landed and the pings are correctly flowing in.

Comment 30

[Seburo]

Hi

What is the time-scale for this being rolled out across all of Live/Beta/Aurora/Nightly?

I am using an Aurora PPA (reinstalled from scratch in the past five days, current version is 47.0a2 (2016-04-13)) on Ubuntu 16.04 and do not have any telemetry options in about:preferences#advanced (Data Choices).

Comment 31

[Alessio Placitelli [:Dexter]]

(In reply to Seburo from comment #30)
> Hi
> 
> What is the time-scale for this being rolled out across all of
> Live/Beta/Aurora/Nightly?
> 
> I am using an Aurora PPA (reinstalled from scratch in the past five days,
> current version is 47.0a2 (2016-04-13)) on Ubuntu 16.04 and do not have any
> telemetry options in about:preferences#advanced (Data Choices).

Thank you Seburo, Telemetry was rolled out across all channels, but as you noticed it appears not to be working on Ubuntu Firefox Aurora. I've filed bug 1271269 to deal with that. Thanks for bringing this up!

Comment 32

[Botond Ballo [:botond]]

What about other Linux distributions?

Comment 33

[Alessio Placitelli [:Dexter]]

(In reply to Botond Ballo [:botond] from comment #32)
> What about other Linux distributions?

We're working on enabling support for the other distributions as well. See bug 1285201 and bug 1285195. More will follow.