1285201 - Arch Linux Firefox builds are not sending Telemetry

Status: RESOLVED FIXED

(Toolkit :: Telemetry, defect, P3)

Description

[Alessio Placitelli [:Dexter]]

The Arch Linux Firefox builds are not sending Telemetry.

This bug is about enabling Telemetry on those builds.

Comment 1

[Jan Alexander Steffens [:heftig]]

Release builds come from tarballs, but I'm also building Aurora nightlies from Git.

Comment 2

[Alessio Placitelli [:Dexter]]

(In reply to Jan Steffens from comment #1)
> Release builds come from tarballs, but I'm also building Aurora nightlies
> from Git.

Cool, thanks for adding the dependency.

Comment 3

[Jan Alexander Steffens [:heftig]]

Well, I got impatient waiting for the tarballs to get fixed and our current build server has disk space to spare, so Arch Linux builds from Mercurial now and we're sending Telemetry.

Comment 4

[Alessio Placitelli [:Dexter]]

(In reply to Jan Steffens from comment #3)
> Well, I got impatient waiting for the tarballs to get fixed and our current
> build server has disk space to spare, so Arch Linux builds from Mercurial
> now and we're sending Telemetry.

Thanks for the efforts and for letting us know, as this might have some impact on analyses on our end :)

I have a few questions though:

1) Did the Telemetry-enabled builds of Firefox already roll out?
2) Could you kindly post a link to the patch/changes used to enable Telemetry?
3) Is Telemetry enabled on all channels for Arch Linux?
4) Are the the channels of Firefox for Arch Linux the same of the ones from Mozilla (e.g. Nightly, Aurora, Beta, Release)? And to they follow the same release schedule?
5) Was any QA performed to make sure nothing broke as a resulting of enabling Telemetry? I've just filed bug 1348517 to make sure we're receiving correct data on our end.

However, great to hear this happened!

Comment 5

[Jan Alexander Steffens [:heftig]]

(In reply to Alessio Placitelli [:Dexter] from comment #4)
> (In reply to Jan Steffens from comment #3)
> > Well, I got impatient waiting for the tarballs to get fixed and our current
> > build server has disk space to spare, so Arch Linux builds from Mercurial
> > now and we're sending Telemetry.
> 
> Thanks for the efforts and for letting us know, as this might have some
> impact on analyses on our end :)
> 
> I have a few questions though:
> 
> 1) Did the Telemetry-enabled builds of Firefox already roll out?

Yes.

> 2) Could you kindly post a link to the patch/changes used to enable
> Telemetry?

https://git.archlinux.org/svntogit/packages.git/commit/?h=packages/firefox&id=0368ebdb85ecc4c0f081406e813e26c829047671

> 3) Is Telemetry enabled on all channels for Arch Linux?

Yes.

> 4) Are the the channels of Firefox for Arch Linux the same of the ones from
> Mozilla (e.g. Nightly, Aurora, Beta, Release)? And do they follow the same
> release schedule?

Only the release channel is part of the official packages. We follow Mozilla releases promptly to the best of our ability.

I publish aurora channel nightlies in the repository at https://pkgbuild.com/~heftig/repo/ (source at https://pkgbuild.com/~heftig/packages/firefox-developer-edition/) but their reach is far smaller. This package also acts as a staging ground for changes that I eventually add to the release package.

> 5) Was any QA performed to make sure nothing broke as a resulting of
> enabling Telemetry?

I've had telemetry enabled in Aurora for a while without ill effects.

> I've just filed bug 1348517 to make sure we're receiving correct data on our end.

Thanks.

> However, great to hear this happened!

Yeah, I think this has been long overdue. Not submitting telemetry is like not going to vote...

Comment 6

[Alessio Placitelli [:Dexter]]

(In reply to Jan Steffens from comment #5)
> > 2) Could you kindly post a link to the patch/changes used to enable
> > Telemetry?
> 
> https://git.archlinux.org/svntogit/packages.git/commit/?h=packages/
> firefox&id=0368ebdb85ecc4c0f081406e813e26c829047671

I can't tell right away if you're changing the distribution id for the ArchLinux Firefox build.
If you are not, then it would be more difficult for us (or impossible) to distinguish between pings coming from ArchLinux Firefox and other pings.

> > 5) Was any QA performed to make sure nothing broke as a resulting of
> > enabling Telemetry?
> 
> I've had telemetry enabled in Aurora for a while without ill effects.

Yeah, still there could be some less obvious consequences. Let's add some QA love on this :-)

Madalin, would you kindly perform some general Telemetry QA on the latest version of ArchLinux Firefox, for the Release channel, on the ArchLinux OS?

I compiled a non comprehensive list of test cases in bug 1243772 comment 2, let me report them here as well.

(In reply to Alessio Placitelli [:Dexter] from bug 1243772 comment #2)
> 
> Test: nothing is sent before the policy infobar is shown
> Outcome: nothing is sent before, pings are correctly sent after
> 
> Test: make sure release Firefox has extended Telemetry opt-in, pre-release
> opt-out
> Outcome: pre-release channels correctly have Telemetry as opt-out. No build
> is available at the moment to test the release channel
> 
> Test: make sure the environment reports the ArchLinux distribution id
> Outcome: The environment.partner.distributionId section of the environment
> contains the "archlinux" string
> 
> Test: make sure no extended telemetry is recorded if Telemetry is turned off
> Outcome: no extended Telemetry is present in pings generated with Telemetry
> being off. No saved-session pings are saved at shutdown.
> 
> Test: make sure nothing gets sent when FHR data upload is disabled.
> Outcome: The ping upload preference is respected, no pings are uploaded
> (except the deletion ping) when the Firefox Health Report option is disabled.
> 
> Test: make sure opt-out Telemetry is sent with only FHR upload enabled.
> Outcome: Only base telemetry is sent.

Comment 7

[Jan Alexander Steffens [:heftig]]

I'm pretty sure we're not setting the distribution ID. What's needed? I found --with-distribution-id, but the existence of references to the "distribution.id" and "distribution.version" prefs suggests that isn't enough.

Comment 8

[Alessio Placitelli [:Dexter]]

Whoops, I didn't mean to clear my ni? yet. I'll follow up once I have an answer to comment 7.

Comment 9

[Jan Alexander Steffens [:heftig]]

Found the "distribution/distribution.ini" file in the Ubuntu Firefox. I'll set ours like this:


[Global]
id=archlinux
version=<package release version>
about=Mozilla Firefox for Arch Linux

[Preferences]
app.distributor = archlinux
app.distributor.channel = <package name>
app.partner.archlinux = archlinux


This will result in a lot of different distributionVersion values, e.g.:

  version=52.0.1-2

  version=54.0a2+20170320+h6a72a11138b9-1

I can set it to a static 1.0 if that's preferred.

Comment 10

[Alessio Placitelli [:Dexter]]

(In reply to Jan Steffens from comment #9)
> Found the "distribution/distribution.ini" file in the Ubuntu Firefox. I'll
> set ours like this:

Hang on, let's figure out if that's the proper change :)

Mike, are you the right person to check the changes from comment 9?

Comment 11

[Mike Connor [:mconnor]]

Happy to help, Mike Kaply (cced) is another useful contact.

I'd use 1.0, the version is generally specific to the distribution config, not the app version.  If there's a significant change to the config, increment it somehow, but otherwise 1.0 is fine.

Comment 12

[Jan Alexander Steffens [:heftig]]

Thanks, 1.0 it is. Is there some other field I can put the package version into? I guess that would make it easier to correlate with our packages than digging through them for buildIds.

Comment 13

[Alessio Placitelli [:Dexter]]

(In reply to Jan Steffens from comment #12)
> Thanks, 1.0 it is. Is there some other field I can put the package version
> into? I guess that would make it easier to correlate with our packages than
> digging through them for buildIds.

In addition to the distribution.ini file, we need to make sure that the revision information is being correctly picked up by the build system.

You can check it like that :

- Go to the about:telemetry page
- Click on "Raw JSON"
- Look up the "revision" entry in the "info" section. You should see something like:

> "revision": "https://hg.mozilla.org/releases/mozilla-release/rev/2f2b4a119565e9b5691187ee5fbe91881c90b249",

Comment 14

[Alessio Placitelli [:Dexter]]

(In reply to Mike Connor [:mconnor] from comment #11)
> Happy to help, Mike Kaply (cced) is another useful contact.
> 
> I'd use 1.0, the version is generally specific to the distribution config,
> not the app version.  If there's a significant change to the config,
> increment it somehow, but otherwise 1.0 is fine.

Also, thanks Mike :)

Comment 15

[Jan Alexander Steffens [:heftig]]

(In reply to Alessio Placitelli [:Dexter] from comment #13)
> (In reply to Jan Steffens from comment #12)
> In addition to the distribution.ini file, we need to make sure that the
> revision information is being correctly picked up by the build system.

Yes, that information is present.

>      "revision": "https://hg.mozilla.org/releases/mozilla-release/rev/a38b8538f42412d2c606417264df4abcc183dd57",

Comment 16

[Alessio Placitelli [:Dexter]]

Thanks Jan.

It would be great if you could also confirm that it's generally working by testing as suggested in comment 6, as QA might not get to this immediately.

Please kindly let me know once you update the distribution.ini file. Could you also link the changeset here so that we can keep track if it too?

Comment 17

[Jan Alexander Steffens [:heftig]]

(In reply to Alessio Placitelli [:Dexter] from bug 1243772 comment #2)
> Test: nothing is sent before the policy infobar is shown

toolkit.telemetry.cachedClientID is not set on first start. After waiting a while, the infobar appears at the bottom and the cachedClientID is set. Not sure how to verify what's actually sent.

> Test: make sure release Firefox has extended Telemetry opt-in, pre-release
> opt-out

about:telemetry reports
   FHR data upload is enabled.
   Extended Telemetry recording is disabled.

> Test: make sure the environment reports the canonical distribution id

about:telemetry reports:
   "environment": {
     "partner": {
       "distributionId": "archlinux",
       "distributionVersion": "1.0",
       "partnerId": null,
       "distributor": "archlinux",
       "distributorChannel": "firefox",
       "partnerNames": [
         "archlinux"
       ]

> Test: make sure no extended telemetry is recorded if Telemetry is turned off

Not sure how to test this.

> Test: make sure nothing gets sent when FHR data upload is disabled.

Not sure how to test this.

> Test: make sure opt-out Telemetry is sent with only FHR upload enabled.

Not sure how to test this.

Comment 18

[Jan Alexander Steffens [:heftig]]

(In reply to Alessio Placitelli [:Dexter] from comment #16)
> Please kindly let me know once you update the distribution.ini file. Could
> you also link the changeset here so that we can keep track if it too?

distribution.ini is shipped as of 52.0.1-2:
https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/firefox&id=f042bdecefce83f18a1383096e3a2f815aa706d0

Comment 19

[Alessio Placitelli [:Dexter]]

Thanks Jan. That should be enough! Let's leave the the rest of the cases to Madalin, you've done more than enough!

Comment 20

[Madalin Cotetiu]

I have started creating a document with test cases based on scenarios from comment 6. Sorry for the delay, hopefully the testing will be completed by the end of the week. Thanks

Comment 21

[Madalin Cotetiu]

The document is completed and the results are added. The results can be seen in here: https://docs.google.com/spreadsheets/d/1O17gFbUWAzCmRllHuigkbnS7pStvKAxhoZgKcmbgBiU/edit#gid=0

We will spend some extra time doing exploratory testing around those scenarios.

Comment 22

[Alessio Placitelli [:Dexter]]

(In reply to Madalin Cotetiu from comment #21)
> The document is completed and the results are added. The results can be seen
> in here:
> https://docs.google.com/spreadsheets/d/
> 1O17gFbUWAzCmRllHuigkbnS7pStvKAxhoZgKcmbgBiU/edit#gid=0
> 
> We will spend some extra time doing exploratory testing around those
> scenarios.

Thanks Madalin!