tests randomly connect to incoming.telemetry.mozilla.org

RESOLVED FIXED in Firefox 30

Status

()

defect
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: froydnj, Assigned: froydnj)

Tracking

Trunk
mozilla32
Points:
---
Dependency tree / graph
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(firefox29 wontfix, firefox30 fixed, firefox31 fixed, firefox32 fixed, firefox-esr24 fixed, b2g-v1.2 fixed, b2g-v1.3 fixed, b2g-v1.3T fixed, b2g-v1.4 fixed, b2g-v2.0 fixed)

Details

Attachments

(3 attachments)

Despite toolkit.telemetry.enabled not being set, which should mean that we've disabled telemetry:

http://mxr.mozilla.org/mozilla-central/source/toolkit/components/telemetry/TelemetryPing.jsm#891

We still see tests connecting to incoming.telemetry.mozilla.org:

https://tbpl.mozilla.org/php/getParsedLog.php?id=38017029&tree=Try
https://tbpl.mozilla.org/php/getParsedLog.php?id=38016931&tree=Try
https://tbpl.mozilla.org/php/getParsedLog.php?id=38016918&tree=Try

We need to fix this to enable non-local connection protection in the testsuites.  We also need to fix this as incoming telemetry data from tests might be seriously skewing results.
OK, so this is because of a number of things:

- Our try builds compile with --enable-update-channel=

- We have code to set MOZ_TELEMETRY_ON_BY_DEFAULT in configure.in:

if test "$MOZ_TELEMETRY_REPORTING"; then
    AC_DEFINE(MOZ_TELEMETRY_REPORTING)

    # Enable Telemetry by default for nightly and aurora channels
    if test -z "$RELEASE_BUILD"; then
      AC_DEFINE(MOZ_TELEMETRY_ON_BY_DEFAULT)
    fi
fi

Notice that decision is made by looking at RELEASE_BUILD, *not* the explicit update channel.

- RELEASE_BUILD is set using some completely different mechanism that's also not consulting the update channel:

# set RELEASE_BUILD and NIGHTLY_BUILD variables depending on the cycle we're in
# The logic works like this:
# - if we have "a1" in GRE_MILESTONE, we're building Nightly (define NIGHTLY_BUILD)
# - otherwise, if we have "a" in GRE_MILESTONE, we're building Nightly or Aurora
# - otherwise, we're building Release/Beta (define RELEASE_BUILD)
case "$GRE_MILESTONE" in
  *a1*)
      NIGHTLY_BUILD=1
      AC_DEFINE(NIGHTLY_BUILD)
      ;;
  *a*)
      ;;
  *)
      RELEASE_BUILD=1
      AC_DEFINE(RELEASE_BUILD)
      ;;
esac
AC_SUBST(NIGHTLY_BUILD)
AC_SUBST(RELEASE_BUILD)

- Finally, we have this bit of code in Preferences.cpp that attempts to be helpful:

  // Set up the correct default for toolkit.telemetry.enabled.
  // If this build has MOZ_TELEMETRY_ON_BY_DEFAULT *or* we're on the beta
  // channel, telemetry is on by default, otherwise not. This is necessary
  // so that beta users who are testing final release builds don't flipflop
  // defaults.
  if (Preferences::GetDefaultType(kTelemetryPref) == PREF_INVALID) {
    bool prerelease = false;
#ifdef MOZ_TELEMETRY_ON_BY_DEFAULT
    prerelease = true;
#else
    if (Preferences::GetDefaultCString(kChannelPref).EqualsLiteral("beta")) {
      prerelease = true;
    }
#endif
    PREF_SetBoolPref(kTelemetryPref, prerelease, true);
  }

which flips on telemetry for our try builds and tests.

We have code in mozprofile to disable telemetry for created profiles, but either we're not using that or it gets overridden by something else that I haven't discovered yet.
Luckily the Preferences.cpp thing seems to be a recent change.
Depends on: 986582
I chose two different strategies for the different kinds of tests here.  I
disabled telemetry entirely for {crash,ref,jsref}tests because we're not
running a proxy server there.  For mochitests, it's ok to try connecting to a
server that's going to 404 us, so we can just change the server and leave it
enabled.  I suppose we could be consistent and disable telemetry everywhere...
Attachment #8409049 - Flags: review?(ted)
Attachment #8409049 - Flags: review?(ted) → review+
Assignee: nobody → nfroyd
https://hg.mozilla.org/mozilla-central/rev/dd9b23471fc1
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla31
I can still see tests on try connecting to incoming.telemetry.mozilla.org, even with this patch in place.  So I'm not quite sure if this has totally stuck yet.
Apparently the preferences setting in reftest-preferences.js runs too late to
properly affect telemetry startup.  Therefore, we need to set the preference in
the profile itself so things get initialized properly.
Attachment #8411875 - Flags: review?(ted)
jsreftests uses a completely different framework, so we need to disable
telemetry there as well.
Attachment #8411876 - Flags: review?(ted)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Target Milestone: mozilla31 → ---
Attachment #8411875 - Flags: review?(ted) → review+
Attachment #8411876 - Flags: review?(ted) → review+
https://hg.mozilla.org/mozilla-central/rev/4797c2555c1b
https://hg.mozilla.org/mozilla-central/rev/9bb8fb4f73f8
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla32
And an esr24 follow-up from rebasing silliness on my part.
https://hg.mozilla.org/releases/mozilla-esr24/rev/2d09a4707dd3
You need to log in before you can comment on or make changes to this bug.