Closed Bug 1382719 Opened 2 years ago Closed 2 years ago

Mochitests against Nightly builds are mostly failing with FATAL ERROR: Non-local network connections are disabled and a connection attempt to tiles.services.mozilla.com (52.10.50.117) was made.

Categories

(Firefox :: New Tab Page, defect, blocker)

defect
Not set
blocker

Tracking

()

VERIFIED FIXED
Firefox 56
Tracking Status
firefox-esr52 --- unaffected
firefox54 --- unaffected
firefox55 --- unaffected
firefox56 + verified

People

(Reporter: KWierso, Assigned: Mardak)

References

Details

(Keywords: regression)

Attachments

(1 file)

Starting with the nightlies built on Tuesday, a lot of the mochitests that run against the Nightly builds have been failing after trying to contact tiles.services.mozilla.com: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&fromchange=56c6c81993e6b0f0e094e0c88b42ce3ec566ead9&noautoclassify&filter-searchStr=nightly%20(bc&group_state=expanded&selectedJob=115339579

I suspect something related to activity stream or the new tab page, since they're touching tiles-related stuff.
The add-on does pref things differently for local/"default"-channel builds vs official builds. I'm assuming there's a pref file somewhere for these "nightly" channel build test runs ?
[Tracking Requested - why for this release]: This is causing severely degraded test coverage on Nightly builds at the moment.
Severity: normal → blocker
Keywords: regression
OS: Unspecified → All
Hardware: Unspecified → All
Version: unspecified → Trunk
(In reply to Ed Lee :Mardak from comment #1)
> The add-on does pref things differently for local/"default"-channel builds
> vs official builds. I'm assuming there's a pref file somewhere for these
> "nightly" channel build test runs ?

That sounds exactly like what I've been suspecting. Would explain why only nightly builds are affected and not regular CI builds.

We set prefs for our CI profiles in a few different places, but the most applicable one for this issue is at:
https://dxr.mozilla.org/mozilla-central/source/testing/profiles/prefs_general.js
Is there a way to `try` as "nightly" builds? I suppose I could somewhere force the channel to be "nightly" to simulate.

I believe we'll want to do

```diff
diff --git a/testing/profiles/prefs_general.js b/testing/profiles/prefs_general.js
--- a/testing/profiles/prefs_general.js
+++ b/testing/profiles/prefs_general.js
@@ -299,2 +299,3 @@ user_pref("browser.aboutHomeSnippets.updateUrl", "nonexistent://test");
 user_pref("browser.newtabpage.activity-stream.default.sites", "");
+user_pref("browser.newtabpage.activity-stream.telemetry", false);
 
```
Assignee: nobody → edilee
Blocks: 1381569
For Linux and OSX, they're scheduled as regular Taskcluster jobs and should be testable on Try with normal syntax - |try: -b do -p linux64-nightly -u mochitest-bc -t none|. Worst case, you should be able to add them to existing Try push via the Add New Jobs option.
You probably can't use artifact builds for this...
(In reply to Ryan VanderMeulen [:RyanVM] from comment #8)
> You probably can't use artifact builds for this...
Ha ha. Doh. |mach try| foiled no more! Bug 1382827 :) https://reviewboard.mozilla.org/r/159484/
Comment on attachment 8888438 [details]
Bug 1382719 - Mochitests against Nightly builds are mostly failing with telemetry connections.

Looks like we add the browser.newtabpage.activity-stream.default.sites pref change to testing/talos/talos/config.py as well. Joel, should we add this pref change there too?
Flags: needinfo?(jmaher)
A quick glance shows no talos running on Nightly builds. I would guess talos is running on "default" channel like a local build, so the `telemetry` pref is already false. We've been getting talos results with activity stream enabled.

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=nightly%20talos
thanks for the heads up- we should add any pref changes to mochitest to talos testing/talos/talos/config.py
Flags: needinfo?(jmaher)
Comment on attachment 8888438 [details]
Bug 1382719 - Mochitests against Nightly builds are mostly failing with telemetry connections.

Try push is green, my hero <3
Attachment #8888438 - Flags: feedback+
Comment on attachment 8888438 [details]
Bug 1382719 - Mochitests against Nightly builds are mostly failing with telemetry connections.

https://reviewboard.mozilla.org/r/159388/#review164868

Looks good; r=dmose.
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/mozilla-central/rev/7d2e89fb9233
Disable Activity Stream telemetry during test runs to avoid nonlocal connection crashes. r=dmose, a=RyanVM
https://hg.mozilla.org/mozilla-central/rev/7d2e89fb92331d7e4296391213c1e63db628e046
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 56
Comment on attachment 8888438 [details]
Bug 1382719 - Mochitests against Nightly builds are mostly failing with telemetry connections.

https://reviewboard.mozilla.org/r/159388/#review164872

r=dmose
Comment on attachment 8888438 [details]
Bug 1382719 - Mochitests against Nightly builds are mostly failing with telemetry connections.

https://reviewboard.mozilla.org/r/159390/#review164876
Attachment #8888438 - Flags: review?(dmose) → review+
Not even sure how it's possible, but looks like this is also happening on esr52:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-esr52&selectedJob=116138221
Sorry that's a different IP.. Guess it could somehow still be related to this.
browser/extensions/activity-stream doesn't even exist there:
https://hg.mozilla.org/releases/mozilla-esr52/file/c52d8ce8d5dfd31e41dd4957428d85d5869e9a72/browser/extensions

dmose and I have seen this as of today when sherrifing our `pine` repository, and we were trying to track that down to see if one of our recent activity stream commits caused it.

We've only seen it once so far. Also on osx opt + debug:
https://treeherder.mozilla.org/#/jobs?repo=pine&revision=58faa12409fe273e163b532e629742ef20602bce&filter-classifiedState=unclassified&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&selectedJob=116029827
ahal: see bug 1382858
Verified fixed on today's nightlies \m/ :-). Thanks again for jumping on this, Ed!
Status: RESOLVED → VERIFIED
Component: Activity Streams: Newtab → New Tab Page
You need to log in before you can comment on or make changes to this bug.