1753649 - Perma Non-local network connections are disabled and a connection attempt to firefox.settings.services.mozilla.com was made when Gecko 98 merges to Beta on 2022-02-07

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect)

Description

[Cristina Cozmuta (:CrissCozmuta)]

Central-as-beta simulation
How to run these simulations

Failure log:

task 2022-02-04T12:44:59.386Z] 12:44:59     INFO -  DEBUG: Adding blocker places.sqlite#1: Bookmarks.jsm: insertBookmarkTree (9) for phase places.sqlite#1: waiting for clients
[task 2022-02-04T12:44:59.398Z] 12:44:59     INFO -  DEBUG: Completed blocker Transaction (8) for phase places.sqlite#1: waiting for clients
[task 2022-02-04T12:44:59.398Z] 12:44:59     INFO -  DEBUG: Completed blocker places.sqlite#1: Bookmarks.jsm: insertBookmarkTree (9) for phase places.sqlite#1: waiting for clients
[task 2022-02-04T12:44:59.399Z] 12:44:59     INFO -  DEBUG: Adding blocker places.sqlite#1: GuidHelper.getItemId (10) for phase places.sqlite#1: waiting for clients
[task 2022-02-04T12:44:59.402Z] 12:44:59     INFO -  DEBUG: Completed blocker places.sqlite#1: GuidHelper.getItemId (10) for phase places.sqlite#1: waiting for clients
[task 2022-02-04T12:44:59.568Z] 12:44:59     INFO -  DEBUG: Adding blocker places.sqlite#1: PlacesUIUtils: maybeAddImportButton (11) for phase places.sqlite#1: waiting for clients
[task 2022-02-04T12:44:59.568Z] 12:44:59     INFO -  DEBUG: Completed blocker places.sqlite#1: PlacesUIUtils: maybeAddImportButton (11) for phase places.sqlite#1: waiting for clients
[task 2022-02-04T12:44:59.598Z] 12:44:59     INFO -  FATAL ERROR: Non-local network connections are disabled and a connection attempt to firefox.settings.services.mozilla.com (99.84.70.86) was made.
[task 2022-02-04T12:44:59.599Z] 12:44:59     INFO -  You should only access hostnames available via the test networking proxy (if running mochitests) or from a test-specific httpd.js server (if running xpcshell tests). Browser services should be disabled or redirected to a local server.
[task 2022-02-04T12:44:59.600Z] 12:44:59     INFO -  ExceptionHandler::GenerateDump cloned child 1285
[task 2022-02-04T12:44:59.601Z] 12:44:59     INFO -  ExceptionHandler::SendContinueSignalToChild sent continue signal to child
[task 2022-02-04T12:44:59.602Z] 12:44:59     INFO -  ExceptionHandler::WaitForContinueSignal waiting for continue signal...
[task 2022-02-04T12:44:59.706Z] 12:44:59     INFO -  Exiting due to channel error.
[task 2022-02-04T12:44:59.706Z] 12:44:59     INFO -  Exiting due to channel error.
[task 2022-02-04T12:44:59.779Z] 12:44:59     INFO - mozcrash Downloading symbols from: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/DLhUmc7BQvGmiK7lVwdC6A/artifacts/public/build/target.crashreporter-symbols.zip
[task 2022-02-04T12:45:03.046Z] 12:45:03     INFO - mozcrash Copy/paste: /builds/worker/fetches/minidump_stackwalk/minidump_stackwalk --symbols-url=https://symbols.mozilla.org/ --human /tmp/tmp7bt971lo.mozrunner/minidumps/70b63d60-ee71-b530-c4be-2a434c62d352.dmp /tmp/tmp4646dgwq
[task 2022-02-04T12:45:04.970Z] 12:45:04     INFO - mozcrash Saved minidump as /builds/worker/workspace/build/blobber_upload_dir/70b63d60-ee71-b530-c4be-2a434c62d352.dmp
[task 2022-02-04T12:45:04.971Z] 12:45:04     INFO - mozcrash Saved app info as /builds/worker/workspace/build/blobber_upload_dir/70b63d60-ee71-b530-c4be-2a434c62d352.extra
[task 2022-02-04T12:45:05.207Z] 12:45:05     INFO - PROCESS-CRASH | marionette.py | application crashed [@ mozilla::net::nsSocketTransport::InitiateSocket()]
[task 2022-02-04T12:45:05.207Z] 12:45:05     INFO - Mozilla crash reason: MOZ_CRASH(Attempting to connect to non-local address!)
[task 2022-02-04T12:45:05.207Z] 12:45:05     INFO - Crash dump filename: /tmp/tmp7bt971lo.mozrunner/minidumps/70b63d60-ee71-b530-c4be-2a434c62d352.dmp
[task 2022-02-04T12:45:05.208Z] 12:45:05     INFO - Operating system: Linux
[task 2022-02-04T12:45:05.209Z] 12:45:05     INFO -                   0.0.0 Linux 4.4.0-1014-aws #14taskcluster1-Ubuntu SMP Tue Apr 3 10:27:00 UTC 2018 x86_64
[task 2022-02-04T12:45:05.209Z] 12:45:05     INFO - CPU: amd64
[task 2022-02-04T12:45:05.210Z] 12:45:05     INFO -      family 6 model 85 stepping 4
[task 2022-02-04T12:45:05.210Z] 12:45:05     INFO -      2 CPUs
[task 2022-02-04T12:45:05.211Z] 12:45:05     INFO - Linux Ubuntu 18.04 - bionic (Ubuntu 18.04.6 LTS)
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO - 
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO - Crash reason:  SIGSEGV / SEGV_MAPERR
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO - Crash address: 0x0
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO - Process uptime: not available
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO - 
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO - 
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO - Thread 3  (crashed)
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO -  0  libxul.so!mozilla::net::nsSocketTransport::InitiateSocket() [nsSocketTransport2.cpp:1ffcbf88fb9bd58441f3e436f6cd6757c5b31199 : 1237 + 0x11]
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO -     rax = 0x00007fe625b97156   rdx = 0x00007fe63284a8b0
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO -     rcx = 0x0000557e3a3caff8   rbx = 0x00007fe605935a80
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO -     rsi = 0x000000000000017d   rdi = 0x00007fe619bfb980
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO -     rbp = 0x00007fe619bfe3a8   rsp = 0x00007fe619bfe160
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO -      r8 = 0x000000000000017d    r9 = 0x000000000000000b
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO -     r10 = 0x00000000fffffff5   r11 = 0x0000000000000000
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO -     r12 = 0x0000000000000000   r13 = 0x00007fe609191400
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO -     r14 = 0x0000000000000000   r15 = 0x00007fe609191610
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO -     rip = 0x00007fe62462708f
[task 2022-02-04T12:45:05.212Z] 12:45:05     INFO -     Found by: given as instruction pointer in context
[task 2022-02-04T12:45:05.213Z] 12:45:05     INFO -  1  libxul.so!mozilla::net::nsSocketEvent::Run() [nsSocketTransport2.cpp:1ffcbf88fb9bd58441f3e436f6cd6757c5b31199 : 94 + 0x1c4]
[task 2022-02-04T12:45:05.213Z] 12:45:05     INFO -     rbx = 0x0000000000000000   rbp = 0x0000000000000000
[task 2022-02-04T12:45:05.213Z] 12:45:05     INFO -     rsp = 0x00007fe619bfe4d0   r12 = 0x0000000000000000
[task 2022-02-04T12:45:05.213Z] 12:45:05     INFO -     r13 = 0x00007fe609191400   r14 = 0x0000000000000000
[task 2022-02-04T12:45:05.214Z] 12:45:05     INFO -     r15 = 0x0000000000000000   rip = 0x00007fe6246332b1
[task 2022-02-04T12:45:05.214Z] 12:45:05     INFO -     Found by: call frame info
[task 2022-02-04T12:45:05.214Z] 12:45:05     INFO -  2  libxul.so!NS_ProcessNextEvent(nsIThread*, bool) [nsThreadUtils.cpp:1ffcbf88fb9bd58441f3e436f6cd6757c5b31199 : 467 + 0x544]
[task 2022-02-04T12:45:05.214Z] 12:45:05     INFO -     rbx = 0x0000000000000000   rbp = 0x0000000000000001
[task 2022-02-04T12:45:05.215Z] 12:45:05     INFO -     rsp = 0x00007fe619bfe530   r12 = 0x0000000000000000
[task 2022-02-04T12:45:05.215Z] 12:45:05     INFO -     r13 = 0x00007fe61e20e2b0   r14 = 0x00000054a4b0cf8d
[task 2022-02-04T12:45:05.215Z] 12:45:05     INFO -     r15 = 0x0000000000000000   rip = 0x00007fe623b75e34
[task 2022-02-04T12:45:05.215Z] 12:45:05     INFO -     Found by: call frame info

Can you please take a look ?
Thank you !

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This is most likely because of:
https://searchfox.org/mozilla-central/rev/bb14d901ac16633801b7f4adaa4fb104e6f072e4/js/xpconnect/src/xpcpublic.h#740

In Marionette we do not set this preference. So maybe MOZ_MARIONETTE could be evaluated here? But this change would be kinda tough because a lot of different areas could be affected too.

Comment 2

[Julian Descottes [:jdescottes]]

As discuss on elements, we might be hitting
https://searchfox.org/mozilla-central/rev/bb14d901ac16633801b7f4adaa4fb104e6f072e4/services/settings/Utils.jsm#60-72

  get SERVER_URL() {
    const env = Cc["@mozilla.org/process/environment;1"].getService(
      Ci.nsIEnvironment
    );
    const isXpcshell = env.exists("XPCSHELL_TEST_PROFILE_DIR");
    const isNotThunderbird = AppConstants.MOZ_APP_NAME != "thunderbird";
    return AppConstants.RELEASE_OR_BETA &&
      !Cu.isInAutomation &&
      !isXpcshell &&
      isNotThunderbird
      ? "https://firefox.settings.services.mozilla.com/v1"
      : gServerURL;
  },

As Henrik mentioned, since we can't set isInAutomation for those tests, we would need to use something else. As we can see xpcshell tests are excluded by checking the env variable XPCSHELL_TEST_PROFILE_DIR. If MOZ_MARIONETTE seems too broad, we can create a new environment variable explicitly from geckoinstance.py (we do have MOZ_MARIONETTE_REQUIRED_PREFS which we introduced recently, but it feels hacky to reuse it here)

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The perma failure was fixed by the backout. So lets continue on bug 1272255.

Comment 4

[Intermittent Failures Robot]

5 failures in 3165 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 5

Platform and build breakdown:

• linux1804-64-asan-qr: 5
  • opt: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1753649&startday=2022-01-31&endday=2022-02-06&tree=all