1026958 - Define MOZ_DISABLE_NONLOCAL_CONNECTIONS for the remaining test suites

Status: RESOLVED INCOMPLETE

(Testing :: General, defect)

Description

[Ed Morley [:emorley]]

Now that bug 995417 has landed, we have coverage against gecko attempting to connect to a non-loopback name (eg external resource outside of the build network) whenever MOZ_DISABLE_NONLOCAL_CONNECTIONS is defined. 

https://hg.mozilla.org/mozilla-central/rev/51342b493983 defined it in:
build/automation.py.in
build/automationutils.py
build/mobile/remoteautomation.py
testing/xpcshell/runxpcshelltests.py

However, there are other places we set env defines (see bug 1023483 comment 3), so there are still test suites for which bug 995417 is not enabled.

These try runs:
https://tbpl.mozilla.org/?tree=Try&rev=c4fe727a12d1
https://tbpl.mozilla.org/?tree=Try&rev=03c4764ac26a
(latter for PGO)

...shows that the define is missing for:
* talos (all valid platforms)
* jetpack (all valid platforms)
* marionette (all valid platforms)
* marionette-webapi (all valid platforms)
* gaia-unit (all valid platforms)
* gaia-integration (all valid platforms)
* gaia-ui-test (all valid platforms)
* mochitest (b2g emulator only)
* reftest/crashtest (b2g emulator only)
* xpcshell (b2g emulator only)

This try run removes the check for MOZ_DISABLE_NONLOCAL_CONNECTIONS:
https://tbpl.mozilla.org/?tree=Try&rev=a06dd6870851

...so should show us what will need fixing once we define it everywhere.

Comment 1

[Ed Morley [:emorley]]

> This try run removes the check for MOZ_DISABLE_NONLOCAL_CONNECTIONS:
> https://tbpl.mozilla.org/?tree=Try&rev=a06dd6870851
> 
> ...so should show us what will need fixing once we define it everywhere.

Another try run, except this time with the error message prefixed with "Automation Error:" so that the TBPL parser will find them for us - and the MOZ_CRASH removed, so we can find several in one go:
https://tbpl.mozilla.org/?tree=Try&rev=c088826545b9

Comment 2

[Nathan Froyd [:froydnj]]

(In reply to Ed Morley [:edmorley UTC+0] from comment #0)
> * mochitest (b2g emulator only)
> * reftest/crashtest (b2g emulator only)
> * xpcshell (b2g emulator only)

I can believe this, but I'm also pretty sure that several bugs fixed for bug 995417 were found only on B2G emulator: bug 1023638 (xpcshell) and bug 1024588 (seen during mochitests), for instance.

Comment 3

[Ed Morley [:emorley]]

(In reply to Ed Morley [:edmorley UTC+0] from comment #1)
> Another try run, except this time with the error message prefixed with
> "Automation Error:" so that the TBPL parser will find them for us - and the
> MOZ_CRASH removed, so we can find several in one go:
> https://tbpl.mozilla.org/?tree=Try&rev=c088826545b9

Note that for this run, the jobs appear passing, but that's just because buildbot/mozharness doesn't check for "Automation Error:". Opening the logs does show TBPL-parsable output however, so it's slightly more useful than the comment 0 try run.

(In reply to Nathan Froyd (:froydnj) from comment #2)
> I can believe this, but I'm also pretty sure that several bugs fixed for bug
> 995417 were found only on B2G emulator: bug 1023638 (xpcshell) and bug
> 1024588 (seen during mochitests), for instance.

Yeah that puzzled me; though we use gecko for other parts of the b2g emulator end-to-end job (eg xpcshell; though that should already be covered?), which are what is crashing in that try run, rather than the mochitests/... themselves.

Comment 4

[Ed Morley [:emorley]]

(In reply to Ed Morley (Away 12th-20th July) [:edmorley] from comment #3)
> (In reply to Nathan Froyd (:froydnj) from comment #2)
> > I can believe this, but I'm also pretty sure that several bugs fixed for bug
> > 995417 were found only on B2G emulator: bug 1023638 (xpcshell) and bug
> > 1024588 (seen during mochitests), for instance.
> 
> Yeah that puzzled me; though we use gecko for other parts of the b2g
> emulator end-to-end job (eg xpcshell; though that should already be
> covered?), which are what is crashing in that try run, rather than the
> mochitests/... themselves.

Ah in fixing bug 1038943, khuey found that the env wasn't being passed in correctly. Once bug 1039019 is fixed we'll need to revert his workaround from https://bugzilla.mozilla.org/attachment.cgi?id=8456564&action=diff#a/testing/mochitest/runtestsb2g.py_sec3

Comment 5

[Ed Morley [:emorley]]

Mass-closing old bugs I filed that have not had recent activity/no longer affect me.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Btw a better method is to file bugs for specific harnesses like Marionette (bug 1272255).

Comment 7

[Ed Morley [:emorley]]

This was a meta bug