1407292 - ./mach crashtest and reftest always fails on Mac (assert len(self._whitelistpaths) <= 2)

Status: RESOLVED FIXED

(Testing :: Reftest, defect)

Description

[Jason Orendorff [:jorendorff]]

$ ./mach crashtest dom/base/crashtests/1406109-1.html 
Error running mach:

    ['crashtest', 'dom/base/crashtests/1406109-1.html']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

AssertionError

  File "/Users/jorendorff/work/gecko/layout/tools/reftest/mach_commands.py", line 223, in run_crashtest
    return self._run_reftest(**kwargs)
  File "/Users/jorendorff/work/gecko/layout/tools/reftest/mach_commands.py", line 233, in _run_reftest
    return reftest.run_desktop_test(**kwargs)
  File "/Users/jorendorff/work/gecko/layout/tools/reftest/mach_commands.py", line 91, in run_desktop_test
    rv = runreftest.run_test_harness(parser, args)
  File "/Users/jorendorff/work/gecko/layout/tools/reftest/runreftest.py", line 878, in run_test_harness
    result = reftest.runTests(options.tests, options)
  File "/Users/jorendorff/work/gecko/layout/tools/reftest/runreftest.py", line 518, in runTests
    return self.runSerialTests(manifests, options, cmdargs)
  File "/Users/jorendorff/work/gecko/layout/tools/reftest/runreftest.py", line 782, in runSerialTests
    startAfter=startAfter)
  File "/Users/jorendorff/work/gecko/layout/tools/reftest/runreftest.py", line 367, in createReftestProfile
    profile = mozprofile.Profile(**kwargs)
  File "/Users/jorendorff/work/gecko/testing/mozbase/mozprofile/mozprofile/profile.py", line 90, in __init__
    self._internal_init()
  File "/Users/jorendorff/work/gecko/testing/mozbase/mozprofile/mozprofile/profile.py", line 120, in _internal_init
    assert len(self._whitelistpaths) <= 2

----

On my machine, whitelistpaths is: [
    '/Users/jorendorff/work/gecko/layout/tools/reftest',
    '/Users/jorendorff/work/gecko',
    u'/Users/jorendorff/work/gecko/obj-x86_64-apple-darwin16.7.0'
]

ReftestRunner.run_desktop_test puts the last two paths in that list:
        args.sandboxReadWhitelist.extend([self.topsrcdir, self.topobjdir])
http://searchfox.org/mozilla-central/source/layout/tools/reftest/mach_commands.py#79

RefTest.createReftestProfile adds the first path:
        sandbox_whitelist_paths = [SCRIPT_DIRECTORY] + options.sandboxReadWhitelist
http://searchfox.org/mozilla-central/source/layout/tools/reftest/runreftest.py#311

Comment 1

[Jason Orendorff [:jorendorff]]

I deleted the SCRIPT_DIRECTORY part (redundant in my case) and it worked for me.

Comment 2

[Boris Zbarsky [:bzbarsky]]

Looks like bug 1392640 got regressed by bug 1403325.

The commit message there says "However, this only happens when running on Linux for some reason"... it might have been a good idea to look at the blame for the reason!

Comment 3

[Alex Gaynor [:Alex_Gaynor]]

Since SCRIPT_DIRECTORY should always be a subdirectory of srcdir, I don't think it should be needed on _any_ platform. So removing should leave this working on all platforms

:gcp, if memory serves, you originally wrote this code, does that sound right? Do you remember why we included SCRIPT_DIRECTORY?

Comment 4

[Gian-Carlo Pascutto [:gcp]]

I don't really remember. Odds are that I was trying to find how to get at the needed paths, and SCRIPT_DIRECTORY was the easiest to find, so it went in first.

Comment 5

[Alex Gaynor [:Alex_Gaynor]]

Attachment: Bug 1407292 - don't whitelist the script directory, it is duplicative of the srcdir;

Review commit: https://reviewboard.mozilla.org/r/188126/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/188126/

Comment 6

[Alex Gaynor [:Alex_Gaynor]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1fcf5cd9fed5380d9297c82c99e8892501471346

Comment 7

[Andrew Halberstadt [:ahal]]

Comment on attachment 8917122 [details]
Bug 1407292 - don't whitelist the script directory, it is duplicative of the srcdir;

https://reviewboard.mozilla.org/r/188126/#review193570

::: layout/tools/reftest/runreftest.py:311
(Diff revision 1)
>          # Bug 1262954: For winXP + e10s disable acceleration
>          if platform.system() in ("Windows", "Microsoft") and \
>             '5.1' in platform.version() and options.e10s:
>              prefs['layers.acceleration.disabled'] = True
>  
> -        sandbox_whitelist_paths = [SCRIPT_DIRECTORY] + options.sandboxReadWhitelist
> +        sandbox_whitelist_paths = options.sandboxReadWhitelist

There is no topsrcdir in CI, but your try run looks good so I guess we are already passing in the proper `--sandbox-read-whitelist` directories there.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ec97e4979a62
don't whitelist the script directory, it is duplicative of the srcdir; r=ahal

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/ec97e4979a62