1392640 - assert len(self._whitelistpaths) <= 2 when running mach reftest

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Yury Delendik (:yury)]

When I run `./mach reftest layout/reftests/printing` command I'm getting:

REFTEST INFO | Checking for orphan ssltunnel processes...
REFTEST INFO | Checking for orphan xpcshell processes...
['/Users/yury/Work/mozilla-central/layout/tools/reftest', '/Users/yury/Work/mozilla-central', u'/Users/yury/Work/mozilla-central/obj-x86_64-apple-darwin16.6.0']
Error running mach:

    ['reftest', 'layout/reftests/printing/']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

AssertionError

  File "/Users/yury/Work/mozilla-central/layout/tools/reftest/mach_commands.py", line 205, in run_reftest
    return self._run_reftest(**kwargs)
  File "/Users/yury/Work/mozilla-central/layout/tools/reftest/mach_commands.py", line 234, in _run_reftest
    return reftest.run_desktop_test(**kwargs)
  File "/Users/yury/Work/mozilla-central/layout/tools/reftest/mach_commands.py", line 94, in run_desktop_test
    rv = runreftest.run_test_harness(parser, args)
  File "/Users/yury/Work/mozilla-central/layout/tools/reftest/runreftest.py", line 802, in run_test_harness
    return reftest.runTests(options.tests, options)
  File "/Users/yury/Work/mozilla-central/layout/tools/reftest/runreftest.py", line 466, in runTests
    return self.runSerialTests(manifests, options, cmdargs)
  File "/Users/yury/Work/mozilla-central/layout/tools/reftest/runreftest.py", line 709, in runSerialTests
    startAfter=startAfter)
  File "/Users/yury/Work/mozilla-central/layout/tools/reftest/runreftest.py", line 373, in createReftestProfile
    profile = mozprofile.Profile(**kwargs)
  File "/Users/yury/Work/mozilla-central/testing/mozbase/mozprofile/mozprofile/profile.py", line 89, in __init__
    self._internal_init()
  File "/Users/yury/Work/mozilla-central/testing/mozbase/mozprofile/mozprofile/profile.py", line 120, in _internal_init
    assert len(self._whitelistpaths) <= 2

(Notice that the third line is my `print self._whitelistpaths`)

Is it somehow related to my configuration?

Comment 1

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

By reverting bug 1386826, I can run `./mach reftest` locally on Mac.

Comment 2

[Gian-Carlo Pascutto [:gcp]]

Hmm, I extended the whitelistpaths with the objdir, which is conflicting on macOS where a limited number of path preference "slots" are available.

Easily fixed by making the objdir adding conditional on Linux, but I'm going to forward this to Haik to see what the best approach is and how mac deals with the objdir otherwise. If it's by making a link and inferring from there that may work on Linux as well, but I'm not sure what decision the build peers ended up there.

Comment 3

[Haik Aftandilian [:haik]]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #2)
> Hmm, I extended the whitelistpaths with the objdir, which is conflicting on
> macOS where a limited number of path preference "slots" are available.
> 
> Easily fixed by making the objdir adding conditional on Linux, but I'm going
> to forward this to Haik to see what the best approach is and how mac deals
> with the objdir otherwise. If it's by making a link and inferring from there
> that may work on Linux as well, but I'm not sure what decision the build
> peers ended up there.

On Mac, with bug 1380690, we ended up changing the build to always store the source dir and repo dir paths in the Mac Info.plist stored in the .app bundle. So those paths are always available -- Firefox already depends on knowing the path to the .app directory. The first version of patch 1 on the bug used symlinks to point to the source and object dir. There was a little resistance to using symlinks so I switched to the Info.plist solution because that seemed to make sense on Mac and there is already library code to read values out of the plist. Sorry that's not useful on Linux. I think symlinks could be a reasonable approach on the Linux side in the absence of .plist files. I'd recommend revisiting the symlinks discussion with the build peers if we want to try that because it's a pretty simple solution and saves a lot of time by not requiring changes to tests.

The .app bundle approach is what we plan to use until bug 1380416 is addressed and the build doesn't use symlinks.

Comment 5

[Gian-Carlo Pascutto [:gcp]]

Attachment: Bug 1392640 - Only use preference based whitelisting for Linux.

Review commit: https://reviewboard.mozilla.org/r/174312/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/174312/

Comment 6

[Haik Aftandilian [:haik]]

Comment on attachment 8902638 [details]
Bug 1392640 - Only use preference based whitelisting for Linux.

https://reviewboard.mozilla.org/r/174312/#review179642

::: layout/tools/reftest/runreftest.py:310
(Diff revision 1)
>              prefs['layers.acceleration.disabled'] = True
>  
>          sandbox_whitelist_paths = [SCRIPT_DIRECTORY]
> +        if platform.system() == "Linux":
> -        try:
> +            try:
> -            if options.workPath:
> +                if options.workPath:

On Mac, I think we still need the workPath for when tests are run on automation on a packaged build.

Comment 8

[Boris Zbarsky [:bzbarsky]]

I was just about to file another duplicate of this, then debugged it and found this bug via bug 1386826.

This is a blocker for layout work, due to not being able to run tests locally....

Comment 9

[Gian-Carlo Pascutto [:gcp]]

Comment on attachment 8902638 [details]
Bug 1392640 - Only use preference based whitelisting for Linux.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/174312/diff/1-2/

Comment 10

[Haik Aftandilian [:haik]]

Comment on attachment 8902638 [details]
Bug 1392640 - Only use preference based whitelisting for Linux.

https://reviewboard.mozilla.org/r/174312/#review180514

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8db33f829a1e
Only use preference based whitelisting for Linux. r=haik

Comment 12

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/8db33f829a1e