1405209 - make crashtest/reftest/jstestbrowser all fail

Status: NEW

(Testing :: Reftest, defect)

Description

[Mike Hommey [:glandium]]

STR:
 - Do a local build
 - Run make -C $objdir reftest (or crashtest, or jstestbrowser)

Expected result:
 - The same as mach reftest (or crashtest, or jstestbrowser)

Actual result:
 - Tests are getting stuck on "Access to the file was denied" network errors, which I bet is related to the sandbox.

Comment 1

[(not currently active) Ted Mielczarek]

Do the mach commands to run these tests work? The makefile targets are not well-supported these days (and should probably be removed).

Comment 2

[Gian-Carlo Pascutto [:gcp]]

Bug 1308400 and follow-ups Bug 1386826, bug 1374557 and more added code that modifies the profile to add exceptions to make our testing JS work with sandboxing. Presumably some of the right paths/flags/etc are  only set if you use mach.

Comment 3

[Andrew Halberstadt [:ahal]]

Yes, the mach commands set the sandbox whitelist paths. So if anyone is motivated to fix this, you need to pass in:
--sandbox-read-whitelist <objdir> --sandbox-read-whitelist <topsrcdir>

Alternatively we could make those values the default if not specified.

That said, I'd definitely be in favour of removing the make targets completely. I'd be surprised if there weren't other issues once this one gets fixed. Are there any compelling reasons to use the make target over mach?

Comment 4

[Mike Hommey [:glandium]]

The only compelling reason is scripts that still use the make targets. Why not keep them but make them use mach under the hood?

Comment 5

[Mike Hommey [:glandium]]

That being said, those scripts may or may not be expecting make variables that could be used to tweak the tests to work, and it might be deceiving to keep the targets while not making those keep working. So maybe making the targets return an error message would be the best course of action.