make crashtest/reftest/jstestbrowser all fail



2 years ago
2 years ago


(Reporter: glandium, Unassigned)


Version 3

Firefox Tracking Flags

(Not tracked)


 - Do a local build
 - Run make -C $objdir reftest (or crashtest, or jstestbrowser)

Expected result:
 - The same as mach reftest (or crashtest, or jstestbrowser)

Actual result:
 - Tests are getting stuck on "Access to the file was denied" network errors, which I bet is related to the sandbox.
Do the mach commands to run these tests work? The makefile targets are not well-supported these days (and should probably be removed).
Bug 1308400 and follow-ups Bug 1386826, bug 1374557 and more added code that modifies the profile to add exceptions to make our testing JS work with sandboxing. Presumably some of the right paths/flags/etc are  only set if you use mach.
Yes, the mach commands set the sandbox whitelist paths. So if anyone is motivated to fix this, you need to pass in:
--sandbox-read-whitelist <objdir> --sandbox-read-whitelist <topsrcdir>

Alternatively we could make those values the default if not specified.

That said, I'd definitely be in favour of removing the make targets completely. I'd be surprised if there weren't other issues once this one gets fixed. Are there any compelling reasons to use the make target over mach?
The only compelling reason is scripts that still use the make targets. Why not keep them but make them use mach under the hood?
That being said, those scripts may or may not be expecting make variables that could be used to tweak the tests to work, and it might be deceiving to keep the targets while not making those keep working. So maybe making the targets return an error message would be the best course of action.
You need to log in before you can comment on or make changes to this bug.