1210848 - Add mach commands for working with standalone spidermonkey builds

Status: NEW

(Core :: JavaScript Engine, defect)

Description

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

These should use a different obj dir from the full firefox build; I almost always have both a firefox and spidermonkey shell build at the same time.

I imagine many people have little custom scripts to do this stuff. It would be nice to have a blessed and properly maintained set.

I think this will be especially helpful for people who hack on both SM and gecko, so there will be less of a tooling context switch when going back and forth.

Thinking something along the lines of:

./mach spidermonkey build

./mach spidermonkey run

./mach spidermonkey clobber

./mach spidermonkey style

./mach spidermonkey vanilla-allocations

./mach spidermonkey jit-test [test_file*]

./mach spidermonkey jsapi-test [test_name*]

I have wrapper/helper scripts that pretty much do most of this already and I imagine it shouldn't be too hard to integrate into mach as they are already written in python.

How do folks feel about this?

Comment 1

[Jakob Stoklund Olesen [:jolesen]]

This sounds very useful.

I tend to have many SpiderMonkey object directories for the same source tree: {debug,release} x {arm-sim,arm64-sim,x86-64,x86}. It seems that mach can handle that via .mozconfig files, though.

Comment 2

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

I imagine we would create a new JS_STANDALONE_OBJDIR option that could be handled in .mozconfig files, yes.

Comment 3

[Nicolas B. Pierron [:nbp]]

I think this is a good idea to have a unified set of commands for new contributors, but to be useful for JS Engine developers, I think it should handle the variability of the configure script.

I indeed have a wrapper script, but I made it to compile & test multiple variants sequentially in different object directories. (I recently removed 190 GB of object directory) For example, I often do:

  m x86-x64-dbg-opt-make -skj8

which basically compiles JS shell 4 times (2 architectures x 2 compilation mode).

In total I have 6 different architectures, 2 compilation modes, 12 compilers, and various configuration flags toggle such as gc-zeal, ASAN, and many others …

(In reply to Nick Fitzgerald [:fitzgen][:nf] from comment #0)
> ./mach spidermonkey style
> ./mach spidermonkey jit-test [test_file*]
> ./mach spidermonkey jsapi-test [test_name*]

We have 3 test suites: tests (reftests), jit-test, and jsapi-test (C++)

Currently I don't think the jsapi-test have any way to run a single test alone, except by removing all other from the compilation.

As of today, I constantly run the jit-test and reftests together.  As we have no name clash between the 2 test suites, we can even specify a test case name / directory and get the right set of test running.

So I think we can include all the tests suites under one unique command, with toggles to ignore some of them.

(In reply to Nick Fitzgerald [:fitzgen][:nf] from comment #0)
> ./mach spidermonkey clobber

Is that needed?  Just removing the object directory works well.

(In reply to Nick Fitzgerald [:fitzgen][:nf] from comment #0)
> ./mach spidermonkey vanilla-allocations

What is that suppose to mean?
Do you mean building with --disable-jemalloc ? (see the variability issue raised above …)

Comment 4

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

(In reply to Nicolas B. Pierron [:nbp] from comment #3)
> I think this is a good idea to have a unified set of commands for new
> contributors, but to be useful for JS Engine developers, I think it should
> handle the variability of the configure script.

Yeah, I'm open to suggestions.

That said, I think we can start small and grow from there.

> (In reply to Nick Fitzgerald [:fitzgen][:nf] from comment #0)
> > ./mach spidermonkey style
> > ./mach spidermonkey jit-test [test_file*]
> > ./mach spidermonkey jsapi-test [test_name*]
> 
> We have 3 test suites: tests (reftests), jit-test, and jsapi-test (C++)
> 
> Currently I don't think the jsapi-test have any way to run a single test
> alone, except by removing all other from the compilation.

You can pass the $NAME in BEGIN_TEST($NAME) to run only that test, eg:

  $ ./js/src/jsapi-tests/jsapi-tests testSavedStacks_withNoStack

There is no way to run all tests in a file, AFAIK.

> As of today, I constantly run the jit-test and reftests together.  As we
> have no name clash between the 2 test suites, we can even specify a test
> case name / directory and get the right set of test running.

It probably reflects that we generally work on different corners of spidermonkey, but I feel the need to run reftests *very* rarely while jit-tests are my day to day.

> So I think we can include all the tests suites under one unique command,
> with toggles to ignore some of them.

I'd be happy with `./mach test`-like behavior:

  # runs all tests
  ./mach spidermonkey test 

  # runs only tests in specified dir
  ./mach spidermonkey test js/src/jit-test/tests/debug

Although I think jsapi-tests would have to be their own thing, because of how they aren't file-based but BEGIN/END_TEST-based.

> (In reply to Nick Fitzgerald [:fitzgen][:nf] from comment #0)
> > ./mach spidermonkey clobber
> 
> Is that needed?  Just removing the object directory works well.

If your obj dir is computed based on (for example) your git branch name, then having a helper is very useful.

> (In reply to Nick Fitzgerald [:fitzgen][:nf] from comment #0)
> > ./mach spidermonkey vanilla-allocations
> 
> What is that suppose to mean?

A wrapper for `make check-vanilla-allocations` (I think); Checks for vanilla new/free/etc rather than js_new/js_free/etc.

Comment 5

[Gregory Szorc [:gps]]

Sorry for the delayed response.

As an alternative, you could create your own "mach driver" in js/src. e.g. js/src/mach. A mach driver program is pretty easy to implement. See https://hg.mozilla.org/hgcustom/version-control-tools/file/783405fe84cb/bugzilla for inspiration.

The "mach" script in mozilla-central is substantially more complicated unfortunately :/

Comment 6

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

I was overly optimistic when I thought I could get around to this fairly quickly.