The default bug view has changed. See this FAQ.

Make it easier for non-Mozilla devs to get and use the JavaScript Shell

RESOLVED FIXED in mozilla7

Status

()

Core
JavaScript Engine
--
enhancement
RESOLVED FIXED
7 years ago
4 years ago

People

(Reporter: humph, Assigned: humph)

Tracking

(Blocks: 4 bugs)

Trunk
mozilla7
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: fixed-in-bs)

Attachments

(3 attachments, 2 obsolete attachments)

(Assignee)

Description

7 years ago
I love the command line js shell.  Lately I've been working with a lot of web devs, and converting them over to using it.  We've built a whole bunch of automated test infrastructure using it, for example.  But, it's currently too hard for people who don't regularly build mozilla-central to obtain.  In much the same way that the python shell makes interactive development easier and more enjoyable, I would love to see a way to point more people at the js shell.

Some people will say that I should have them use rhino.  That is one option, but its java dependencies make it almost as hard to get running.  I also know about the various in-browser versions of this.  I'm specifically talking about making it easier for more people to do what I do every day, and type ./js

If there are issues standing in the way of making this happen, I'd be glad to look for some students to work on fixing them on the way to making this happen.
>In much the same way that the python shell makes interactive development
>easier and more enjoyable, I would love to see a way to point more people
>at the js shell.

Sounds great to me. What specifically do you think we should do? Building a JS shell from source is pretty easy, although I don't know if we have easy directions posted in an easy-to-design place. Or do you think we should have a binary distribution?
(Assignee)

Comment 2

7 years ago
Building this isn't hard if you're setup to do it, no.  But setting up a build environment (esp on Windows) for this is going to cause many pure web-devs to forget it.

So I think that having binaries is important.  I'm not sure that it needs to keep pace with trunk or anything, since there won't be an update mechanism.  But a binary of the last major release would be awesome.

For Linux distros (or even on Mac with Mac Ports), we could consider getting something in there to build it from source.
I thought we packaged a shell in our nightlies.  Maybe that's xpcshell, but ./js is tiny enough that we could add it without much pain, I suspect.
We stick xpcshell in the test package, but not js. Of course, that makes for a pretty big download if all you want is the shell. We could pretty easily just zip up the js shell by itself and upload it alongside everything else.
My expectation is that many such developers would already have a Firefox install, and therefore could just look in it for the shell.  No?
If we shipped the js shell in the Firefox package, that would work, yes. We don't currently do so. (We don't ship xpcshell in there either, we upload it separately in the test package.)
(Assignee)

Comment 7

7 years ago
What needs to happen next on this?  Is there something I can help do?
(In reply to comment #7)
> What needs to happen next on this?  Is there something I can help do?

How about a list of what things need to be included in a binary distro? Such as:

- a 'js' binary built with what flags? threadsafe, tracevis, etc?
- multiple 'js' binaries built with different flags?
- js static/dynamic libraries?
- documentation?
- sample JS programs?
- perf tools?
- debugging tools?
- documentation?
- anything else?
(Assignee)

Comment 9

7 years ago
Likely there are various use cases, and your list seems to cover many, if not all of them.  So as to not turn this into a project scoped too large to get done, I'd suggest a first effort might include these basics from your list:

- a single 'js' binary ("default" flags)
- simple documentation on the shell itself, perhaps just a README with links to articles on MDC
- some simple example js programs, perhaps to demonstrate features of our js engine

The other items you list all sound great, but will they turn this into a bigger/later project?  If I look at Rhino as a comparison, I see:

- js binary
- src, build tools, tests
- examples

I don't think we need to ship src or tests with this, since that is readily available now, and we're targeting people who can't easily build the src (e.g., web developers).
I agree, keep it simple is the way to go. Would you mind filing bugs on the README and the example programs? 

Once we have those in place do we just want to do the builds manually on a few key platforms, or should we try to get that automated? Or farm it out to whoever has platform X?

The last question is, where do we host the downloads? Who can help us with that?
(Assignee)

Updated

7 years ago
Depends on: 555287
(Assignee)

Updated

7 years ago
Depends on: 555288
(Assignee)

Comment 11

7 years ago
> I agree, keep it simple is the way to go. Would you mind filing bugs on the
> README and the example programs? 

Done.

> Once we have those in place do we just want to do the builds manually on a few
> key platforms, or should we try to get that automated? Or farm it out to
> whoever has platform X?

It's not up to me, but if this isn't automated, it won't happen again.  Ideally we would make this available on the same platforms as Firefox.  Can this get tied into our build/release process?
Our 1.9.2 and trunk builds already build the js shell
by default. It'd be simple to tar or zip up the binary and upload it alongside the firefox package.
(Assignee)

Comment 13

7 years ago
We were discussing this on irc the other day, and I just wanted to add a note here that getting the "fast new js in Firefox 4" out there as a shell devs can use, too, would be very awesome.
(Assignee)

Updated

6 years ago
Whiteboard: [contrib-engagement]

Updated

6 years ago
Blocks: 652719
Whiteboard: [contrib-engagement]
(Assignee)

Comment 14

6 years ago
Watching projects like the spidermonkey node stuff happening, I'm even more interested in seeing this happen.  Given that I'm the one asking for this, I'll volunteer to do it if I can get Ted/Dave to give me some guidance (I'll find you on mail/irc).
Assignee: general → david.humphrey
Status: NEW → ASSIGNED
(Assignee)

Comment 15

6 years ago
14:06 < ted> so, what i'd do is hook into "make package" somewhere and package up a separate js.zip or js.gz or whatever
14:06 < ted> and then just add it to the UPLOAD_FILES list, and it will get uploaded alongside our other bits
(Assignee)

Comment 16

6 years ago
OK, I've got the plumbing in place to do this now.  Can a JS person help me understand which bits, besides dist/bin/js, need to get put into the packaged zip?  e.g., which libraries need to go with it, so that the js binary will run?
(Assignee)

Comment 17

6 years ago
Ted helped me on irc, adding these now:

@executable_path/libplds4.dylib (compatibility version 1.0.0, current version 1.0.0)
@executable_path/libplc4.dylib (compatibility version 1.0.0, current version 1.0.0)
@executable_path/libnspr4.dylib (compatibility version 1.0.0, current version 1.0.0)
(Assignee)

Comment 18

6 years ago
Created attachment 537569 [details] [diff] [review]
Package jsshell with make package
Attachment #537569 - Flags: review?(ted.mielczarek)
Comment on attachment 537569 [details] [diff] [review]
Package jsshell with make package

Review of attachment 537569 [details] [diff] [review]:
-----------------------------------------------------------------

Once you fix up a few things, you should push this to try. If it works, I think you ought to get your JS bin package in your try build directory.

::: toolkit/mozapps/installer/packager.mk
@@ +80,5 @@
>  SDK_PATH = sdk/
>  endif
>  SDK_SUFFIX    = $(PKG_SUFFIX)
>  SDK           = $(SDK_PATH)$(PKG_BASENAME).sdk$(SDK_SUFFIX)
> +JSSHELL_BIN   = $(DIST)/bin/js$(BIN_SUFFIX) \

maybe BINS (plural?)

@@ +81,5 @@
>  endif
>  SDK_SUFFIX    = $(PKG_SUFFIX)
>  SDK           = $(SDK_PATH)$(PKG_BASENAME).sdk$(SDK_SUFFIX)
> +JSSHELL_BIN   = $(DIST)/bin/js$(BIN_SUFFIX) \
> +                $(DIST)/bin/libplds4$(DLL_SUFFIX) \

s/lib/$(LIB_PREFIX)/ (so this works on Windows)

@@ +84,5 @@
> +JSSHELL_BIN   = $(DIST)/bin/js$(BIN_SUFFIX) \
> +                $(DIST)/bin/libplds4$(DLL_SUFFIX) \
> +                $(DIST)/bin/libplc4$(DLL_SUFFIX) \
> +                $(DIST)/bin/libnspr4$(DLL_SUFFIX) \
> +                $(NULL)

Our (undocumented) Makfile style is two-space indent for continuations, like:
JSSHELL_BIN = \
  $(DIST)/bin/js$(BIN_SUFFIX) \
  ...
  $(NULL)

@@ +100,5 @@
>  PKG_SUFFIX	= .tar
>  INNER_MAKE_PACKAGE 	= $(CREATE_FINAL_TAR) - $(MOZ_PKG_DIR) > $(PACKAGE)
>  INNER_UNMAKE_PACKAGE	= $(UNPACK_TAR) < $(UNPACKAGE)
>  MAKE_SDK = $(CREATE_FINAL_TAR) - $(MOZ_APP_NAME)-sdk > $(SDK)
> +MAKE_JSSHELL = $(CREATE_FINAL_TAR) - $(JSSHELL_BIN) -h > $(JSSHELL_PKG)

If you don't want to go through all this trouble, you can just make it a zip file on all platforms. Shouldn't matter, since it's a new thing.

@@ +679,5 @@
>  	$(SYSINSTALL) $(IFLAGS1) $(MOZ_PKG_REMOVALS_GEN) $(DIST)/$(STAGEPATH)$(MOZ_PKG_DIR)$(_BINPATH)
>  endif # MOZ_PKG_REMOVALS
> +# Package JavaScript Shell
> +	@echo "Packaging JavaScript Shell..."
> +	$(MAKE_JSSHELL)

I think we build the JS shell in all build configurations now, but I'm not 100% sure. This is probably okay.
Attachment #537569 - Flags: review?(ted.mielczarek) → review+
(Assignee)

Updated

6 years ago
Blocks: 663108
(Assignee)

Comment 20

6 years ago
Created attachment 538344 [details] [diff] [review]
Patch v2

Fixed nits, switched to zip for all platforms, fixed per-platform library dependencies.  Try server builds pass, and resulting shells work:

https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/david.humphrey@senecac.on.ca-476a0609ae75/
Attachment #537569 - Attachment is obsolete: true
Attachment #538344 - Flags: review?(ted.mielczarek)
Comment on attachment 538344 [details] [diff] [review]
Patch v2

Review of attachment 538344 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good!

::: toolkit/mozapps/installer/packager.mk
@@ +95,5 @@
> +  $(DIST)/bin/$(LIB_PREFIX)plc4$(DLL_SUFFIX) \
> +  $(NULL)
> +endif
> +JSSHELL_PKG   = $(DIST)/jsshell.zip
> +MAKE_JSSHELL  = $(ZIP) -9j $(JSSHELL_PKG) $(JSSHELL_BINS)

Does this wind up with relative paths in your zip? (like bin/js.exe) Is that what you want?
Attachment #538344 - Flags: review?(ted.mielczarek) → review+
(Assignee)

Comment 22

6 years ago
Thanks for the reviews.  I don't know, I'm torn.  I think a zip can dump 3 or 4 files into the current dir.  Let's take it as is, I think.
(Assignee)

Comment 23

6 years ago
Removing these other bugs, as I don't think they really need to block.  I'll fix them separately.
No longer depends on: 555287, 555288
http://hg.mozilla.org/projects/build-system/rev/d12dda590f4a
Whiteboard: fixed-in-bs
(Assignee)

Updated

6 years ago
Blocks: 555287
(Assignee)

Updated

6 years ago
Blocks: 555288
(Assignee)

Comment 25

6 years ago
I'm just looking at https://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/ and wondering if I need to put an OS name on jsshell.zip (jsshell-$(some os var).zip) so that they don't overwrite each other in that combined dir.  The try builds are split into per-OS dirs.

Ted?
Yeah, good call. You can stick a $(MOZ_PKG_PLATFORM) in there. If you'd like to avoid duplicating that junk, you can define a variable for the zip filename in package-name.mk where we define all the other stuff.
Can you make it end with 'jsshell.zip' instead of putting the platform between 'jsshell' and the file extension?  If you prefer that naming scheme, we need to change the relevant part of automation.

http://hg.mozilla.org/build/buildbotcustom/file/default/process/factory.py#l189
(Assignee)

Comment 28

6 years ago
Created attachment 538696 [details] [diff] [review]
Add platform name to jsshell package

This is on top of what you landed yesterday, and changes the name of the jsshell package to include the platform.
Attachment #538696 - Flags: review?(ted.mielczarek)
(Assignee)

Comment 29

6 years ago
Created attachment 538697 [details] [diff] [review]
Buildbot changes for platform name addition

John, I don't think putting the platform name first makes sense.  Here's a fix for the platform name as done in my latest patch.
Attachment #538697 - Flags: review?(jhford)
Comment on attachment 538696 [details] [diff] [review]
Add platform name to jsshell package

You need to change the mention in UPLOAD_FILES as well.

John: why does the automation need to know about this file anyway?
Attachment #538696 - Flags: review?(ted.mielczarek) → review+
(Assignee)

Comment 31

6 years ago
Created attachment 538698 [details] [diff] [review]
Add platform name to jsshell package + upload fix

Thanks Ted.  Fixed that other use.
Attachment #538696 - Attachment is obsolete: true
Attachment #538698 - Flags: review?(ted.mielczarek)
Attachment #538698 - Flags: review?(ted.mielczarek) → review+
http://hg.mozilla.org/projects/build-system/rev/055e5a11a673
http://hg.mozilla.org/mozilla-central/rev/d12dda590f4a
http://hg.mozilla.org/mozilla-central/rev/055e5a11a673
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Attachment #538697 - Flags: review?(jhford) → review+
Comment on attachment 538697 [details] [diff] [review]
Buildbot changes for platform name addition

http://hg.mozilla.org/build/buildbotcustom/rev/8b5fc9186d03 landed on default and will roll into production later today/very early tomorrow as releases permit (currently running the 3.6.18 release).
Attachment #538697 - Flags: checkin+
Blocks: 663796
Backed out of m-c because of bug 663796. The buildbot patch here should fix that, but I can't leave m-c broken until that gets reconfigured. We'll validate that it's fixed on build-system and these can reland on m-c.

http://hg.mozilla.org/mozilla-central/rev/479e2681c25f
http://hg.mozilla.org/mozilla-central/rev/4540ce8760b6
http://hg.mozilla.org/mozilla-central/rev/d36cf8e6da68
http://hg.mozilla.org/mozilla-central/rev/6e42ce13b93d
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(In reply to comment #30)
> John: why does the automation need to know about this file anyway?

it only needs to know so that it doesn't assign that url to the packageUrl.  I am writing up a bug to fix parse_make_upload to hopefully avoid this in future.
(In reply to comment #36)
> (In reply to comment #30)
> > John: why does the automation need to know about this file anyway?
> 
> it only needs to know so that it doesn't assign that url to the packageUrl. 
> I am writing up a bug to fix parse_make_upload to hopefully avoid this in
> future.

filed 663825
The big problem with the js shell is that it's not suitable for widespread use, because the debugging functions it provides are dangerous. It's been a mistake from linux distros (and here I'm talking with my Debian hat on) to ship the js shell as it is. If we want to ship a js shell, we should ship one that is minimalistic, not the one that currently exists.
http://hg.mozilla.org/projects/build-system/rev/76b7d4ad1659
http://hg.mozilla.org/projects/build-system/rev/925918881390
http://hg.mozilla.org/mozilla-central/rev/76b7d4ad1659
http://hg.mozilla.org/mozilla-central/rev/925918881390
Status: REOPENED → RESOLVED
Last Resolved: 6 years ago6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla7
Depends on: 667295
(In reply to comment #38)
> The big problem with the js shell is that it's not suitable for widespread
> use, because the debugging functions it provides are dangerous.

I don't really now much about the details here, but I trust your experience. That being said, we're not shipping this with much visibility at the moment, so the right course of action is probably to file a followup on creating a new shell that's more suitable, perhaps related to the much-needed refactoring that bug (that I can't find right now) about how js.cpp and xpcshell have so much copy-pasted code.
(In reply to comment #38)
> The big problem with the js shell is that it's not suitable for widespread
> use, because the debugging functions it provides are dangerous. It's been a
> mistake from linux distros (and here I'm talking with my Debian hat on) to
> ship the js shell as it is. If we want to ship a js shell, we should ship
> one that is minimalistic, not the one that currently exists.

I'd love to know more details about this, including what should be removed from the shell to make it "safe".
(In reply to comment #42)
> (In reply to comment #38)
> > The big problem with the js shell is that it's not suitable for widespread
> > use, because the debugging functions it provides are dangerous. It's been a
> > mistake from linux distros (and here I'm talking with my Debian hat on) to
> > ship the js shell as it is. If we want to ship a js shell, we should ship
> > one that is minimalistic, not the one that currently exists.
> 
> I'd love to know more details about this, including what should be removed
> from the shell to make it "safe".

I don't remember the details. I might have that somewhere on my archives, but I remember this was brought a long time ago on #jsapi. Anyways, the best IMO would be to start from scratch and add what would be required for the shell to be useful, not as a js engine debugging tool, but as a generic javascript script excution environment. Apart from a few functions for console and files, I don't think there's much that is required... But as Ted notes, we should file a new bug for that.
Blocks: 668452

Updated

6 years ago
Blocks: 676209
Blocks: 897407
You need to log in before you can comment on or make changes to this bug.