Closed Bug 964303 Opened 11 years ago Closed 11 years ago

Restore js shell executable in SpiderMonkey obj dir

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: till, Unassigned)

Details

Recently, the js shell's obj dir layout has changed considerably. Mostly, the changes are positive or neutral, but there's one that is negatively impacting at least my workflow: the change to put the resulting executable under js/src/shell/js. It seems weird to me to put executables under a src dir, but that isn't the problem. What *is* a problem is having [obj dir]/js be a directory instead of the executable I'm interested in. Can we put the js directory somewhere else and link to the executable in the main obj dir, or maybe even put it there again? I understand that it is nice and clean to have all build sub-targets output to their individual directories; maybe that could happen in something like a "build" sub dir?
How about adding a symbolic link to /dist/bin/js in the output directory?
/dist/bin/js is itself a symbolic link to js/src/shell/js, so that boils down to the same thing as what I'm proposing. The problem is that the name "js" is taken by the directory. If that could be renamed, and the symbolic link created, everything would be just splendid.
> maybe that could happen in something like a "build" sub dir? That's what the objdir is, really. If you really want the shell in the objdir, there are only two "reasonable" solutions: rename the js source directory (and no, renaming it in the objdir only doesn't work), or rename the shell binary. But the real question is why can't you run objdir/dist/bin/js? Also note you can build with --with-dist-prefix=/some/directory, in which case the shell will be in /some/directory/bin/js.
Kind of relatedly, also note that the js source tree could benefit from js/src/* moving under js/
(In reply to Mike Hommey [:glandium] from comment #3) > That's what the objdir is, really. If you really want the shell in the > objdir, there are only two "reasonable" solutions: rename the js source > directory (and no, renaming it in the objdir only doesn't work), or rename > the shell binary. Can you elaborate on why that doesn't work? > > But the real question is why can't you run objdir/dist/bin/js? Also note you > can build with --with-dist-prefix=/some/directory, in which case the shell > will be in /some/directory/bin/js. People have deeply-engrained workflows that have js being where it used to be as a part. This change breaks them. If there's any way to revert that, I think it's better to do that than to swallow the loss in energy and productivity that goes with changing those workflows. (For me personally, the fact that selfhosted.js is yet somewhere else is annoying, too, but I doubt there are other people whose workflow that breaks, so I'll deal with it.)
How about we add the ability to |mach run-js| (similar to |mach run| for the firefox binary in dist/bin/) and then people can use that?
While I'd love for shell builds to be supported by mach, this isn't a catch-all solution either. SpiderMonkey developers have various different shell builds - I currently have five - with different compilation configurations. One usual workflow is to have various shells open where you compile different configurations, and one shell where you are in /js/src/test or /jit-test and invoke tests with these various shells. At least for me, switching between them quickly, and switching over to the shell where I'm dealing with a particular configuration and running tests or arbitrary js scripts in there, while building in between is elementary. And that's where the loss in energy comes into play: this is already a lot of juggling (with the switches to the editor thrown into the mix, of course), so any further complications make this less productive. I only created this bug after trying to adapt, realizing that that'll be hard, and talking to other people and hearing similar things.
If PWD is inside an object directory, mach commands should always interact with that object directory. If you are producing multiple binaries in your object directory (e.g. you create one, copy it, then produce another), be forewarned that you are playing with fire. The build system is slowly gaining the ability to prune "untracked" files from the object directory during builds. It's only a matter of time before your custom copied files are pruned as part of the build. We /could/ build exceptions into the mix to facilitate local development. We'll cross that bridge later.
It's not about having multiple binaries in the same object dir. That'd be terrible. It's about having various tailor-made scripts that interact with the one binary, and expect it to be in the object dir itself - as do the workflows of SpiderMonkey devs. Now, if you say that it'd be really difficult to change the layout as it is now, then that's one thing. But if keeping it this way is a matter of principle, then I very much think that the loss in productivity should override that.
It's probably 2 or 3 lines of Python to get the path of a binary out of the build system internals. Or at least it could be. For non-Python scripts, we could add a mach command to dump paths or extend |mach environment| to dump things in a machine readable manner. What I'm trying to say is parts of the objdir are effectively a black box. Different people have different expectations about what is stable. None of this is documented too well. It's tough to strike a balance between developers' desire to be "close to the metal" and build system maintainers' desire to move fast.
(In reply to Till Schneidereit [:till] from comment #5) > (In reply to Mike Hommey [:glandium] from comment #3) > > That's what the objdir is, really. If you really want the shell in the > > objdir, there are only two "reasonable" solutions: rename the js source > > directory (and no, renaming it in the objdir only doesn't work), or rename > > the shell binary. > > Can you elaborate on why that doesn't work? Because we don't have anything in the build system to tell it "hey, the source directory for this is js/src, but i want it in something/else in the objdir". Obviously, there's nothing preventing us from implementing it, but that would be a lot of work for just one use-case. Backing out bug 950298 is not an option. We've allowed the js build system to be a drag for too long. (In reply to Till Schneidereit [:till] from comment #9) > It's not about having multiple binaries in the same object dir. That'd be > terrible. It's about having various tailor-made scripts that interact with > the one binary, and expect it to be in the object dir itself - as do the > workflows of SpiderMonkey devs. The shell has always also been in objdir/dist/bin which means scripts can be updated and will still work when bisecting. Muscle memory is a different story. Note that chances are the shell won't be anywhere else than objdir/dist/bin in the near future.
(In reply to Mike Hommey [:glandium] from comment #11) > > Can you elaborate on why that doesn't work? > > Because we don't have anything in the build system to tell it "hey, the > source directory for this is js/src, but i want it in something/else in the > objdir". Obviously, there's nothing preventing us from implementing it, but > that would be a lot of work for just one use-case. Backing out bug 950298 is > not an option. We've allowed the js build system to be a drag for too long. Thanks, that's the kind of explanation I was looking for. While I'm not happy about having to adapt my workflow, I'll cope.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WONTFIX
(In reply to Gregory Szorc [:gps] from comment #8) > If you are producing multiple binaries in your object directory (e.g. you > create one, copy it, then produce another), be forewarned that you are > playing with fire. The build system is slowly gaining the ability to prune > "untracked" files from the object directory during builds. It's only a > matter of time before your custom copied files are pruned as part of the > build. We /could/ build exceptions into the mix to facilitate local > development. We'll cross that bridge later. I don't think we should ever automatically delete user-generated files, even if they are in the object directory. Instead of comparing the contents of the file-system with the build manifest and deleting any extra files, we should compare the current build manifest with the previous build manifest and delete any files that are no longer in the current one. In other words, only delete files that were at one point in the past created by the build system.
(In reply to Michael Shal [:mshal] from comment #13) > I don't think we should ever automatically delete user-generated files, even > if they are in the object directory. Instead of comparing the contents of > the file-system with the build manifest and deleting any extra files, we > should compare the current build manifest with the previous build manifest > and delete any files that are no longer in the current one. In other words, > only delete files that were at one point in the past created by the build > system. While I agree, we're far from having everything under control of manifests.
Indeed. By the time we have everything under the control of manifests, we'll hopefully be using Tup :)
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.