485946 - nsIProcess shows DOS command prompt window when launching batch file

Status: RESOLVED WONTFIX

(Core :: XPCOM, defect)

Description

[Marcus]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b4pre) Gecko/20090329 Shiretoko/3.5b4pre
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b4pre) Gecko/20090329 Shiretoko/3.5b4pre

My extension uses Windows batch files. When launched a DOS command prompt window now opens as of 3.5b4. This did not happen in the 3.0.x series.
It's a major problem for my extension because it is a text-to-speech application which uses command line tools.

Reproducible: Always

Steps to Reproduce:
1. Call a batch script via nslProcess, that's all.



Actual Results:  
Opens a DOS window over the top of the browser until the process has finished.


Expected Results:  
No DOS window should appear, as in the 3.0.x series.


I think it has to do with how the process is launched. It should be in GUI mode to avoid opening the DOS console window.

Here's some code:

------------------

var launchShell = "C:\\myscript.bat";

// create an nsILocalFile for the executable
var launchFile = Components.classes["@mozilla.org/file/local;1"]
                     .createInstance(Components.interfaces.nsILocalFile);
launchFile.initWithPath(launchShell);

// create an nsIProcess
var process = Components.classes["@mozilla.org/process/util;1"]
                        .createInstance(Components.interfaces.nsIProcess);
process.init(launchFile);

var args = [shellScriptPath];
process.run(false, args, args.length);
--------------------

Comment 2

[:Gavin Sharp [email: [e-mail]]]

Looks like this was actually a report of a regression of bug 421751...

Caused by bug 453185?

Comment 3

[Marcus]

Bug 453185 concerns Vista permissions. I get the console window on XP logged in Administrator, so I think they two separate issues.

Comment 4

[Robert Strong (they/them - no direct email)]

I very highly suspect that Bug 453185 caused this since it was a change from using createprocess to using shellexecute and it likely doesn't try to control the window creation and instead uses the OS default.

btw: I understand the desire of not wanting to display a window under some circumstances but IMO it should be the caller that manages it by using a technology that doesn't display console windows (e.g. wscript, non-console binary, etc. vs. batch files).

Whatever the case, this appears to be a regression from Bug 453185

Comment 5

[Mike Beltzner [:beltzner, not reading bugmail]]

I'm sorry, why do we think blocks the release? Which extension is it causing problems for?

Comment 6

[Marcus]

Robert wrote:
>IMO it should be the caller that manages it by using a
>technology that doesn't display console windows (e.g. wscript, non-console
>binary, etc. vs. batch files).


I agree, but unfortunately that approach does not work. I tried several apps which were set to not open a window (i.e. used the hidden flag) or were minimized. Shiretoko on Windows XP will still override that and open a window for console apps.

Secondly, this is more than just cosmetic or simply a small annoyance. The window, being launched via nsIProcess will block communication between the browser extension and the launched process while the window is open. Meaning, Mozilla extension menus and keyboard shortcuts do not work while the window is open. In my extension this means FF cannot tell the process to stop.

It basically means that non-GUI apps cannot be used by Mozilla under Windows XP (and Vista?) except in the rare case the user will interact with a command prompt window.

Marcus

Comment 7

[Marcus]

(In reply to comment #5)
> I'm sorry, why do we think blocks the release? Which extension is it causing
> problems for?

FoxVox:

https://addons.mozilla.org/en-US/firefox/addon/9759

Comment 8

[Mike Beltzner [:beltzner, not reading bugmail]]

Don't think we can block the final release on this; can you try to see if there are workarounds on your end? Perhaps Rob Strong can help you?

If there's a patch that makes us do the right thing, I think we should consider it for 3.5.1

Comment 9

[Dave Townsend [:mossop]]

(In reply to comment #6)
> Robert wrote:
> >IMO it should be the caller that manages it by using a
> >technology that doesn't display console windows (e.g. wscript, non-console
> >binary, etc. vs. batch files).
> 
> 
> I agree, but unfortunately that approach does not work. I tried several apps
> which were set to not open a window (i.e. used the hidden flag) or were
> minimized. Shiretoko on Windows XP will still override that and open a window
> for console apps.

This doesn't seem to be true. I just tried using nsIProcess to execute a small
vbscript using cscript and wscript. In the case of the former a command window
does briefly appear. In the case of the later it does not.

> Secondly, this is more than just cosmetic or simply a small annoyance. The
> window, being launched via nsIProcess will block communication between the
> browser extension and the launched process while the window is open. Meaning,
> Mozilla extension menus and keyboard shortcuts do not work while the window is
> open. In my extension this means FF cannot tell the process to stop.

This is also not the case. I tried executing a simple batch file that contained
just the pause command. The command window opens but Firefox is still perfectly
responsive and can even kill the running batch program.

> It basically means that non-GUI apps cannot be used by Mozilla under Windows XP
> (and Vista?) except in the rare case the user will interact with a command
> prompt window.

All it appears to mean is that non-GUI apps throw up a command window while
they are running, nothing else. I don't think this should block.

That said, it looks like the fix might be trivial. It appears we mis-read the
docs and the confusingly named SEE_NO_CONSOLE might actually force console
window creation rather than stopping it. I'm testing this now.

Comment 10

[Dan Mills [:thunder]]

Note: this bug is a regression of bug 421751.  Weave no longer needs this functionality, however.

Comment 11

[Marcus]

(In reply to comment #9)
> I just tried using nsIProcess to execute a small
> vbscript using cscript and wscript. In the case of the former a command window
> does briefly appear. In the case of the later it does not.


It does for me on XP with batch files and binaries set as hidden. I haven't tried wscript because I wasn't sure that permissions or firewall apps would more likely block it than batch files.


Could you send me your wscript so I can try it as a possible workaround?

 

> This is also not the case. I tried executing a simple batch file that contained
> just the pause command. The command window opens but Firefox is still perfectly
> responsive and can even kill the running batch program.


It is not the browser which becomes unresponsive, it is the extension which is blocked. The menus I mean are the extension's menus and keyboard functions.

 
> That said, it looks like the fix might be trivial. It appears we mis-read the
> docs and the confusingly named SEE_NO_CONSOLE might actually force console
> window creation rather than stopping it. I'm testing this now.


It would be great if can find a fix. AFAIK, it should only be a flag to control window behaviour. Ideally nsIProcess should let the developer specify the flag. 

If shellexecute is being called then is there any reason not to allow the developer access to its functionality?
An external window has no effect on the browser, or does it?

There are some extensions using command line calls to apps like GnuPG, but I guess they haven't tested 3.5 yet.

Comment 12

[Robert Strong (they/them - no direct email)]

(In reply to comment #11)
> (In reply to comment #9)
> > I just tried using nsIProcess to execute a small
> > vbscript using cscript and wscript. In the case of the former a command window
> > does briefly appear. In the case of the later it does not.
> 
> 
> It does for me on XP with batch files and binaries set as hidden.
Not sure what you mean by this... how do you set a batch file or console binary as hidden when using nsIProcess.

Comment 13

[Robert Strong (they/them - no direct email)]

btw:
(In reply to comment #4)
>... 
> btw: I understand the desire of not wanting to display a window under some
> circumstances but IMO it should be the caller that manages it by using a
> technology that doesn't display console windows (e.g. wscript, non-console
> binary, etc. vs. batch files).
You are testing with a batch file and a console application both of which will display a console window. Try testing with a non-console application or anything else that doesn't use a console and I highly suspect you will find that a console window isn't displayed as both Dave and myself have found.

I am also somewhat concerned that another consumer of nsIProcess will want to display a console window and they won't be able to due to this though I fully admit this would be the minority case. On the other hand if a console window is not desired then either the consumer should not be launching a batch file or a console application even though it is much easier to create a batch file, etc. in the first place since that is their behavior. Adding a new property to nsIProcess to control this seems overkill and I'm ok with it being the default.

Comment 14

[Marcus]

(In reply to comment #12)
> Not sure what you mean by this... how do you set a batch file or console binary
> as hidden when using nsIProcess.


Not batch files, just binaries. The console binary can be set to open a window or not on Windows. So I used a binary which hides the console window when launched to be sure it wasn't the binary opening the window. However, nsIProcess in 3.5 would open a window no matter how the binary was set, at least in my tests.

One possible workaround is to create shortcuts (.lnk files) which are set to open minimized (unfortunately they cannot be hidden AFAIK). It's an awful kludge though, and still means a minimized window will flash on the taskbar.
The exact path has to be known and the shortcuts packaged in the XPI though. Either that or create Windows shortcuts from mozilla (no idea how to do that).


Marcus

Comment 15

[Robert Strong (they/them - no direct email)]

(In reply to comment #14)
> (In reply to comment #12)
> > Not sure what you mean by this... how do you set a batch file or console binary
> > as hidden when using nsIProcess.
> 
> 
> Not batch files, just binaries. The console binary can be set to open a window
> or not on Windows. So I used a binary which hides the console window when
> launched to be sure it wasn't the binary opening the window. However,
> nsIProcess in 3.5 would open a window no matter how the binary was set, at
> least in my tests.
That would be expected if it is a console application

> One possible workaround is to create shortcuts (.lnk files) which are set to
> open minimized (unfortunately they cannot be hidden AFAIK). It's an awful
> kludge though, and still means a minimized window will flash on the taskbar.
> The exact path has to be known and the shortcuts packaged in the XPI though.
> Either that or create Windows shortcuts from mozilla (no idea how to do that).
The proper way would be to create a non-console application (e.g. compiled as such, wscript, etc.).

Just so I don't give the wrong impression... I am ok with not showing a window for console apps and fixing this bug as long as it doesn't break anything.

Comment 16

[Marcus]

(In reply to comment #15)
> That would be expected if it is a console application

I'll contact the author of the console app I'm using to see if he can make sure the window is hidden. However, if you say nsIProcess is expected to always open a window, then compiling the binary to keep the window hidden would not help. I'm confused on this point.

 
> The proper way would be to create a non-console application (e.g. compiled as
> such, wscript, etc.).


You don't always have the possibility to compile a new binary, or do you mean that behaviour can be controlled from wscript?


I agree of course that authors who need a console window need the ability to open one. For example "Git for Windows" runs in a console window. If an extension was created for Git repos (which I've thought about), you'd want to open the console at times. At other times, also using Git, you would want to hide the console to benefit from using Mozilla's GUI, so you really need that flexibility.

Comment 17

[Dave Townsend [:mossop]]

(In reply to comment #11)
> > This is also not the case. I tried executing a simple batch file that contained
> > just the pause command. The command window opens but Firefox is still perfectly
> > responsive and can even kill the running batch program.
> 
> 
> It is not the browser which becomes unresponsive, it is the extension which is
> blocked. The menus I mean are the extension's menus and keyboard functions.

Can you please elaborate on this since I don't understand the distinction. The extension runs in the browser, it isn't really possible to block one and not the other. The javascript I used to execute the process continued running after the process was started. Popup menus etc. all seemed to function fine.

Comment 18

[Dave Townsend [:mossop]]

Attachment: testcases

(In reply to comment #9)
> That said, it looks like the fix might be trivial. It appears we mis-read the
> docs and the confusingly named SEE_NO_CONSOLE might actually force console
> window creation rather than stopping it. I'm testing this now.

This doesn't seem to be the fix I was hoping for. Possibly because Firefox has no console of its own the console app has nothing to inherit so it creates one anyway?

I've put together some testcases that we can use to verify any fix. They appear to be correct as they show a fail for the console test app and pass for the gui test app. The makefile needs some ifdef love before if can be checked in.

Comment 19

[Marcus]

(In reply to comment #17)
> Can you please elaborate on this since I don't understand the distinction.


The browser itself is fine, and the extension's menu items show, but all is not well. Here's an example of the extension's problem: my app calls a speech engine from the command line. While the console window is open, selecting the nsIProcess call which would end the process via a batch file does not work. I'm not saying nsIProcess does not work, it may well place the call I'm not sure, only the call is unsuccessful. For some reason the extension cannot communicate when the window is open. If I create a shortcut and set the shortcut to minimize then it works okay.

I'm still concerned that many user's will have wscript blocked by their firewall or anti-virus programs, which generally doesn't happen to batch files. I need a way from Firefox to detect if WSH is enabled and tell the user if it is not. Otherwise I'll get lots of 1-star reviews saying "it doesn't work".

In any case, since I've never used wscript, could you send me the code you used so the console did not show? It probably means rewriting my batch files.

Comment 20

[Dave Townsend [:mossop]]

(In reply to comment #19)
> (In reply to comment #17)
> > Can you please elaborate on this since I don't understand the distinction.
> 
> 
> The browser itself is fine, and the extension's menu items show, but all is not
> well. Here's an example of the extension's problem: my app calls a speech
> engine from the command line. While the console window is open, selecting the
> nsIProcess call which would end the process via a batch file does not work. I'm
> not saying nsIProcess does not work, it may well place the call I'm not sure,
> only the call is unsuccessful. For some reason the extension cannot communicate
> when the window is open. If I create a shortcut and set the shortcut to
> minimize then it works okay.

I'm afraid I still don't understand what you are saying here. What does "selecting the nsIProcess call which would end the process" mean? Are you talking about calling nsIProcess.kill? You say your extension cannot communicate when the window is open. What sort of communication are you referring to?

> I'm still concerned that many user's will have wscript blocked by their
> firewall or anti-virus programs, which generally doesn't happen to batch files.
> I need a way from Firefox to detect if WSH is enabled and tell the user if it
> is not. Otherwise I'll get lots of 1-star reviews saying "it doesn't work".
> 
> In any case, since I've never used wscript, could you send me the code you used
> so the console did not show? It probably means rewriting my batch files.

wscript runs vbscript, not batch files. The code I used wouldn't be of any help to you since it doesn't actually do anything.

Comment 21

[Robert Strong (they/them - no direct email)]

(In reply to comment #16)
> (In reply to comment #15)
> > That would be expected if it is a console application
> 
> I'll contact the author of the console app I'm using to see if he can make sure
> the window is hidden. However, if you say nsIProcess is expected to always open
> a window, then compiling the binary to keep the window hidden would not help.
> I'm confused on this point.
I'm not saying that nsIProcess is expected to always open a window. I am saying that console apps always display a console unless the caller specifically hides the window while a non-console app doesn't have to display ui. For example, launch firefox.exe -silent. So, if you don't want a console window don't use a console app.

Comment 22

[Marcus]

I've been thinking this over. The binary cannot know in advance how it is to be used. So it means building a second binary with a different window behaviour, or coding a parameter to show/hide the window. You don't always have that option. Also, Windows and Linux have different default behaviors for command line apps. Windows opens a window, Linux does not show a window.

What if you are calling a Microsoft command. It is closed source. Why change the code and make new builds when there is a flag intended to control window behaviour?

What if your extension controls a service/daemon, or use rsync. You don't want Windows showing the window. Do you now recompile a special build for mozilla to use? That's only possible if you can change the code, have the knowledge and build environment to do so.

The window behavior flag is part of the nShowCmd parameters passed to ShellExecute and part of the design:

HINSTANCE ShellExecute(      
    HWND hwnd,
    LPCTSTR lpOperation,
    LPCTSTR lpFile,
    LPCTSTR lpParameters,
    LPCTSTR lpDirectory,
    INT nShowCmd
);

Why would Microsoft consistently expose the "window type" flag if it were not intended to be used by the caller application?

What could break in mozilla by exposing the "window type" flag (referred to as "INT nShowCmd" in the API docs) when calling ShellExecute?

Comment 23

[Dave Townsend [:mossop]]

(In reply to comment #22)
> What could break in mozilla by exposing the "window type" flag (referred to as
> "INT nShowCmd" in the API docs) when calling ShellExecute?

Nothing would break as such, except that it would be an API change which we wouldn't take for Firefox 3.5 at this late stage. However I'm not sure such a change would be accepted considering that there isn't a great deal of benefit to it.

Comment 24

[Marcus]

So what should I do? Should I show a dialog to Windows users saying they will have to put up with the flashing console window since it's part of the design of FF 3.5?

Or is there a workaround?
I don't think patching command line apps is a doable solution, especially since many are closed-source.

Comment 25

[Dave Townsend [:mossop]]

(In reply to comment #24)
> Or is there a workaround?
> I don't think patching command line apps is a doable solution, especially since
> many are closed-source.

The workaround is pretty simple. Run something that doesn't create a command window that in turn runs your command line app in a way that doesn't create a command window. A simple app using the ShellExecute method would do the trick. I suspect there is a way to do it in a few lines of vbscript too.

Comment 26

[Bob Davies]

I've been trying to workaround this issue all day (as I'm unable to recode/rebuild the exe I'd like to call), but am hoping to prevent adding anything too 'bodgy' like async vbs scripts as call-marshalls.

I found this post (not sure who the author is):
http://jamesboston.ca/cms/?q=node/68
Which seems to suggest the problem is fixed by his patch, though my knowledge of C++ is limited at best. Is anyone able to confirm the status of this bug please?
Cheers.

Comment 27

[Dave Townsend [:mossop]]

(In reply to comment #26)
> I found this post (not sure who the author is):
> http://jamesboston.ca/cms/?q=node/68
> Which seems to suggest the problem is fixed by his patch, though my knowledge
> of C++ is limited at best. Is anyone able to confirm the status of this bug
> please?
> Cheers.

That patch was the initial revision of bug 442393 which has already been fixed and landed, so no it does not solve the problem.

Comment 28

[Marcus]

(In reply to comment #26)
> I've been trying to workaround this issue all day (as I'm unable to
> recode/rebuild the exe I'd like to call), but am hoping to prevent adding
> anything too 'bodgy' like async vbs scripts as call-marshalls.


My only option was to use VB script. The good news is it works, fairly easily. The bad news is that VB script is blocked by many anti-virus and personal firewalls. So I get bug reports from users saying the extension does not work because for them nothing happens. I have a big notice about it in the extensions listing.

You can tell people to turn off the AV and firewall monitoring but many do not know how to do that. In fact, I tried myself and with the version of McAfee installed I could only either switch VB monitoring off or leave it on. Then you get alerts from various security monitors which in turn worries users. So VB script is not a good option in my experience.

Comment 29

[Bob Davies]

Is this bug likely to be resolved anytime soon? I'd really like to avoid an unreliable kludge with vb script if at all possible.

Comment 30

[Dave Townsend [:mossop]]

(In reply to comment #29)
> Is this bug likely to be resolved anytime soon? I'd really like to avoid an
> unreliable kludge with vb script if at all possible.

I do not believe anyone is working on it at this time.

Comment 31

[Marcus]

VB is not a real solution anyway. Windows firewalls and anti-virus scanners block the scripts. Average users have little chance to remove the blocks themselves because it's too complicated. Result, they dump the application. I had great difficulty removing the blocks myself for testing on Windows.

What would it take for this bug to become of interest? Should we ask users to comment in this thread? The reason you don't hear about this bug from end users is that end users don't know it is a Firefox bug, so it is not reported as one.

I could place a link in the app to this thread for users who encounter the problem. Would that help if there is more feedback?

Comment 32

[Dan Mills [:thunder]]

Honestly... sorry to <me too> here, but wow, I can't believe this is still open.

When I pushed to get this fixed in bug 421751, it was unbelievable to me that I even had to fix it then.  That it regressed and is still unfixed is completely perplexing to me.  This is absolutely trivial to do in other environments, e.g. in perl you just `use backticks`. It's that easy.

Then again, I guess I don't care enough to write patch, so maybe I should shut up ;)

But seriously, Marcus (or anyone), if you write a patch I will pester folks to get it reviewed and landed.

Comment 33

[Dave Townsend [:mossop]]

(In reply to comment #31)
> VB is not a real solution anyway. Windows firewalls and anti-virus scanners
> block the scripts. Average users have little chance to remove the blocks
> themselves because it's too complicated. Result, they dump the application. I
> had great difficulty removing the blocks myself for testing on Windows.

This can be worked around with a normal executable too of course.

> What would it take for this bug to become of interest? Should we ask users to
> comment in this thread? The reason you don't hear about this bug from end users
> is that end users don't know it is a Firefox bug, so it is not reported as one.
>
> I could place a link in the app to this thread for users who encounter the
> problem. Would that help if there is more feedback?

Not really no, it would likely just make the bug spammy and unusable. Since
there are fairly straightforward workarounds available I can't see this ever
becoming a priority unless someone who knows how to fix the code needs this for
something they are working on.

The fix is not necessarily straightforward but probably not too difficult. It's just a shame that the efforts to replace nsIProcess with a better API haven't gotten very far yet.

Comment 34

[Bob Davies]

Aye, my extension is still in closed beta, and 60% of testers are complaining about how annoying the console popover stealing focus is. (It's importing lots of images and using an executable to read and write lots of metadata asynchronously so causes the console to appear hundreds of times.) 
The use of either vb script or of executables such as cmdow or hstart have common and well-known problems with lots of anti-virus/anti-malware software blocking, or alerting about them.
Sadly I don't have the resources (or honestly the skills) to rewrite and recompile the code in a non-console mode, so really want to find a decent solution to this problem soon, I just wish I had the skills to patch it myself :)

Comment 35

[Dan Mills [:thunder]]

I don't know about cmdow or hstart, but it should be relatively simple to write a wrapper binary that calls your current app.  You don't need to rewrite your entire app.

Comment 36

[Michel Gutierrez]

To any interested developer: I would gladly pay a reasonable amount of money to get that bug fixed. If someone is interested, i can be reached at http://www.downloadhelper.net/partnership.php

Comment 37

[Marcus]

I have tried wrappers. The problem is that the way nsIProcess is being called will always open a command prompt window. You have to call the wrapper, so the wrapper will open the window. If anyone knows a workaround please post it.

All sorts of solutions *should* work in theory, but in practice they do not. We need a working fix. Maybe some Windows expert out there knows a trick?

Comment 38

[Dave Townsend [:mossop]]

(In reply to comment #37)
> I have tried wrappers. The problem is that the way nsIProcess is being called
> will always open a command prompt window. You have to call the wrapper, so the
> wrapper will open the window. If anyone knows a workaround please post it.

What you need is an executable that is compiled to not show a command prompt window. You can do this by making sure your executable contains the WinMain function instead of the usual main, or force it by compiling with /SUBSYSTEM:WINDOWS. I imagine there are settings controlling that exposed somewhere in Visual Studio.

Comment 39

[Bob Davies]

I've put together a very simple wrapper to CreateProcess which can be used to call executables silently. It's a console app, compiled as a windows app so it doesn't raise a console window. It's a very simple app but should cover most simple cases.
You can get it here: http://www.robertadavies.co.uk/2010/04/silentstart/
My knowledge of C/C++ is very limited, so I'd appreciate some eyes on the code (included in the download) and any feedback or improvements are appreciated.

Comment 40

[Alexandre Poirot [:ochameau]]

Attachment: Implement nsIProcess.hideWindow on Windows, to prevent display of command line app window.

Comment 41

[Alexandre Poirot [:ochameau]]

As we are facing this issue in the simulator when running ADB, I took some time to cook a patch for it.
It doesn't look really nice to add a platform specific flag on nsIProcess,
but that's the only thing I found to make this window disappear.
(I'm not sure we can make it so that it automagically prevent command line window to popup and still display regular GUI applications)

I've also reenabled nsIProcess tests on windows and only disabled the precise test that was failing (bug 582821).

https://tbpl.mozilla.org/?tree=Try&rev=75b7d368c8c3

Comment 42

[Benjamin Smedberg]

Comment 38 is correct. We should not fix this issue by changing nsIProcess, but by changing the executable you are launching.

Comment 43

[Myk Melez [:myk] [@mykmelez]]

That's unfortunate, since we'd rather not modify ADB.  But we'll look for another way to spawn it.

Comment 44

[David]

I have the same problem, I need to execute a command repeatedly but I don't want the window (or the console) to be shown.

Like Comment 38 suggested, I made a small c++ code that executes my script without showing the window. Problem is, the cursor is modified to a launch cursor every time the script is launched and it's pretty annoying because my script is launched every 5 secondes.

So, no, you cannot make an "executable that is compiled to not show a command prompt window". It's not enough to run an application silently because the mouse cursor is modified too...