Last Comment Bug 505907 - Support C++ calling from JSctypes
: Support C++ calling from JSctypes
Status: RESOLVED WONTFIX
[snappy][ts]
:
Product: Core
Classification: Components
Component: js-ctypes (show other bugs)
: unspecified
: x86 Linux
: -- normal with 3 votes (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
: js-ctypes
Mentors:
Depends on: 770880 552533
Blocks: 447581
  Show dependency treegraph
 
Reported: 2009-07-22 16:57 PDT by (dormant account)
Modified: 2016-06-10 01:38 PDT (History)
24 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments

Description (dormant account) 2009-07-22 16:57:14 PDT
Andreas mentioned that this wouldn't be hard. I think we need that as a first step away from xpconnect. Would be nice if someone could figure out how to make it work for calling C++.
Comment 1 Mike Shaver (:shaver -- probably not reading bugmail closely) 2009-07-22 17:08:16 PDT
You want jsctypes: https://wiki.mozilla.org/JSctypes .
Comment 2 (dormant account) 2009-07-22 17:15:38 PDT
(In reply to comment #1)
> You want jsctypes: https://wiki.mozilla.org/JSctypes .

yes, but I want to call C++.
Comment 3 Mike Shaver (:shaver -- probably not reading bugmail closely) 2009-07-22 17:34:27 PDT
PInvoke of course doesn't really do C++, for the same reasons that it's hard for us.  But you could file a bug on the JSctypes project to get whatever subset-of-C++ calling you specifically want (see the bottom of that page for the Bugzilla component info).  Tempted to close this bug in favour of one focused on C++ in the jsctypes component.

Do we have reason to believe that PInvoke would be faster than XPConnect in a useful way?  Just wondering a the [Ts] and [Tsnap] annotations here.
Comment 4 (dormant account) 2009-07-22 17:36:58 PDT
I haven't benchmarked, but I was hoping that it would be cheaper than 0.2ms per xpconnect call on n810. You can close it as a dup if we plan on getting jsctypes C++ subset stuff into the tree.
Comment 5 Mike Shaver (:shaver -- probably not reading bugmail closely) 2009-07-22 17:38:34 PDT
What specific C++ types do you need to manipulate?
Comment 6 (dormant account) 2009-07-22 17:40:10 PDT
i think i'd be ok with basic c types for now, but I want to be able to call virtual methods on a pointer.
Comment 7 Mike Shaver (:shaver -- probably not reading bugmail closely) 2009-07-22 17:42:01 PDT
I think libffi can do that, no?
Comment 8 (dormant account) 2009-07-22 17:54:06 PDT
(In reply to comment #7)
> I think libffi can do that, no?

Dude! I somehow missed that libffi feature. I'll have to test it out.
 I'd close this bug, but not sure which jsctype bug to use instead of this one.
Comment 9 Mike Shaver (:shaver -- probably not reading bugmail closely) 2009-07-22 17:56:10 PDT
I live to serve.
Comment 10 Benjamin Smedberg [:bsmedberg] 2009-07-22 20:14:04 PDT
We just need somebody to shepherd it to testing and completion ;-)
Comment 11 Joshua Cranmer [:jcranmer] 2009-10-26 07:47:22 PDT
For some stuff I'm working on, the ability to use thiscall convention for a call would be wonderful. I don't think libffi supports thiscall conventions, but I should note that xpcom/reflect/xptcall has a large amount of thiscall code ;-).
Comment 12 dwitte@gmail.com 2009-10-26 10:27:05 PDT
This wouldn't be hard to add - for gcc you basically get it for free; for msvc, we already have a libffi_msvc fork, which you could add it to. (Tweaking the existing assembly to pass a pointer in ecx would probably be quite easy.)

The vtable bit would be a bit stickier. To get at the function pointer for an arbitrary virtual method you need two-and-a-half things:

0) The thunk, if any, to get to the derived class. Let's ignore this and assume we have a pointer to the correct instance.

1) The offset of the vtable pointer within the class instance. (Let's assume this is zero for all useful compilers for now, as it is with gcc, but this may not be true.)

2) The offset of the virtual method in question within the vtable. Let's say you know this.

In that case, the minimal ctypes API is something that says "call this function pointer", and you can do the vtable arithmetic yourself. Something more usable might say "call the 5th virtual method on this class", and the endgame would be "here's my class heirarchy and layout; parse it, figure out the arch-dependent offsets, and fetch the function pointer at call time".
Comment 13 Joshua Cranmer [:jcranmer] 2009-10-26 11:33:21 PDT
(In reply to comment #12)
> The vtable bit would be a bit stickier. To get at the function pointer for an
> arbitrary virtual method you need two-and-a-half things:

With suitable hackery, I believe NS_InvokeByIndex can handle arbitrary vtable calls. In any case, the particular use case I'm more concerned about doesn't involve vtable arithmetic.

> In that case, the minimal ctypes API is something that says "call this function
> pointer", and you can do the vtable arithmetic yourself. Something more usable
> might say "call the 5th virtual method on this class", and the endgame would be
> "here's my class heirarchy and layout; parse it, figure out the arch-dependent
> offsets, and fetch the function pointer at call time".

In the true endgame, this could replace start replacing xpconnect in simple cases :-).
Comment 14 dwitte@gmail.com 2010-03-15 15:11:11 PDT
If we're going to scope this, at least initially, to "parity with xpconnect", then we want to go the compiler-generated-metadata route. So at build time, we'd have xpidl call up a new parser -- let's call it ctypesidl -- which takes the class specification and generates JSON for all the class info.

We'd then have a ctypes JS API, intended for internal use and thus subject to evolve, which takes this JSON and creates, say, a ClassType object that has all the virtual methods on it. In the script, you'd include the generated JSON, which would set up the appropriate ClassType objects.

No name mangling, no pretty API for JS consumers, just something minimal. This wouldn't be hard, just a bunch of code to write.
Comment 15 dwitte@gmail.com 2010-03-15 15:21:21 PDT
(In reply to comment #14)
> If we're going to scope this, at least initially, to "parity with xpconnect",

I should clarify, lest anyone explode -- I meant "the ability to call virtual functions as declared in an idl".

Also, if we implement this, we're going from ctypes and xpconnect being mutually exclusive in functionality to having overlap. Therefore we want to make sure there's an upside to doing so. Performance results here would help.
Comment 16 Ted Mielczarek [:ted.mielczarek] 2010-03-15 16:52:39 PDT
Note that we've already got xpidl.py:
http://mxr.mozilla.org/mozilla-central/source/xpcom/idl-parser/

which is being used for quickstubs.
Comment 17 (dormant account) 2011-11-18 14:44:49 PST
No immediate need for this
Comment 18 Robert Kaiser (not working on stability any more) 2011-11-19 06:12:10 PST
(In reply to Taras Glek (:taras) from comment #17)
> No immediate need for this

So just because you don't need it right now, you decide that it will never be fixed?
I thought that would be a reason for setting it to low priority but keep it open as something to be done at some date. If we are serious on replacing XPCOM completely in the long term, we need this or an equivalent, I don't see how we can say it's not in our plans to be done at some point, which is what I though that WONTFIX meant.
Comment 19 Benjamin Smedberg [:bsmedberg] 2011-11-19 08:32:04 PST
I don't think there is value in keeping this bug open: we don't want this to be implemented without a specific understanding of how somebody (internal or external) would actually use it, and deciding what we do and don't want to support. This bug doesn't provide any of that and so I'd like it to stay closed. If anyone wants to think about this in the future, please email mozilla.dev.platform and we can discuss the specifics and get a spec written.
Comment 20 noitidart 2015-03-02 22:22:36 PST
Some vtable work has been done:

https://bugzilla.mozilla.org/show_bug.cgi?id=738501 link here: https://bugzilla.mozilla.org/attachment.cgi?id=608538 - Cleaner (without all the plus signs) copy paste to scratchpad version here: https://github.com/Noitidart/_scratchpad/blob/master/IShellLink%20IPersistFile%20VTBL%20and%20COM.js

This adds IPropertyStore to the IShellLink IPersistFile done above by Tim: https://github.com/Noitidart/_scratchpad/blob/master/IShellLink%20IPersistFile%20WITH%20IPropertyStore%20ctypes.js

Then here is IPropertyStore COM working with windows:
https://github.com/Noitidart/_scratchpad/blob/master/IPropertyStore%20COM%20jsctypes.js
Cuz lack of union, i did it this way to get unsinged integer values: https://github.com/Noitidart/_scratchpad/blob/master/IPropertyStore%20ulVal%20COM%20jsctypes.js

I found this on github: IShellItem: https://github.com/FunkMonkey/Loomo/blob/06a5881a4f520ede092059a4115bf117568b914f/Loomo/chrome/content/modules/Utils/COM/IShellItem.jsm
IFileOperation: https://github.com/FunkMonkey/Loomo/blob/06a5881a4f520ede092059a4115bf117568b914f/Loomo/chrome/content/modules/Utils/COM/IFileOperation.jsm

These are all windows examples. If anyone has any linux/osx examples that would be awesome I've been looking for them.
Comment 21 noitidart 2015-03-02 22:25:00 PST
it would be super cool if someone wrote some examples using XPCOM and other firefox internals via vtable's from js-ctypes. i dont know how ff internals work so i dont know if this is possible. just would be nice :P like to use HashString: https://dxr.mozilla.org/mozilla-central/source/mfbt/HashFunctions.h#294
http://mxr.mozilla.org/mozilla-release/source/widget/windows/WinTaskbar.cpp#254
Comment 22 noitidart 2015-05-23 20:37:52 PDT
I started some documenation on using c++ from js-ctypes i did it for winapi com if anyone can add to it or make a seperate page for C++ that would be awesome:

https://developer.mozilla.org/en-US/docs/Mozilla/js-ctypes/Examples/Using_COM_from_js-ctypes#Converted_js-ctypes_code
Comment 23 manatlan 2016-06-10 01:38:10 PDT
Hi, just a msg here, because I wanted to call DLL/c++, and I've got a lot of trouble ...
But finaly, it works like a charm, with the help of https://github.com/ochameau/jscpptypes
If I had found this pointer before, It could save me 5days at least ;-)
Hope It can help someone in the future !

Note You need to log in before you can comment on or make changes to this bug.