593484 - Add "this process" library handle to js-ctypes

Status: NEW

(Core :: js-ctypes, enhancement)

Description

[Wesley W. Garland]

It would be handy, when using js-ctypes, to just access for symbols in the current process space.

This would insulate users against worrying about which libraries certain symbols are in, and worrying about which versions of which libraries were loaded with the browser (or XULRunner) if there are multiple versions of the same libraries on the system.

Concretely, if firefox has an libnspr4 specified with the rpath at link-time that isn't in the LD_LIBRARY_PATH etc that js-ctypes is aware of, it's difficult for an extension developer to say  "I want the PR_Lock() symbol that's already loaded, I don't need one from somewhere else".

We do this with GPSEE's FFI and have found it really convenient; we implement by simply dlopen()ing RTLD_DEFAULT instead of "libraryName.so".   Since we don't support win32, I can't comment about portability there, but it works fine on current versions of Linux and UNIX.

Filing now because it would have solved a bug I saw harth experience in IRC today (unable to open libnspr4 on Firefox 4, presumably because it had moved since Firefox 3.6 to a new location)

suggested syntax: var hnd = ctypes.open(null)

Comment 1

[[e-mail]]

This is useful enough that I think we want it soon.

Comment 2

[Benjamin Smedberg]

Note that this is impossible on Windows: this would be a platform-specific hack. I'm a little skeptical that this will work on Mac either, because of how two-level namespace lookup normally works.

Comment 3

[Wesley W. Garland]

This: http://mxr.mozilla.org/mozilla-central/source/services/crypto/WeaveCrypto.js#110

Is precisely the pattern that I'm hoping this idiom could overcome: firefox already loads libnss (well, presumably) -- why should the extension developer have to figure all this stuff out?

Benjamin: 

It would be unfortunate if this cann't work on Windows, but as I said, I have very little expertise there.

I haven't bumped into any problems on Leopard, even when calling my code from a lightly modified js-shell; but we do have a pretty vanilla compilation environment.  I know you guys have some symbol-hiding magic going on which might cause you to maybe force your dylibs to load RTLD_LOCAL?

Comment 4

[Mike Hommey [:glandium]]

(In reply to comment #3)
> This:
> http://mxr.mozilla.org/mozilla-central/source/services/crypto/WeaveCrypto.js#110
> 
> Is precisely the pattern that I'm hoping this idiom could overcome: firefox
> already loads libnss (well, presumably) -- why should the extension developer
> have to figure all this stuff out?

Note that "all this stuff" can now be reduced to a few lines because all the file name logic can be replaced with ctypes.libraryName("nss3").

That'd make it as simple as:
let nssfile = Services.dirsvc.get("GreD", Ci.nsILocalFile);
nssfile.append(ctypes.libraryName("nss3"));
let nsslib = ctypes.open(nssfile.path);

Though arguably, it could be even simpler. See bug 583209.

But definitely, RTLD_NOLOAD + RTLD_DEFAULT would awesome.

Comment 5

[[e-mail]]

Notes to self:

On Windows we have GetProcAddress / LoadLibrary, both of which are capable of loading a handle from the current process. They require a (relative) library name to work, so the ctypes API would require a library name in conjunction with an RTLD_DEFAULT-like flag.

On Linux we would probably want dlopen(..., RTLD_NOLOAD), since using dlsym(RTLD_DEFAULT, ...) won't give you libraries loaded without RTLD_GLOBAL (note RTLD_LOCAL is the default behavior).

Haven't looked at Mac yet.

Comment 6

[[e-mail]]

Reassigning to nobody. If anyone wants to work on this, feel free!

Comment 7

[Joshua Cranmer [:jcranmer]]

Doing some snooping around Apple documentation to see how hard this would be on Mac, and I find <http://developer.apple.com/library/mac/#documentation/DeveloperTools/Reference/MachOReference/Reference/reference.html#//apple_ref/doc/uid/TP40001398>.

NSAddImage and NSLookupSymbolInImage, along with the name of the loaded shared library, should be a similar route to Windows to do this. I don't have a mac to test it on, though.

Comment 8

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

(In reply to Brian Smith (:briansmith, :bsmith, use NEEDINFO?) from bug 1034979 comment #49)
> IIRC, NSPR has a function that returns a handle to the shared library that
> would be returned during normal dynamic linking for a given symbol such as
> "NSS_Init" or "NSS_Initialize", called PR_FindSymbolAndLibrary. I am not
> sure exactly how it works, but it seems like the best solution is to have
> jsctypes use *that* function, and handle this automatically for you:
> 
> /*
> ** Finds a symbol in one of the currently loaded libraries. Given the
> ** name of a procedure, return the address of the function that
> ** implements the procedure, and return the library that contains that
> ** symbol, or NULL if no such function can be found. This does not find
> ** symbols in the main program (the ".exe"); use PR_AddStaticLibrary to
> ** register symbols in the main program.  
> **
> ** This increments the reference count of the library.
> */
> NSPR_API(void*) PR_FindSymbolAndLibrary(const char *name,
> 						      PRLibrary* *lib);
> 
> /*
> ** Similar to PR_FindSymbolAndLibrary, except that the return value is
> ** a pointer to a function, and not a pointer to void. Casting between a
> ** data pointer and a function pointer is not portable according to the C
> ** standard. Any function pointer can be cast to any other function pointer.
> **
> ** This increments the reference count of the library.
> */
> NSPR_API(PRFuncPtr) PR_FindFunctionSymbolAndLibrary(const char *name,
> 						      PRLibrary* *lib);