Closed Bug 671266 Opened 9 years ago Closed 6 years ago

Crash [@ StartRequest] when calling FuncType callback from native code (WNDPROC)

Categories

(Core :: js-ctypes, defect, critical)

5 Branch
x86
Windows NT
defect
Not set
critical

Tracking

()

RESOLVED DUPLICATE of bug 915613

People

(Reporter: nmaier, Unassigned)

References

Details

(Keywords: crash)

Crash Data

Attachments

(1 file)

681.82 KB, application/x-xpinstall
Details
Attached file Crashing extension
This bug was filed from the Socorro interface and is 
report bp-0a70b104-9104-42b4-a767-820732110713 .
============================================================= 
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0

I'm in the process of converting one of my binary components to js-ctypes.
The crash occurs when calling a FuncType.ptr callback function from native code under certain circumstances.

* FuncType.ptr definition:
https://github.com/nmaier/mintrayr/blob/ctypes/modules/platform.jsm#L158
* FuncType.ptr instance:
https://github.com/nmaier/mintrayr/blob/ctypes/modules/platform.jsm#L158
* Corresponding C definition:
https://github.com/nmaier/mintrayr/blob/ctypes/lib_win/tray/tray.h#L7
* Crashing call:
https://github.com/nmaier/mintrayr/blob/ctypes/lib_win/tray/tray.cpp#L148
* Working call (same FuncType.ptr instance):
https://github.com/nmaier/mintrayr/blob/ctypes/lib_win/tray/tray.cpp#L125

Steps to reproduce:
- Install extension from attachment
- Or build yourself: Clone https://github.com/nmaier/mintrayr/tree/ctypes and checkout ctypes branch and build libwin/tray/tray.sln
- Close the window by Alt+F4
- Crash:
|cx->thread| (and hence |t|) is 0 in:
http://mxr.mozilla.org/mozilla-central/source/js/src/jsapi.cpp#835

Expected:
- The call should succeed and the browser window should be put into the tray.

The extension from the attachment contains the dll compiled under a Debug configuration/static crt.

(The extension is a WIP; no need to point out unrelated bugs like incomplete cleanup)
Taking this bug to get it on my radar. It might take a little while before I get to it though, because I don't currently have a win32 dev setup.
Assignee: nobody → bobbyholley+bmo
The crash occurs when calling your native WatchedWindow callback function from JS, so it's probably unrelated to the prior native -> JS call.

Where do the WatchedWindows get added? Where do those callback pointers come from? Where is the declaration for the callback type?

Does the native -> JS call consistently work at the one callsite and not work at the other?
(To clarify, it actually occurs immediately after the native call returns, which might indicate an incorrect function declaration.)
Exactly: your C declaration has WINAPI (which is stdcall), but your ctypes declaration is ctypes.default_abi (which is cdecl). Switch to ctypes.winapi_abi.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → INVALID
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #4)
> Exactly: your C declaration has WINAPI (which is stdcall), but your ctypes
> declaration is ctypes.default_abi (which is cdecl). Switch to
> ctypes.winapi_abi.

Actually the definition is:
typedef BOOL (*minimize_callback_t)(void *handle, int reason);
No WINAPI there and the default ABI on windows should be cdecl == ctypes.default_abi.

What's crashing in my debugger is a null pointer dereference because |cx->thread == 0|.

(In reply to Dan Witte (:dwitte) from comment #2)
> Where do the WatchedWindows get added? Where do those callback pointers come
> from? Where is the declaration for the callback type?

Sorry, linked the wrong line in comment 0 for the definition. Should be:
https://github.com/nmaier/mintrayr/blob/ctypes/modules/platform.jsm#L23
The callback pointer is always the same ctypes instance, initialized here:
https://github.com/nmaier/mintrayr/blob/ctypes/modules/platform.jsm#L158

> Does the native -> JS call consistently work at the one callsite and not
> work at the other?

Yep, the two callsites (one working, one crashing, same pointer) are stated in comment 0.
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
I'm busy enough right now that I'm unlikely to get to this in the immediate future. Updating the assignee to reflect reality. :-(
Assignee: bobbyholley+bmo → nobody
Status: REOPENED → RESOLVED
Closed: 9 years ago6 years ago
Depends on: 915613
Resolution: --- → DUPLICATE
Duplicate of bug: 915613
You need to log in before you can comment on or make changes to this bug.