Crash [@ StartRequest] when calling FuncType callback from native code (WNDPROC)

RESOLVED DUPLICATE of bug 915613

Status

()

Core
js-ctypes
--
critical
RESOLVED DUPLICATE of bug 915613
7 years ago
5 years ago

People

(Reporter: nmaier, Unassigned)

Tracking

({crash})

5 Branch
x86
Windows NT
crash
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(crash signature)

Attachments

(1 attachment)

681.82 KB, application/x-xpinstall
Details
(Reporter)

Description

7 years ago
Created attachment 545660 [details]
Crashing extension

This bug was filed from the Socorro interface and is 
report bp-0a70b104-9104-42b4-a767-820732110713 .
============================================================= 
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0

I'm in the process of converting one of my binary components to js-ctypes.
The crash occurs when calling a FuncType.ptr callback function from native code under certain circumstances.

* FuncType.ptr definition:
https://github.com/nmaier/mintrayr/blob/ctypes/modules/platform.jsm#L158
* FuncType.ptr instance:
https://github.com/nmaier/mintrayr/blob/ctypes/modules/platform.jsm#L158
* Corresponding C definition:
https://github.com/nmaier/mintrayr/blob/ctypes/lib_win/tray/tray.h#L7
* Crashing call:
https://github.com/nmaier/mintrayr/blob/ctypes/lib_win/tray/tray.cpp#L148
* Working call (same FuncType.ptr instance):
https://github.com/nmaier/mintrayr/blob/ctypes/lib_win/tray/tray.cpp#L125

Steps to reproduce:
- Install extension from attachment
- Or build yourself: Clone https://github.com/nmaier/mintrayr/tree/ctypes and checkout ctypes branch and build libwin/tray/tray.sln
- Close the window by Alt+F4
- Crash:
|cx->thread| (and hence |t|) is 0 in:
http://mxr.mozilla.org/mozilla-central/source/js/src/jsapi.cpp#835

Expected:
- The call should succeed and the browser window should be put into the tray.

The extension from the attachment contains the dll compiled under a Debug configuration/static crt.

(The extension is a WIP; no need to point out unrelated bugs like incomplete cleanup)
Taking this bug to get it on my radar. It might take a little while before I get to it though, because I don't currently have a win32 dev setup.
Assignee: nobody → bobbyholley+bmo

Comment 2

7 years ago
The crash occurs when calling your native WatchedWindow callback function from JS, so it's probably unrelated to the prior native -> JS call.

Where do the WatchedWindows get added? Where do those callback pointers come from? Where is the declaration for the callback type?

Does the native -> JS call consistently work at the one callsite and not work at the other?

Comment 3

7 years ago
(To clarify, it actually occurs immediately after the native call returns, which might indicate an incorrect function declaration.)

Comment 4

7 years ago
Exactly: your C declaration has WINAPI (which is stdcall), but your ctypes declaration is ctypes.default_abi (which is cdecl). Switch to ctypes.winapi_abi.
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → INVALID
(Reporter)

Comment 5

7 years ago
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #4)
> Exactly: your C declaration has WINAPI (which is stdcall), but your ctypes
> declaration is ctypes.default_abi (which is cdecl). Switch to
> ctypes.winapi_abi.

Actually the definition is:
typedef BOOL (*minimize_callback_t)(void *handle, int reason);
No WINAPI there and the default ABI on windows should be cdecl == ctypes.default_abi.

What's crashing in my debugger is a null pointer dereference because |cx->thread == 0|.

(In reply to Dan Witte (:dwitte) from comment #2)
> Where do the WatchedWindows get added? Where do those callback pointers come
> from? Where is the declaration for the callback type?

Sorry, linked the wrong line in comment 0 for the definition. Should be:
https://github.com/nmaier/mintrayr/blob/ctypes/modules/platform.jsm#L23
The callback pointer is always the same ctypes instance, initialized here:
https://github.com/nmaier/mintrayr/blob/ctypes/modules/platform.jsm#L158

> Does the native -> JS call consistently work at the one callsite and not
> work at the other?

Yep, the two callsites (one working, one crashing, same pointer) are stated in comment 0.
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
I'm busy enough right now that I'm unlikely to get to this in the immediate future. Updating the assignee to reflect reality. :-(
Assignee: bobbyholley+bmo → nobody

Updated

5 years ago
Status: REOPENED → RESOLVED
Last Resolved: 7 years ago5 years ago
Depends on: 915613
Resolution: --- → DUPLICATE
Duplicate of bug: 915613
You need to log in before you can comment on or make changes to this bug.