Closed Bug 1587162 Opened 5 years ago Closed 5 years ago

call to function DnsPrefChanged(char const*, nsHostResolver*) through pointer to incorrect function type in modules/libpref/Preferences.cpp:5040

Categories

(Core :: Preferences: Backend, defect, P3)

defect

Tracking

()

VERIFIED FIXED
mozilla72
Tracking Status
firefox71 --- wontfix
firefox72 --- fixed

People

(Reporter: tsmith, Assigned: n.nethercote)

References

(Blocks 1 open bug)

Details

(Keywords: csectype-undefined)

Attachments

(2 files)

This is triggered on start up with an UBSan build. To enable this check add the following to your mozconfig:

ac_add_options --enable-address-sanitizer
ac_add_options --enable-undefined-sanitizer="function"
ac_add_options --disable-jemalloc

This issue can be triggered by running gtests or attempting to launch the browser.

Running GTest tests...
modules/libpref/Preferences.cpp:5040:5: runtime error: call to function DnsPrefChanged(char const*, nsHostResolver*) through pointer to incorrect function type 'void (*)(const char *, void *)'
netwerk/dns/nsHostResolver.cpp:555: note: DnsPrefChanged(char const*, nsHostResolver*) defined here
    #0 0x7fd7f2bb211a in mozilla::Preferences::RegisterCallbackAndCall(void (*)(char const*, void*), nsTSubstring<char> const&, void*, mozilla::Preferences::MatchKind) modules/libpref/Preferences.cpp:5040:5
    #1 0x7fd7f2eb36d2 in nsresult mozilla::Preferences::RegisterCallbackAndCall<20, nsHostResolver>(mozilla::TypedPrefChangeFunc<nsHostResolver>::Type, char const (&) [20], nsHostResolver*) objdir-ff-ubsan/dist/include/mozilla/Preferences.h:437:12
    #2 0x7fd7f2eb36d2 in nsHostResolver::Init() netwerk/dns/nsHostResolver.cpp:613
    #3 0x7fd7f2ec37f3 in nsHostResolver::Create(unsigned int, unsigned int, unsigned int, nsHostResolver**) netwerk/dns/nsHostResolver.cpp:2111:22
    #4 0x7fd7f2ef384c in nsDNSService::Init() netwerk/dns/nsDNSService2.cpp:663:17
    #5 0x7fd7f2ef338a in nsDNSService::GetSingleton() netwerk/dns/nsDNSService2.cpp:544:9
    #6 0x7fd7f2ef32d9 in nsDNSService::GetXPCOMSingleton() netwerk/dns/nsDNSService2.cpp:529:12
    #7 0x7fd7f2a2b831 in mozilla::xpcom::CreateInstanceImpl(mozilla::xpcom::ModuleID, nsISupports*, nsID const&, void**) objdir-ff-ubsan/xpcom/components/StaticComponents.cpp:9202:36
    #8 0x7fd7f2a6b063 in (anonymous namespace)::EntryWrapper::CreateInstance(nsISupports*, nsID const&, void**) xpcom/components/nsComponentManager.cpp:224:46
    #9 0x7fd7f2a6b063 in nsComponentManagerImpl::GetServiceLocked((anonymous namespace)::MutexLock&, (anonymous namespace)::EntryWrapper&, nsID const&, void**) xpcom/components/nsComponentManager.cpp:1383
    #10 0x7fd7f2a61cae in nsComponentManagerImpl::GetService(nsID const&, nsID const&, void**) xpcom/components/nsComponentManager.cpp:1435:10
    #11 0x7fd7f9149fc0 in nsresult CallGetService<nsIDNSService>(nsID const&, nsIDNSService**) objdir-ff-ubsan/dist/include/nsServiceManagerUtils.h:63:10
    #12 0x7fd7f9149fc0 in nsHTMLDNSPrefetch::Initialize() dom/html/nsHTMLDNSPrefetch.cpp:67
    #13 0x7fd7fbd7e64d in nsLayoutStatics::Initialize() layout/build/nsLayoutStatics.cpp:196:8
    #14 0x7fd7fbd7e479 in nsLayoutModuleInitialize() layout/build/nsLayoutModule.cpp:111:7
    #15 0x7fd7f2a62f2b in nsComponentManagerImpl::Init() xpcom/components/nsComponentManager.cpp:493:5
    #16 0x7fd7f2b26a61 in NS_InitXPCOM xpcom/build/XPCOMInit.cpp:445:51
    #17 0x7fd7f18e7659 in ScopedXPCOM::ScopedXPCOM(char const*, nsIDirectoryServiceProvider*) objdir-ff-ubsan/dist/include/testing/TestHarness.h:85:19
    #18 0x7fd7f18e6d2e in mozilla::RunGTestFunc(int*, char**) testing/gtest/mozilla/GTestRunner.cpp:113:15
    #19 0x7fd7fef5f89d in XREMain::XRE_mainStartup(bool*) toolkit/xre/nsAppRunner.cpp:3788:16
    #20 0x7fd7fef6e551 in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) toolkit/xre/nsAppRunner.cpp:4722:12
    #21 0x7fd7fef6fdfb in XRE_main(int, char**, mozilla::BootstrapConfig const&) toolkit/xre/nsAppRunner.cpp:4816:21
    #22 0x55c9aa692145 in do_main(int, char**, char**) browser/app/nsBrowserApp.cpp:218:22
    #23 0x55c9aa6913ff in main browser/app/nsBrowserApp.cpp:300:16

There are more instances of this, anybody who passes a function pointer whose last argument is not void* would fail this check: https://searchfox.org/mozilla-central/search?q=symbol:_ZN7mozilla11Preferences23RegisterCallbackAndCallENS_19TypedPrefChangeFuncIT0_E4TypeERAT__KcPS2_&redirect=false

The priority flag is not set for this bug.
:njn, could you have a look please?

For more information, please visit auto_nag documentation.

Flags: needinfo?(n.nethercote)

Huh. I'll write a patch to fix this (and similar cases) tomorrow.

Flags: needinfo?(n.nethercote)
Priority: -- → P3

I hit a different error when I run it locally, in nsCOMArray.cpp:

/home/njn/moz/au1/xpcom/ds/nsCOMArray.cpp:103:10: runtime error: call to function (unknown) through pointer to incorrect function type 'int (*)(nsISupports *, nsISupports *, void *)'
(/home/njn/moz/au1/d64ubsan/dist/bin/libxul.so+0x1433b440): note: (unknown) defined here
    #0 0x7efe26cf7068  (/home/njn/moz/au1/d64ubsan/dist/bin/libxul.so+0x13e68068)
    #1 0x7efe26d3c80d  (/home/njn/moz/au1/d64ubsan/dist/bin/libxul.so+0x13ead80d)
    #2 0x7efe26cf7309  (/home/njn/moz/au1/d64ubsan/dist/bin/libxul.so+0x13e68309)
    #3 0x7efe27235b9a  (/home/njn/moz/au1/d64ubsan/dist/bin/libxul.so+0x143a6b9a)
    ...
    #37 0x5642c11e3029  (/home/njn/moz/au1/d64ubsan/dist/bin/firefox+0x19d029)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/njn/moz/au1/xpcom/ds/nsCOMArray.cpp:103:10 in 

And then execution halts. So it's hard for me to check if my change is good. I did go through all the examples like dmajor suggested, there are quite a few without a void* arg, I'll fix them on Monday.

/home/njn/moz/au1/xpcom/ds/nsCOMArray.cpp:103:10: runtime error: call to function (unknown) through pointer to incorrect function type 'int (*)(nsISupports *, nsISupports *, void *)'

The nsISupports makes me think it might be bug 1587176. Would you be interested in fixing that one too?

I can take a look at that one too. Any ideas why I wasn't getting useful code locations in my stack trace on my Linux box? I tried running it through tools/rb/fix-linux-stack.py, but that didn't help.

Flags: needinfo?(twsmith)
Attached file mozconfig

(In reply to Nicholas Nethercote [:njn] from comment #6)

I can take a look at that one too. Any ideas why I wasn't getting useful code locations in my stack trace on my Linux box? I tried running it through tools/rb/fix-linux-stack.py, but that didn't help.

I'm not sure why the line numbers would be missing. Here is a copy of the mozconfig I have been using on a few (Linux) machines without any issues.

Flags: needinfo?(twsmith)

Lots of these callbacks have a non-void* final parameter, which UBSAN
complains about. This commit changes them to have a void* parameter.

This requires undoing the machinery added in the first two commits of bug
1473631: TypePrefChangeFunc and PREF_CHANGE_METHOD. The resulting code is
simpler (which is good) and more boilerplate-y (which is bad) but avoids the
undefined behaviour (which is good).

Tyson: I tried your mozconfig but still didn't get line numbers :( Still, this patch should fix the problem.

Pushed by nnethercote@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/30ee2029152c
Fix UBSAN complaints about pref callbacks. r=erahm
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla72
Assignee: nobody → n.nethercote

I can confirm this does fix the issue. Thanks!

Status: RESOLVED → VERIFIED
Regressions: 1593426

For posterity: I finally got filenames/line numbers working with ASAN_SYMBOLIZER_PATH=~/.mozbuild/clang/bin/llvm-symbolizer.

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: