1792086 - /builds/worker/fetches/vs/vc/tools/msvc/14.16.27023/include/regex:138:1: runtime error: load of value -8193, which is not a valid value for type 'std::regex_constants::match_flag_type'

Status: RESOLVED FIXED

(Core :: Graphics, defect)

Description

[Tyson Smith [:tsmith]]

This is detected on startup on Windows with the enum UBSan check enabled.

To enable this check add the following to your mozconfig:

ac_add_options --enable-undefined-sanitizer="enum"

/builds/worker/fetches/vs/vc/tools/msvc/14.16.27023/include/regex:138:1: runtime error: load of value -8193, which is not a valid value for type 'std::regex_constants::match_flag_type'
    #0 0x7ffe62f8e75a in std::regex_constants::operator&= /builds/worker/fetches/vs/vc/tools/msvc/14.16.27023/include/regex:138
    #1 0x7ffe62f8e75a in std::_Matcher<std::_String_const_iterator<std::_String_val<std::_Simple_types<char> > >,char,std::regex_traits<char>,std::_String_const_iterator<std::_String_val<std::_Simple_types<char> > > >::_Clearf /builds/worker/fetches/vs/vc/tools/msvc/14.16.27023/include/regex:2021
    #2 0x7ffe62f8e75a in std::_Regex_search1<class std::_String_const_iterator<class std::_String_val<struct std::_Simple_types<char>>>, class std::allocator<class std::sub_match<class std::_String_const_iterator<class std::_String_val<struct std::_Simple_types<char>>>>>, char, class std::regex_traits<char>, class std::_String_const_iterator<class std::_String_val<struct std::_Simple_types<char>>>>(class std::_String_const_iterator<class std::_String_val<struct std::_Simple_types<char>>>, class std::_String_const_iterator<class std::_String_val<struct std::_Simple_types<char>>>, class std::match_results<class std::_String_const_iterator<class std::_String_val<struct std::_Simple_types<char>>>, class std::allocator<class std::sub_match<class std::_String_const_iterator<class std::_String_val<struct std::_Simple_types<char>>>>>> *, class std::basic_regex<char, class std::regex_traits<char>> const &, enum std::regex_constants::match_flag_type, class std::_String_const_iterator<class std::_String_val<struct std::_Simple_types<char>>>) /builds/worker/fetches/vs/vc/tools/msvc/14.16.27023/include/regex:2862
    #3 0x7ffe62f3e3e4 in std::regex_search /builds/worker/fetches/vs/vc/tools/msvc/14.16.27023/include/regex:2956
    #4 0x7ffe62f3e3e4 in mozilla::gl::ParseVersion /builds/worker/checkouts/gecko/gfx/gl/GLContext.cpp:234
    #5 0x7ffe62f38ddd in mozilla::gl::GLContext::InitImpl(void) /builds/worker/checkouts/gecko/gfx/gl/GLContext.cpp:557
    #6 0x7ffe62f349e9 in mozilla::gl::GLContext::Init(void) /builds/worker/checkouts/gecko/gfx/gl/GLContext.cpp:324
    #7 0x7ffe62f58431 in mozilla::gl::GLContextEGL::Init(void) /builds/worker/checkouts/gecko/gfx/gl/GLContextProviderEGL.cpp:400
    #8 0x7ffe62f54df1 in mozilla::gl::GLContextEGL::CreateGLContext(class std::shared_ptr<class mozilla::gl::EglDisplay>, struct mozilla::gl::GLContextDesc const &, void *, void *, bool, class nsTSubstring<char> *const) /builds/worker/checkouts/gecko/gfx/gl/GLContextProviderEGL.cpp:747
    #9 0x7ffe62f5e6ee in mozilla::gl::GLContextEGL::CreateEGLPBufferOffscreenContextImpl(class std::shared_ptr<class mozilla::gl::EglDisplay>, struct mozilla::gl::GLContextCreateDesc const &, struct mozilla::gfx::IntSizeTyped<struct mozilla::gfx::UnknownUnits> const &, bool, class nsTSubstring<char> *const) /builds/worker/checkouts/gecko/gfx/gl/GLContextProviderEGL.cpp:1150
    #10 0x7ffe62f631c9 in mozilla::gl::GLContextEGL::CreateEGLPBufferOffscreenContext(class std::shared_ptr<class mozilla::gl::EglDisplay>, struct mozilla::gl::GLContextCreateDesc const &, struct mozilla::gfx::IntSizeTyped<struct mozilla::gfx::UnknownUnits> const &, class nsTSubstring<char> *const) /builds/worker/checkouts/gecko/gfx/gl/GLContextProviderEGL.cpp:1179
    #11 0x7ffe63995a4e in CreateGLContextANGLE /builds/worker/checkouts/gecko/gfx/webrender_bindings/RenderThread.cpp:1249
    #12 0x7ffe63995a4e in CreateGLContext /builds/worker/checkouts/gecko/gfx/webrender_bindings/RenderThread.cpp:1313
    #13 0x7ffe63995a4e in mozilla::wr::RenderThread::CreateSingletonGL(class nsTSubstring<char> &) /builds/worker/checkouts/gecko/gfx/webrender_bindings/RenderThread.cpp:1051
    #14 0x7ffe63985d9f in mozilla::wr::RenderThread::InitDeviceTask(void) /builds/worker/checkouts/gecko/gfx/webrender_bindings/RenderThread.cpp:882
    #15 0x7ffe621d2306 in mozilla::detail::runnable_args_base<0>::Run(void) /builds/worker/checkouts/gecko/dom/media/webrtc/transport/runnable_utils.h:41
    #16 0x7ffe60c4821d in nsThread::ProcessNextEvent(bool, bool *) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:1199
    #17 0x7ffe60c56216 in NS_ProcessNextEvent(class nsIThread *, bool) /builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.cpp:465
    #18 0x7ffe62311ea0 in mozilla::ipc::MessagePumpForNonMainThreads::Run(class base::MessagePump::Delegate *) /builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp:330
    #19 0x7ffe6221f155 in MessageLoop::RunInternal /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:381
    #20 0x7ffe6221f155 in MessageLoop::RunHandler(void) /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:374
    #21 0x7ffe6221ef15 in MessageLoop::Run(void) /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:356
    #22 0x7ffe60c3db0e in nsThread::ThreadFunc(void *) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:384
    #23 0x7ffe7ca2e0d7 in _PR_NativeRunThread /builds/worker/checkouts/gecko/nsprpub/pr/src/threads/combined/pruthr.c:399
    #24 0x7ffe7ca03bab in pr_root /builds/worker/checkouts/gecko/nsprpub/pr/src/md/windows/w95thred.c:139
    #25 0x7ffebce41bb1  (C:\WINDOWS\System32\ucrtbase.dll+0x180021bb1)
    #26 0x7ffe7ce49dc3 in __asan::AsanThread::ThreadStart(unsigned __int64) /builds/worker/fetches/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:277
    #27 0x7ffebe967033  (C:\WINDOWS\System32\KERNEL32.DLL+0x180017033)
    #28 0x7ffe975ee4b0 in mozilla::interceptor::FuncHook<mozilla::interceptor::WindowsDllInterceptor<mozilla::interceptor::VMSharingPolicyShared>,void (*)(int, void *, void *)>::operator() /builds/worker/checkouts/gecko/toolkit/xre/dllservices/mozglue/nsWindowsDllInterceptor.h:150
    #29 0x7ffe975ee4b0 in patched_BaseThreadInitThunk /builds/worker/checkouts/gecko/toolkit/xre/dllservices/mozglue/WindowsDllBlocklist.cpp:582
    #30 0x7ffebf3e26a0  (C:\WINDOWS\SYSTEM32\ntdll.dll+0x1800526a0)

Comment 1

[Sotaro Ikeda [:sotaro]]

Hi Kelsey Gilbert, can you comment to the bug?

Comment 2

[Daniel Holbert [:dholbert]]

(Drive-by comment, after seeing this bug listed in an autonag bugzilla bot's "Bugs with a ni on a bug marked as affecting a released version without activity for the last 1 week for the 2022-10-10" periodic-email):

This looks like a bug in MSVC's implementation of std::regex_search, I think.

The warning is complaining about load of value -8193, which is not a valid value for type 'std::regex_constants::match_flag_type'. And Mozilla's own code is not explicitly passing -8193 here, or working with this match_flag_type at all.

It looks like Mozilla code is just calling std::regex_search here, and we're not passing any numeric values that could be guilty of tripping this warning.

Under-the-hood, MSVC implements regex_search using an internal function called std::_Regex_search1 (stack level 2 in comment 0) which takes an arg of type enum std::regex_constants::match_flag_type, which is supposed to represent configuration options regex matching. So that arg (or a version of it at stack level 1) is probably what's involved here.

Some googling turns up https://learn.microsoft.com/en-us/cpp/standard-library/regex-constants-class?view=msvc-170#match_flag_type which has information about the available options:

enum match_flag_type
    {    // specify matching and formatting rules
    match_default = 0x0000,
    match_not_bol = 0x0001,
    match_not_eol = 0x0002,
    match_not_bow = 0x0004,
    match_not_eow = 0x0008,
    match_any = 0x0010,
    match_not_null = 0x0020,
    match_continuous = 0x0040,
    match_prev_avail = 0x0100,
    format_default = 0x0000,
    format_sed = 0x0400,
    format_no_copy = 0x0800,
    format_first_only = 0x1000,
    _Match_not_null = 0x2000

I have a theory about how we would end up with -8193 here. The value -8193 in hex is 0xffffdfff which can also be expressed as ~0x2000, i.e. ~_Match_not_null if the above enum list can be trusted. So I suspect part of Microsoft's regex implementation is trying to clear that bit (e.g. after testing for and handling it) by doing a bitwise operation with the inverse of that value, e.g. flags &= ~_Match_not_null or something along those lines. And the named literal happens to be treated as if it were signed, so its inverse is signed and negative, and that makes it appear that we're putting a negative value into an unsigned enum type.

(Support for this theory: stack level 1 is a function called _Clearf which sounds like it's clearing a flag; and stack level 0 is operator&= which is the operation that you would use if you were going to clear a bit by bitwise-and'ing it with its inverse.)

In any case, this is unlikely to be "really" undefined since it's in code provided by the compiler itself (in its implementation of the standard libraries), and unlikely to be something we could do anything about (except by wholly avoiding std::regex). Possibly something fixed in newer MSVC versions, or something we should report upstream to Microsoft / MSVC folks about their regex impl.

Tyson, do we have procedures here for suppressing/ignoring UBSan warnings in external code? If so we probably want to bucket this under that category.

Comment 3

[Daniel Holbert [:dholbert]]

A Google search for "UBSan Regex" turns up this blog post from 2020:
https://devblogs.microsoft.com/cppblog/c20-features-and-fixes-in-vs-2019-16-1-through-16-6/
...which mentions the following issue as being fixed in Visual Studio 2019 16.4:

Explicitly specified the underlying types of some enumeration types in <regex> that use bitwise operations to avoid storing an unrepresentable value (which is formally undefined behavior, as noted by Clang’s UBSAN).

That sounds like an exact match for what I described in comment 2. So: hooray, it's fixed upstream, as of 2020!

Presumably we're building with an older MSVC here (I see the paths in comment 0 have 14.16.27023; not sure about version numbering here but maybe this is "version 14" which is less than "version 16" from the blog post?)

Tyson, not sure if comment 0 was your local machine vs. some infrastructure in the cloud; either way, I wonder if you know what MSVC version we're using? The best fix here is probably to install updates to pick up the fix noted in the blog post above, if we can do that.

I think we're safe to call this INVALID as a Firefox bug, though feel free to reopen if there's code changes we do in fact need to make here.

Comment 4

[Tyson Smith [:tsmith]]

Thanks for investigating. This build was done via try: https://treeherder.mozilla.org/jobs?repo=try&revision=16d3c8f48673d6459d238a186532a3aeb8c7b3e1.

Comment 5

[Mathew Hodson]

(In reply to Daniel Holbert [:dholbert] from comment #3)

A Google search for "UBSan Regex" turns up this blog post from 2020:
https://devblogs.microsoft.com/cppblog/c20-features-and-fixes-in-vs-2019-16-1-through-16-6/
...which mentions the following issue as being fixed in Visual Studio 2019 16.4:

Explicitly specified the underlying types of some enumeration types in <regex> that use bitwise operations to avoid storing an unrepresentable value (which is formally undefined behavior, as noted by Clang’s UBSAN).

That sounds like an exact match for what I described in comment 2. So: hooray, it's fixed upstream, as of 2020!

Presumably we're building with an older MSVC here (I see the paths in comment 0 have 14.16.27023; not sure about version numbering here but maybe this is "version 14" which is less than "version 16" from the blog post?)

Builds were switched to Visual Studio 2019 version 16.11 in bug 1832467.