1584639 - undefined shift in src/gfx/cairo/cairo/src/cairoint.h:222

Status: RESOLVED FIXED

(Core :: Graphics, defect, P3)

Description

[Tyson Smith [:tsmith]]

This is triggered when running gtests with an UBSan build.

To enable this check add the following to your mozconfig:

ac_add_options --enable-address-sanitizer
ac_add_options --enable-undefined-sanitizer="shift"
ac_add_options --disable-jemalloc

browser/components/extensions/test/browser/browser_ext_tabs_saveAsPDF.js
src/gfx/cairo/cairo/src/cairoint.h:222:29: runtime error: left shift of 65174 by 16 places cannot be represented in type 'int'
    #0 0x7f3cfb074307 in cpu_to_be32 src/gfx/cairo/cairo/src/cairoint.h:222:29
    #1 0x7f3cfb074307 in be32_to_cpu src/gfx/cairo/cairo/src/cairoint.h:228
    #2 0x7f3cfb074307 in cairo_truetype_font_calculate_checksum src/gfx/cairo/cairo/src/cairo-truetype-subset.c:858
    #3 0x7f3cfb06b319 in cairo_truetype_font_update_entry src/gfx/cairo/cairo/src/cairo-truetype-subset.c:874:29
    #4 0x7f3cfb06b319 in cairo_truetype_font_generate src/gfx/cairo/cairo/src/cairo-truetype-subset.c:913
    #5 0x7f3cfb06b319 in _cairo_truetype_subset_init src/gfx/cairo/cairo/src/cairo-truetype-subset.c:1067
    #6 0x7f3cfb045146 in _cairo_pdf_surface_emit_truetype_font_subset src/gfx/cairo/cairo/src/cairo-pdf-surface.c:4238:14
    #7 0x7f3cfb045146 in _cairo_pdf_surface_emit_unscaled_font_subset src/gfx/cairo/cairo/src/cairo-pdf-surface.c:4654
    #8 0x7f3cfb108b27 in _cairo_sub_font_collect src/gfx/cairo/cairo/src/cairo-scaled-font-subsets.c:590:30
    #9 0x7f3cfb108b27 in _cairo_scaled_font_subsets_foreach_internal src/gfx/cairo/cairo/src/cairo-scaled-font-subsets.c:904
    #10 0x7f3cfb03fc2d in _cairo_pdf_surface_emit_font_subsets src/gfx/cairo/cairo/src/cairo-pdf-surface.c:4704:14
    #11 0x7f3cfb03fc2d in _cairo_pdf_surface_finish src/gfx/cairo/cairo/src/cairo-pdf-surface.c:1626
    #12 0x7f3cfb12bda4 in INT__moz_cairo_surface_finish src/gfx/cairo/cairo/src/cairo-surface.c:728:11
    #13 0x7f3cfb0d815f in _cairo_paginated_surface_finish src/gfx/cairo/cairo/src/cairo-paginated-surface.c:173:2
    #14 0x7f3cfb12bda4 in INT__moz_cairo_surface_finish src/gfx/cairo/cairo/src/cairo-surface.c:728:11
    #15 0x7f3cf4ea8cd4 in mozilla::gfx::PrintTargetPDF::Finish() src/gfx/thebes/PrintTargetPDF.cpp:68:16
    #16 0x7f3cf48a7241 in nsDeviceContext::EndDocument() src/gfx/src/nsDeviceContext.cpp:543:19
    #17 0x7f3cfa258f6d in mozilla::layout::RemotePrintJobParent::RecvFinalizePrint() src/layout/printing/ipc/RemotePrintJobParent.cpp:163:51
    #18 0x7f3cf3be0df9 in mozilla::layout::PRemotePrintJobParent::OnMessageReceived(IPC::Message const&) src/obj-firefox/ipc/ipdl/PRemotePrintJobParent.cpp:302:28
    #19 0x7f3cf388fcd9 in mozilla::dom::PContentParent::OnMessageReceived(IPC::Message const&) src/obj-firefox/ipc/ipdl/PContentParent.cpp:5873:32
    #20 0x7f3cf369e4cc in mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) src/ipc/glue/MessageChannel.cpp:2185:25
    #21 0x7f3cf369b24e in mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) src/ipc/glue/MessageChannel.cpp:2109:9
    #22 0x7f3cf369c4a3 in mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) src/ipc/glue/MessageChannel.cpp:1954:3
    #23 0x7f3cf369ccbe in mozilla::ipc::MessageChannel::MessageTask::Run() src/ipc/glue/MessageChannel.cpp:1985:13
    #24 0x7f3cf26d8216 in nsThread::ProcessNextEvent(bool, bool*) src/xpcom/threads/nsThread.cpp:1225:14
    #25 0x7f3cf26dfe4d in NS_ProcessNextEvent(nsIThread*, bool) src/xpcom/threads/nsThreadUtils.cpp:486:10
    #26 0x7f3cf36a5cb8 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:88:21
    #27 0x7f3cf35d9e77 in RunInternal src/ipc/chromium/src/base/message_loop.cc:315:10
    #28 0x7f3cf35d9e77 in RunHandler src/ipc/chromium/src/base/message_loop.cc:308
    #29 0x7f3cf35d9e77 in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:290
    #30 0x7f3cf9440559 in nsBaseAppShell::Run() src/widget/nsBaseAppShell.cpp:137:27
    #31 0x7f3cfc91d000 in nsAppStartup::Run() src/toolkit/components/startup/nsAppStartup.cpp:276:30
    #32 0x7f3cfcb3d67b in XREMain::XRE_mainRun() src/toolkit/xre/nsAppRunner.cpp:4601:22
    #33 0x7f3cfcb3f6f5 in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) src/toolkit/xre/nsAppRunner.cpp:4736:8
    #34 0x7f3cfcb40cc3 in XRE_main(int, char**, mozilla::BootstrapConfig const&) src/toolkit/xre/nsAppRunner.cpp:4817:21
    #35 0x562bd8c617c8 in do_main src/browser/app/nsBrowserApp.cpp:218:22
    #36 0x562bd8c617c8 in main src/browser/app/nsBrowserApp.cpp:300
    #37 0x7f3d11a3282f in __libc_start_main /build/glibc-LK5gWL/glibc-2.23/csu/../csu/libc-start.c:291
    #38 0x562bd8b82e18 in _start (application/firefox/firefox+0x8ae18)

Comment 1

[Ryan VanderMeulen [:RyanVM]]

Upstream code for this function looks a bit different nowadays:
https://cgit.freedesktop.org/cairo/tree/src/cairoint.h#n235

Don't suppose we can just crib from that to fix this bug?

Comment 2

[Jonathan Kew [:jfkthame]]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #1)

Upstream code for this function looks a bit different nowadays:
https://cgit.freedesktop.org/cairo/tree/src/cairoint.h#n235

Don't suppose we can just crib from that to fix this bug?

That should be fine, AFAICS.

Comment 3

[Ryan VanderMeulen [:RyanVM]]

Testing this locally, I'm able to reproduce a similar-looking UBSan error on this testcase, but the stack is different-ish. Backporting the cairoint.h change from upstream doesn't help.

 0:09.40 TEST_START: browser/components/extensions/test/browser/browser_ext_tabs_saveAsPDF.js
 0:09.44 INFO Entering test bound testSaveAsPDF_saved
 0:10.37 INFO Extension loaded
 0:11.49 gfx/cairo/cairo/src/cairo-fixed-private.h:62:14: runtime error: left shift of negative value -612
 0:12.88     #0 0x7fd232ede4ef in _cairo_clip_intersect_rectangle gfx/cairo/cairo/src/cairo-clip.c
 0:12.88     #1 0x7fd232ede4ef in _cairo_clip_rectangle gfx/cairo/cairo/src/cairo-clip.c:361
 0:12.88     #2 0x7fd232f4c75e in _moz_cairo_recording_surface_create gfx/cairo/cairo/src/cairo-recording-surface.c:164:11
 0:12.89     #3 0x7fd232f09d36 in _create_recording_surface_for_target gfx/cairo/cairo/src/cairo-paginated-surface.c:85:9
 0:12.89     #4 0x7fd232f09d36 in _cairo_paginated_surface_create gfx/cairo/cairo/src/cairo-paginated-surface.c:119
 0:12.89     #5 0x7fd232e793a1 in _cairo_pdf_surface_create_for_stream_internal gfx/cairo/cairo/src/cairo-pdf-surface.c:395:35
 0:12.89     #6 0x7fd22b508418 in mozilla::gfx::PrintTargetPDF::CreateOrNull(nsIOutputStream*, mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> const&) gfx/thebes/PrintTargetPDF.cpp:47:30
 0:12.92     #7 0x7fd231042e6f in nsDeviceContextSpecGTK::MakePrintTarget() widget/gtk/nsDeviceContextSpecG.cpp:162:12
 0:12.93     #8 0x7fd22ad9b806 in nsDeviceContext::InitForPrinting(nsIDeviceContextSpec*) gfx/src/nsDeviceContext.cpp:495:27
 0:12.96     #9 0x7fd2320a90e2 in mozilla::layout::RemotePrintJobParent::InitializePrintDevice(nsTString<char16_t> const&, nsTString<char16_t> const&, int const&, int const&) layout/printing/ipc/RemotePrintJobParent.cpp:73:29
 0:12.96     #10 0x7fd2320a8b28 in mozilla::layout::RemotePrintJobParent::RecvInitializePrint(nsTString<char16_t> const&, nsTString<char16_t> const&, int const&, int const&) layout/printing/ipc/RemotePrintJobParent.cpp:37:7
 0:12.98     #11 0x7fd229d73214 in mozilla::layout::PRemotePrintJobParent::OnMessageReceived(IPC::Message const&) obj-x86_64-pc-linux-gnu/ipc/ipdl/PRemotePrintJobParent.cpp:262:28
 0:13.09     #12 0x7fd2298e7647 in mozilla::dom::PContentParent::OnMessageReceived(IPC::Message const&) obj-x86_64-pc-linux-gnu/ipc/ipdl/PContentParent.cpp:5873:32
 0:13.11     #13 0x7fd22969d0b7 in mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) ipc/glue/MessageChannel.cpp:2185:25
 0:13.11     #14 0x7fd229699383 in mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) ipc/glue/MessageChannel.cpp:2109:9
 0:13.11     #15 0x7fd22969afa9 in mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) ipc/glue/MessageChannel.cpp:1954:3
 0:13.11     #16 0x7fd22969b5e7 in mozilla::ipc::MessageChannel::MessageTask::Run() ipc/glue/MessageChannel.cpp:1985:13
 0:13.13     #17 0x7fd228592e9f in nsThread::ProcessNextEvent(bool, bool*) xpcom/threads/nsThread.cpp:1225:14
 0:13.13     #18 0x7fd2285998f1 in NS_ProcessNextEvent(nsIThread*, bool) xpcom/threads/nsThreadUtils.cpp:486:10
 0:13.13     #19 0x7fd2296a45c2 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) ipc/glue/MessagePump.cpp:110:5
 0:13.15     #20 0x7fd2295be602 in MessageLoop::RunInternal() ipc/chromium/src/base/message_loop.cc:315:10
 0:13.15     #21 0x7fd2295be602 in MessageLoop::RunHandler() ipc/chromium/src/base/message_loop.cc:308
 0:13.15     #22 0x7fd2295be602 in MessageLoop::Run() ipc/chromium/src/base/message_loop.cc:290
 0:13.17     #23 0x7fd230f5d98a in nsBaseAppShell::Run() widget/nsBaseAppShell.cpp:137:27
 0:13.17     #24 0x7fd234acf432 in nsAppStartup::Run() toolkit/components/startup/nsAppStartup.cpp:276:30
 0:13.18     #25 0x7fd234da34cf in XREMain::XRE_mainRun() toolkit/xre/nsAppRunner.cpp:4601:22
 0:13.18     #26 0x7fd234da5b2b in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) toolkit/xre/nsAppRunner.cpp:4736:8
 0:13.18     #27 0x7fd234da6ff0 in XRE_main(int, char**, mozilla::BootstrapConfig const&) toolkit/xre/nsAppRunner.cpp:4817:21
 0:13.18     #28 0x556fd7a11357 in do_main(int, char**, char**) browser/app/nsBrowserApp.cpp:218:22
 0:13.18     #29 0x556fd7a11357 in main browser/app/nsBrowserApp.cpp:300
 0:13.30     #30 0x7fd24318fb6a in __libc_start_main /build/glibc-KRRWSm/glibc-2.29/csu/../csu/libc-start.c:308:16
 0:13.30     #31 0x556fd7932cf9 in _start (obj-x86_64-pc-linux-gnu/dist/bin/firefox+0x3fcf9)
 0:13.30 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior gfx/cairo/cairo/src/cairo-fixed-private.h:62:14 in

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

The priority flag is not set for this bug.
:jbonisteel, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 5

[Lee Salzman [:lsalzman]]

Attachment: Bug 1584639 - silence Cairo shift warnings. r?aosmond

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e2ba562e4321
silence Cairo shift warnings. r=aosmond

Comment 7

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/e2ba562e4321