1767836 - Assertion failure: initialBytes + nbytes > initialBytes, at /builds/worker/checkouts/gecko/js/src/gc/Scheduling.h:762 while calling CanvasRenderingContext2D::AddAssociatedMemory()

Status: NEW

(Core :: Graphics: Canvas2D, defect, P3)

Description

[Tyson Smith [:tsmith]]

Attachment: testcase.html

Found while fuzzing m-c 20220430-6e268f45bed2 (--enable-debug --enable-fuzzing)

The fuzzers are seeing this on Android (x86).

Assertion failure: initialBytes + nbytes > initialBytes, at /builds/worker/checkouts/gecko/js/src/gc/Scheduling.h:762

eax = 0xb99f17cb    ebp = 0xbcf076d8    ebx = 0xbb9c9aa0
ecx = 0xbc5edc74    edi = 0xab07802c    edx = 0x00000000
eflags = 0x00210286 eip = 0xb5bbcfec    esi = 0xab078000
esp = 0xbcf076c0
OS|Android|0.0.0 Linux 5.4.86-android11-2-00006-gae78026f427c-ab7595864 #1 SMP PREEMPT Thu Jul 29 20:54:47 UTC 2021 i686
CPU|x86|GenuineIntel family 6 model 6 stepping 3|4
Crash|SIGSEGV / SEGV_MAPERR|0x00000000|17
17|0|libxul.so|js::ZoneAllocator::addCellMemory(js::gc::Cell*, unsigned int, js::MemoryUse)|hg:hg.mozilla.org/mozilla-central:js/src/gc/ZoneAllocator.h:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|71|0xbc
17|1|libxul.so|JS::AddAssociatedMemory(JSObject*, unsigned int, JS::MemoryUse)|hg:hg.mozilla.org/mozilla-central:js/src/jsapi.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|1262|0x4a
17|2|libxul.so|mozilla::dom::CanvasRenderingContext2D::AddAssociatedMemory()|hg:hg.mozilla.org/mozilla-central:dom/canvas/CanvasRenderingContext2D.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|1653|0x2f
17|3|libxul.so|mozilla::dom::CanvasRenderingContext2D::ClearTarget(int, int)|hg:hg.mozilla.org/mozilla-central:dom/canvas/CanvasRenderingContext2D.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|1681|0x65
17|4|libxul.so|mozilla::dom::CanvasRenderingContext2D::SetDimensions(int, int)|hg:hg.mozilla.org/mozilla-central:dom/canvas/CanvasRenderingContext2D.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|1645|0x3c
17|5|libxul.so|mozilla::dom::CanvasRenderingContextHelper::UpdateContext(JSContext*, JS::Handle<JS::Value>, mozilla::ErrorResult&)|hg:hg.mozilla.org/mozilla-central:dom/canvas/CanvasRenderingContextHelper.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|260|0xc8
17|6|libxul.so|mozilla::dom::HTMLCanvasElement::AfterMaybeChangeAttr(int, nsAtom*, bool)|hg:hg.mozilla.org/mozilla-central:dom/html/HTMLCanvasElement.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|590|0x7a
17|7|libxul.so|mozilla::dom::HTMLCanvasElement::AfterSetAttr(int, nsAtom*, nsAttrValue const*, nsAttrValue const*, nsIPrincipal*, bool)|hg:hg.mozilla.org/mozilla-central:dom/html/HTMLCanvasElement.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|569|0x22
17|8|libxul.so|mozilla::dom::Element::SetAttrAndNotify(int, nsAtom*, nsAtom*, nsAttrValue const*, nsAttrValue&, nsIPrincipal*, unsigned char, bool, bool, bool, mozilla::dom::Document*, mozAutoDocUpdate const&)|hg:hg.mozilla.org/mozilla-central:dom/base/Element.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|2583|0x4b5
17|9|libxul.so|mozilla::dom::Element::SetAttr(int, nsAtom*, nsAtom*, nsTSubstring<char16_t> const&, nsIPrincipal*, bool)|hg:hg.mozilla.org/mozilla-central:dom/base/Element.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|2439|0x1b4
17|10|libxul.so|mozilla::dom::Element::SetAttr(int, nsAtom*, nsTSubstring<char16_t> const&, nsIPrincipal*, bool)|hg:hg.mozilla.org/mozilla-central:dom/base/Element.h:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|951|0x30
17|11|libxul.so|mozilla::dom::Element::SetAttribute(nsTSubstring<char16_t> const&, nsTSubstring<char16_t> const&, nsIPrincipal*, mozilla::ErrorResult&)|hg:hg.mozilla.org/mozilla-central:dom/base/Element.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|1476|0x119
17|12|libxul.so|mozilla::dom::Element_Binding::setAttribute(JSContext*, JS::Handle<JSObject*>, void*, JSJitMethodCallArgs const&)|s3:gecko-generated-sources:3c9e7d53efe20dcf6dd7ef537ee6db76d07c9a7ebb00ae1305675355f6404725b2f5f62a7fc34b4804de0c04618b4e5471830368e7ad8081964373afd071d52a/dom/bindings/ElementBinding.cpp:|1518|0x214
17|13|libxul.so|bool mozilla::dom::binding_detail::GenericMethod<mozilla::dom::binding_detail::NormalThisPolicy, mozilla::dom::binding_detail::ThrowExceptions>(JSContext*, unsigned int, JS::Value*)|hg:hg.mozilla.org/mozilla-central:dom/bindings/BindingUtils.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|3270|0x11d
17|14|libxul.so|CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), js::CallReason, JS::CallArgs const&)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|420|0x1c1
17|15|libxul.so|js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|507|0x434
17|16|libxul.so|InternalCall(JSContext*, js::AnyInvokeArgs const&, js::CallReason)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|574|0x7b
17|17|libxul.so|Interpret(JSContext*, js::RunState&)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|3314|0x93c6
17|18|libxul.so|js::RunScript(JSContext*, js::RunState&)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|389|0x18c
17|19|libxul.so|js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|539|0x365
17|20|libxul.so|InternalCall(JSContext*, js::AnyInvokeArgs const&, js::CallReason)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|574|0x7b
17|21|libxul.so|js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>, js::CallReason)|hg:hg.mozilla.org/mozilla-central:js/src/vm/Interpreter.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|605|0xf2
17|22|libxul.so|JS::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>)|hg:hg.mozilla.org/mozilla-central:js/src/vm/CallAndConstruct.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|117|0x173
17|23|libxul.so|mozilla::dom::EventListener::HandleEvent(mozilla::dom::BindingCallContext&, JS::Handle<JS::Value>, mozilla::dom::Event&, mozilla::ErrorResult&)|s3:gecko-generated-sources:6abfc13115b948309184a7e9ce543dc15ead92ad0e3ca71d0317d30e904bb6ec61f75c018d28aff86d703b76fb4e3f03c2b28e269b262119a2f0a86ea09bf0f4/dom/bindings/EventListenerBinding.cpp:|62|0x29e
17|24|libxul.so|void mozilla::dom::EventListener::HandleEvent<mozilla::dom::EventTarget*>(mozilla::dom::EventTarget* const&, mozilla::dom::Event&, mozilla::ErrorResult&, char const*, mozilla::dom::CallbackObject::ExceptionHandling, JS::Realm*)|s3:gecko-generated-sources:ff6cfa65e7a3aa3391e453d54449e1fab039167897fdaca5e6e5ede48a417d6674bc67fbbc56ee14b05d1a66b905fbcc52e118e49e903e4dd2f35db579c48abd/dist/include/mozilla/dom/EventListenerBinding.h:|65|0xdd
17|25|libxul.so|mozilla::EventListenerManager::HandleEventSubType(mozilla::EventListenerManager::Listener*, mozilla::dom::Event*, mozilla::dom::EventTarget*)|hg:hg.mozilla.org/mozilla-central:dom/events/EventListenerManager.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|1310|0xd9
17|26|libxul.so|mozilla::EventListenerManager::HandleEventInternal(nsPresContext*, mozilla::WidgetEvent*, mozilla::dom::Event**, mozilla::dom::EventTarget*, nsEventStatus*, bool)|hg:hg.mozilla.org/mozilla-central:dom/events/EventListenerManager.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|1507|0x586
17|27|libxul.so|mozilla::EventTargetChainItem::HandleEvent(mozilla::EventChainPostVisitor&, mozilla::ELMCreationDetector&)|hg:hg.mozilla.org/mozilla-central:dom/events/EventDispatcher.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|348|0x1a1
17|28|libxul.so|mozilla::EventTargetChainItem::HandleEventTargetChain(nsTArray<mozilla::EventTargetChainItem>&, mozilla::EventChainPostVisitor&, mozilla::EventDispatchingCallback*, mozilla::ELMCreationDetector&)|hg:hg.mozilla.org/mozilla-central:dom/events/EventDispatcher.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|550|0x31e
17|29|libxul.so|mozilla::EventDispatcher::Dispatch(nsISupports*, nsPresContext*, mozilla::WidgetEvent*, mozilla::dom::Event*, nsEventStatus*, mozilla::EventDispatchingCallback*, nsTArray<mozilla::dom::EventTarget*>*)|hg:hg.mozilla.org/mozilla-central:dom/events/EventDispatcher.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|1119|0xc51
17|30|libxul.so|nsDocumentViewer::LoadComplete(nsresult)|hg:hg.mozilla.org/mozilla-central:layout/base/nsDocumentViewer.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|1084|0x81b
17|31|libxul.so|nsDocShell::EndPageLoad(nsIWebProgress*, nsIChannel*, nsresult)|hg:hg.mozilla.org/mozilla-central:docshell/base/nsDocShell.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|6458|0x277
17|32|libxul.so|nsDocShell::OnStateChange(nsIWebProgress*, nsIRequest*, unsigned int, nsresult)|hg:hg.mozilla.org/mozilla-central:docshell/base/nsDocShell.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|5850|0x375
17|33|libxul.so|{virtual override thunk({offset(-260)}, nsDocShell::OnStateChange(nsIWebProgress*, nsIRequest*, unsigned int, nsresult))}|||0x2b
17|34|libxul.so|nsDocLoader::DoFireOnStateChange(nsIWebProgress*, nsIRequest*, int&, nsresult)|hg:hg.mozilla.org/mozilla-central:uriloader/base/nsDocLoader.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|1377|0x144
17|35|libxul.so|nsDocLoader::doStopDocumentLoad(nsIRequest*, nsresult)|hg:hg.mozilla.org/mozilla-central:uriloader/base/nsDocLoader.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|975|0xc8
17|36|libxul.so|nsDocLoader::DocLoaderIsEmpty(bool, mozilla::Maybe<nsresult> const&)|hg:hg.mozilla.org/mozilla-central:uriloader/base/nsDocLoader.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|794|0x60a
17|37|libxul.so|nsDocLoader::OnStopRequest(nsIRequest*, nsresult)|hg:hg.mozilla.org/mozilla-central:uriloader/base/nsDocLoader.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|677|0x329
17|38|libxul.so|nsDocShell::OnStopRequest(nsIRequest*, nsresult)|hg:hg.mozilla.org/mozilla-central:docshell/base/nsDocShell.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|13855|0x65
17|39|libxul.so|{virtual override thunk({offset(-4)}, nsDocShell::OnStopRequest(nsIRequest*, nsresult))}|||0x27
17|40|libxul.so|mozilla::net::nsLoadGroup::NotifyRemovalObservers(nsIRequest*, nsresult)|hg:hg.mozilla.org/mozilla-central:netwerk/base/nsLoadGroup.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|614|0x10d
17|41|libxul.so|mozilla::net::nsLoadGroup::RemoveRequest(nsIRequest*, nsISupports*, nsresult)|hg:hg.mozilla.org/mozilla-central:netwerk/base/nsLoadGroup.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|518|0x56
17|42|libxul.so|mozilla::dom::Document::DoUnblockOnload()|hg:hg.mozilla.org/mozilla-central:dom/base/Document.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|11664|0xaa
17|43|libxul.so|mozilla::dom::Document::UnblockOnload(bool)|hg:hg.mozilla.org/mozilla-central:dom/base/Document.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|11602|0x82
17|44|libxul.so|mozilla::dom::Document::DispatchContentLoadedEvents()|hg:hg.mozilla.org/mozilla-central:dom/base/Document.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|8137|0x5ef
17|45|libxul.so|mozilla::detail::RunnableMethodImpl<mozilla::dom::Document*, void (mozilla::dom::Document::*)(), true, (mozilla::RunnableKind)0, >::Run()|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThreadUtils.h:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|1200|0x32
17|46|libxul.so|mozilla::SchedulerGroup::Runnable::Run()|hg:hg.mozilla.org/mozilla-central:xpcom/threads/SchedulerGroup.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|140|0x4d
17|47|libxul.so|mozilla::RunnableTask::Run()|hg:hg.mozilla.org/mozilla-central:xpcom/threads/TaskController.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|467|0xe4
17|48|libxul.so|mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/TaskController.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|780|0x53f
17|49|libxul.so|mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/TaskController.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|612|0x3f
17|50|libxul.so|mozilla::TaskController::ProcessPendingMTTask(bool)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/TaskController.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|390|0x6d
17|51|libxul.so|mozilla::detail::RunnableFunction<mozilla::TaskController::InitializeInternal()::$_0>::Run()|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThreadUtils.h:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|531|0x22
17|52|libxul.so|nsThread::ProcessNextEvent(bool, bool*)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThread.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|1180|0x5db
17|53|libxul.so|NS_ProcessNextEvent(nsIThread*, bool)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThreadUtils.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|465|0x6c
17|54|libxul.so|mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessagePump.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|85|0x9b
17|55|libxul.so|MessageLoop::RunInternal()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|380|0x66
17|56|libxul.so|MessageLoop::Run()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|355|0x3c
17|57|libxul.so|nsBaseAppShell::Run()|hg:hg.mozilla.org/mozilla-central:widget/nsBaseAppShell.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|137|0x36
17|58|libxul.so|XRE_RunAppShell()|hg:hg.mozilla.org/mozilla-central:toolkit/xre/nsEmbedFunctions.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|877|0x55
17|59|libxul.so|mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessagePump.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|235|0x45
17|60|libxul.so|MessageLoop::RunInternal()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|380|0x66
17|61|libxul.so|MessageLoop::Run()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|355|0x3c
17|62|libxul.so|XRE_InitChildProcess(int, char**, XREChildData const*)|hg:hg.mozilla.org/mozilla-central:toolkit/xre/nsEmbedFunctions.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|736|0x551
17|63|libxul.so|mozilla::BootstrapImpl::XRE_InitChildProcess(int, char**, XREChildData const*)|hg:hg.mozilla.org/mozilla-central:toolkit/xre/Bootstrap.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|67|0x23
17|64|libmozglue.so|Java_org_mozilla_gecko_mozglue_GeckoLoader_nativeRun|hg:hg.mozilla.org/mozilla-central:mozglue/android/APKOpen.cpp:6e268f45bed2487cce05cfae5cabb7c1c8f415a6|400|0x19a
17|65|libart.so||||

Comment 1

[Andrew McCreight [:mccr8]]

This looks like an integer overflow, so I'm going to hide it for now. My guess would be that this is really a Canvas issue with passing in bad arguments to the JS engine rather than an issue with SpiderMonkey. The canvas element has a height and width of 32767, which does suggest there's some integer overflow issues.

Comment 2

[Andrew McCreight [:mccr8]]

Jon, is there any kind of security issue here if we overflow, or is this just used to guide GC heuristics?

Comment 3

[Andrew McCreight [:mccr8]]

I guess arguably here this addBytes function should do proper integer overflow checking which would again put it in the realm of the GC. Canvas does check that the size of the canvas doesn't overflow, but I guess adding size of the base object makes it overflow, and that seems a bit tedious to manually check.

Comment 4

[Lee Salzman [:lsalzman]]

It looks like everywhere along the chain we're using size_t and properly checking for overflow. It is not immediately obvious why this assert is being triggered at all. If anything, I am almost suspicious of the initialBytes + nbytes calculation itself because there is no operator+ in DebugOnly. A Pernosco repro might help here?

Comment 5

[Jon Coppeard (:jonco)]

(In reply to Andrew McCreight [:mccr8] from comment #2)
There's no security issue, it's just used to trigger GCs.

Comment 6

[Tyson Smith [:tsmith]]

(In reply to Lee Salzman [:lsalzman] from comment #4)

A Pernosco repro might help here?

We are only able to get Pernosco sessions on Linux. At the moment the fuzzers are only reporting this on Android. If a test case that reproduces the issue on Linux is provided I'd be happy to get a Pernosco session.

Comment 7

[Andrew McCreight [:mccr8]]

Given that the overflow appears to be happening in GC code, I'll move it back there. Sorry for the noise.

Comment 8

[Intermittent Failures Robot]

1 failures in 3339 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1767836&startday=2022-08-08&endday=2022-08-14&tree=all

Comment 9

[Tyson Smith [:tsmith]]

This is still being reported by fuzzers. It does not seem to be OS specific but has only be reported while targeting 32-bit builds.

I am able to reproduce the issue on a 32-bit Linux build. Unfortunately Pernosco does not support 32-bit builds. I can create an rr recording and share it via google drive if needed.

Comment 10

[Matthew Gaudet (he/him) [:mgaudet]]

(If you shared a recording I'd be willing to take a peek)

Comment 11

[Tyson Smith [:tsmith]]

That's be great. I'll create a rr recording.

Comment 12

[Tyson Smith [:tsmith]]

I've shared the rr recording via Google Drive.

Comment 13

[Matthew Gaudet (he/him) [:mgaudet]]

Hrm. Initial pass I couldn't replay

matthew@xtower:~/data/rr-traces$ rr replay latest-trace -p 66305
Trace XCR0 value 0x1f != our XCR0 value 0xff; Replay will probably fail because glibc dynamic loader examines XCR0

It's late on a Friday though, so don't take too much from this.

Comment 14

[Tyson Smith [:tsmith]]

Not sure what happened. I can run the rr trace locally but I see the same error on another machine.

In the mean time this is reproducible with the attached test case and a 32-bit build.

To reproduce via Grizzly Replay:

$ pip install grizzly-framework
$ python -m grizzly.replay.bugzilla ./firefox/firefox <bugid>

Comment 15

[Matthew Gaudet (he/him) [:mgaudet]]

Ok, so I can reproduce locally, and I can load the trace -- however, for some reason I get no debug info. Still, without the debug info we can still figure out what's happening here -- though this is a restatement of a bunch of the discussion above, just trying to put it in one place for clarity.

So the test case is important here:

<script>
window.addEventListener('load', () => {
  let a = document.createElementNS('http://www.w3.org/1999/xhtml', 'canvas')
  let b = a.getContext('2d')
  a.setAttribute('height', 32767)
  a.toDataURL('text/html; charset=utf-8', 0.5)
  a.setAttribute('width', 32767)
})
</script>

So, the trick here is we set attribute twice, setting height and width to 32767 x 32767 (Notably this is exactly gfx.canvas.max_size=0x7fff).

This produces a number of bytes for the canvas of 32767 x 32767 x 4 (r,g,b,a presumably)-- this is 0xfffc0004 bytes. As a result of the second setAttribute we end up calling into mozilla::dom::CanvasRenderingContextHelper::UpdateContext, which in turn calls down to (skipping some frames) to mozilla::dom::CanvasRenderingContext2D::ClearTarget which wants to update the memory accounting.

The problem here is threefold:

1. We're creating a context which almost certainly can never be instantiated -- 0xfffc0004 bytes is almost all the 32 bit address space.
2. We do memory accounting speculatively, before the memory is actually allocated it seems.
3. The JS engine is asserting that we don't overflow our accounting.

I'm going to argue this should go back to gfx; the issue is that the CanvasRenderingContext2D is reporting an unreal amount of associated memory; perhaps the accounting calls need to be moved (or limits enforced elsewhere?)