1735482 - Intermittent macOS 10.15 Assertion failure: [GFX1]: void mozilla::gl::GLContext::fGenerateMipmap(GLenum): Generated unexpected GL_OUT_OF_MEMORY error, at /builds/worker/checkouts/gecko/gfx/2d/Logging.h:751

Status: RESOLVED FIXED

(Core :: Graphics, defect, P1)

Description

[Treeherder Bug Filer]

Filed by: imoraru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=354619687&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/am_HfGf5RjGBxNuBdEPALw/runs/0/artifacts/public/logs/live_backing.log

---

[task 2021-10-13T08:59:03.403Z] 08:59:03     INFO - TEST-START | dom/canvas/test/webgl-conf/generated/test_2_conformance2__textures__misc__generate-mipmap-with-large-base-level.html
[task 2021-10-13T08:59:03.704Z] 08:59:03     INFO - GECKO(1420) | [gl:0x11d6b8900] void mozilla::gl::GLContext::fGenerateMipmap(GLenum): Generated unexpected GL_OUT_OF_MEMORY error.
[task 2021-10-13T08:59:03.704Z] 08:59:03     INFO - GECKO(1420) | [GFX1]: void mozilla::gl::GLContext::fGenerateMipmap(GLenum): Generated unexpected GL_OUT_OF_MEMORY error
[task 2021-10-13T08:59:03.705Z] 08:59:03     INFO - GECKO(1420) | Assertion failure: [GFX1]: void mozilla::gl::GLContext::fGenerateMipmap(GLenum): Generated unexpected GL_OUT_OF_MEMORY error, at /builds/worker/checkouts/gecko/gfx/2d/Logging.h:751
[task 2021-10-13T08:59:03.706Z] 08:59:03     INFO -  Initializing stack-fixing for the first stack frame, this may take a while...
[task 2021-10-13T08:59:11.755Z] 08:59:11     INFO - GECKO(1420) | #01: mozilla::gfx::Log<1, mozilla::gfx::CriticalLogger>::WriteLog(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) [gfx/2d/Logging.h:752]
[task 2021-10-13T08:59:11.756Z] 08:59:11     INFO - GECKO(1420) | #02: mozilla::gfx::Log<1, mozilla::gfx::CriticalLogger>::Flush() [gfx/2d/Logging.h:279]
[task 2021-10-13T08:59:11.756Z] 08:59:11     INFO - GECKO(1420) | #03: mozilla::gl::GLContext::AfterGLCall_Debug(char const*) const [gfx/gl/GLContext.cpp:2593]
[task 2021-10-13T08:59:11.757Z] 08:59:11     INFO - GECKO(1420) | #04: mozilla::WebGLTexture::GenerateMipmap() [dom/canvas/WebGLTexture.cpp:811]
[task 2021-10-13T08:59:11.757Z] 08:59:11     INFO - GECKO(1420) | #05: mozilla::WebGLContext::GenerateMipmap(unsigned int) [dom/canvas/WebGLContextTextures.cpp:121]
[task 2021-10-13T08:59:11.758Z] 08:59:11     INFO - GECKO(1420) | #06: auto bool mozilla::MethodDispatcher<mozilla::WebGLMethodDispatcher, (unsigned long)71, void (mozilla::HostWebGLContext::*)(unsigned int) const, &mozilla::HostWebGLContext(unsigned int)::GenerateMipmap const>::DispatchCommand<mozilla::HostWebGLContext>(mozilla::HostWebGLContext&, unsigned long, mozilla::webgl::RangeConsumerView&)::{lambda(auto:1&...)#1}::operator()<unsigned int>(unsigned int&) const [dom/canvas/WebGLCommandQueue.h:248]
[task 2021-10-13T08:59:11.758Z] 08:59:11     INFO - GECKO(1420) | #07: bool mozilla::MethodDispatcher<mozilla::WebGLMethodDispatcher, (unsigned long)68, void (mozilla::HostWebGLContext::*)(unsigned long long, unsigned int, unsigned int, unsigned int, unsigned int) const, &mozilla::HostWebGLContext(unsigned long long, unsigned int, unsigned int, unsigned int, unsigned int)::RenderbufferStorageMultisample const>::DispatchCommand<mozilla::HostWebGLContext>(mozilla::HostWebGLContext&, unsigned long, mozilla::webgl::RangeConsumerView&) [dom/canvas/WebGLCommandQueue.h:253]
[task 2021-10-13T08:59:11.759Z] 08:59:11     INFO - GECKO(1420) | #08: mozilla::dom::WebGLParent::RecvDispatchCommands(mozilla::ipc::Shmem&&, unsigned long long) [dom/canvas/WebGLParent.cpp:60]
[task 2021-10-13T08:59:11.760Z] 08:59:11     INFO - GECKO(1420) | #09: mozilla::dom::PWebGLParent::OnMessageReceived(IPC::Message const&) [s3:gecko-generated-sources:d6f8e82aaa582ed97a01177de996c3081a57a1445e06a0474f5d2215c2d0a66003ac43605f398d691317296ace4c8da9be96372b8149bfdbb03572823cdc2209/ipc/ipdl/PWebGLParent.cpp::200]
[task 2021-10-13T08:59:11.760Z] 08:59:11     INFO - GECKO(1420) | #10: mozilla::layers::PCompositorManagerParent::OnMessageReceived(IPC::Message const&) [s3:gecko-generated-sources:eb342a872f30e7a318c73d6d20c67dc9dc81ec3c962b8c11d24efd84ddef0a26428459d42b993e21de3e3a67852ae5e89b621a82a92715fa8b48d258312f5ea1/ipc/ipdl/PCompositorManagerParent.cpp::200]
[task 2021-10-13T08:59:11.761Z] 08:59:11     INFO - GECKO(1420) | #11: mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) [ipc/glue/MessageChannel.cpp:2043]
[task 2021-10-13T08:59:11.761Z] 08:59:11     INFO - GECKO(1420) | #12: mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) [ipc/glue/MessageChannel.cpp:1968]
[task 2021-10-13T08:59:11.761Z] 08:59:11     INFO - GECKO(1420) | #13: mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) [ipc/glue/MessageChannel.cpp:1827]
[task 2021-10-13T08:59:11.762Z] 08:59:11     INFO - GECKO(1420) | #14: mozilla::ipc::MessageChannel::MessageTask::Run() [ipc/glue/MessageChannel.cpp:1857]
[task 2021-10-13T08:59:11.762Z] 08:59:11     INFO - GECKO(1420) | #15: nsThread::ProcessNextEvent(bool, bool*) [xpcom/threads/nsThread.cpp:1145]
[task 2021-10-13T08:59:11.763Z] 08:59:11     INFO - GECKO(1420) | #16: NS_ProcessNextEvent(nsIThread*, bool) [xpcom/threads/nsThreadUtils.cpp:467]
[task 2021-10-13T08:59:11.763Z] 08:59:11     INFO - GECKO(1420) | #17: mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) [ipc/glue/MessagePump.cpp:330]
[task 2021-10-13T08:59:11.763Z] 08:59:11     INFO - GECKO(1420) | #18: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:307]
[task 2021-10-13T08:59:11.764Z] 08:59:11     INFO - GECKO(1420) | #19: nsThread::ThreadFunc(void*) [xpcom/threads/nsThread.cpp:392]
[task 2021-10-13T08:59:11.952Z] 08:59:11     INFO - GECKO(1420) | #20: _pt_root [nsprpub/pr/src/pthreads/ptthread.c:204]
[task 2021-10-13T08:59:11.954Z] 08:59:11     INFO - GECKO(1420) | #21: _pthread_start [/usr/lib/system/libsystem_pthread.dylib + 0x6109]
[task 2021-10-13T08:59:11.954Z] 08:59:11     INFO - GECKO(1420) | Hit MOZ_CRASH(GFX: An assert from the graphics logger) at /builds/worker/checkouts/gecko/gfx/2d/Logging.h:752
<...>
[task 2021-10-13T08:59:11.974Z] 08:59:11     INFO - GECKO(1420) | [Child 1430, IPC I/O Child] WARNING: [9455BACB85E5A716.C144634C9FB92FB3]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.974Z] 08:59:11     INFO - GECKO(1420) | Exiting due to channel error.
[task 2021-10-13T08:59:11.975Z] 08:59:11     INFO - GECKO(1420) | [Child 1426, IPC I/O Child] WARNING: [1749B46ECE6470C6.96AB4E0F4F592BAC]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.975Z] 08:59:11     INFO - GECKO(1420) | [Child 1429, IPC I/O Child] WARNING: [C69226D599C39CE1.9CB94E685E6FA7C3]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.975Z] 08:59:11     INFO - GECKO(1420) | [Child 1426, IPC I/O Child] WARNING: [1749B46ECE6470C6.96AB4E0F4F592BAC]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.976Z] 08:59:11     INFO - GECKO(1420) | [Child 1426, IPC I/O Child] WARNING: [1749B46ECE6470C6.96AB4E0F4F592BAC]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.977Z] 08:59:11     INFO - GECKO(1420) | [Child 1426, IPC I/O Child] WARNING: [1749B46ECE6470C6.96AB4E0F4F592BAC]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.977Z] 08:59:11     INFO - GECKO(1420) | [Child 1427, IPC I/O Child] WARNING: [DA10893B92215436.AFC06EC36FD542B0]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.978Z] 08:59:11     INFO - GECKO(1420) | [Child 1426, IPC I/O Child] WARNING: [1749B46ECE6470C6.96AB4E0F4F592BAC]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.978Z] 08:59:11     INFO - GECKO(1420) | [Child 1429, IPC I/O Child] WARNING: [C69226D599C39CE1.9CB94E685E6FA7C3]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.978Z] 08:59:11     INFO - GECKO(1420) | ###!!! [Child][MessageChannel::SendAndWait] Error: Channel error: cannot send/recv
[task 2021-10-13T08:59:11.979Z] 08:59:11     INFO - GECKO(1420) | [Child 1426, IPC I/O Child] WARNING: [1749B46ECE6470C6.96AB4E0F4F592BAC]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.979Z] 08:59:11     INFO - GECKO(1420) | [Child 1427, IPC I/O Child] WARNING: [DA10893B92215436.AFC06EC36FD542B0]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.980Z] 08:59:11     INFO - GECKO(1420) | [Child 1429, IPC I/O Child] WARNING: [C69226D599C39CE1.9CB94E685E6FA7C3]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.980Z] 08:59:11     INFO - GECKO(1420) | [Child 1427, IPC I/O Child] WARNING: [DA10893B92215436.AFC06EC36FD542B0]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.981Z] 08:59:11     INFO - GECKO(1420) | [Child 1429, IPC I/O Child] WARNING: [C69226D599C39CE1.9CB94E685E6FA7C3]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.981Z] 08:59:11     INFO - GECKO(1420) | [Child 1427, IPC I/O Child] WARNING: [DA10893B92215436.AFC06EC36FD542B0]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.982Z] 08:59:11     INFO - GECKO(1420) | Exiting due to channel error.
[task 2021-10-13T08:59:11.982Z] 08:59:11     INFO - GECKO(1420) | [Child 1426, IPC I/O Child] WARNING: [1749B46ECE6470C6.96AB4E0F4F592BAC]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.982Z] 08:59:11     INFO - GECKO(1420) | Exiting due to channel error.
[task 2021-10-13T08:59:11.983Z] 08:59:11     INFO - GECKO(1420) | [GFX1-]: Receive IPC close with reason=AbnormalShutdown
[task 2021-10-13T08:59:11.983Z] 08:59:11     INFO - GECKO(1420) | [Child 1427, IPC I/O Child] WARNING: [DA10893B92215436.AFC06EC36FD542B0]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.984Z] 08:59:11     INFO - GECKO(1420) | [Child 1427, IPC I/O Child] WARNING: [DA10893B92215436.AFC06EC36FD542B0]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.984Z] 08:59:11     INFO - GECKO(1420) | [Child 1427, IPC I/O Child] WARNING: [DA10893B92215436.AFC06EC36FD542B0]: Ignoring message 'EVENT_MESSAGE' to peer 1.1 due to a missing broker: file /builds/worker/checkouts/gecko/ipc/glue/NodeController.cpp:297
[task 2021-10-13T08:59:11.985Z] 08:59:11     INFO - GECKO(1420) | Exiting due to channel error.
[task 2021-10-13T08:59:11.985Z] 08:59:11     INFO - TEST-INFO | Main app process: exit 1
[task 2021-10-13T08:59:11.985Z] 08:59:11     INFO - Buffered messages logged at 08:59:03
[task 2021-10-13T08:59:11.986Z] 08:59:11     INFO - TEST-PASS | dom/canvas/test/webgl-conf/generated/test_2_conformance2__textures__misc__generate-mipmap-with-large-base-level.html | A valid string reason is expected 

Comment 2

[Intermittent Failures Robot]

20 failures in 3315 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 17
• try: 1
• mozilla-central: 2

Platform and build breakdown:

• macosx1015-64-qr: 18
  • debug: 15
  • opt: 3
• macosx1015-64-shippable-qr: 1
  • opt: 1
• macosx1015-64-devedition-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1735482&startday=2021-10-11&endday=2021-10-17&tree=all

Comment 3

[Alexandru Michis [:malexandru]]

Paul, could this be caused by the changes in Bug 1669392?
https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&searchStr=os%2Cx%2C10.15%2Cwebrender%2Cdebug%2Cmochitests%2Cwith%2Cwebgl%2Cover%2Cipc%2Ctest-macosx1015-64-qr%2Fdebug-mochitest-webgl2-core-gli-e10s%2Cgl2c&tochange=660b572e30496bda577acc308f9cd5e4dc0946c5&fromchange=edcf14d246f935a238fd2dc5d7e15fcaabd530de&selectedTaskRun=dlS3EQu3SiG0XCg52dx5Mg.0

Comment 4

[Intermittent Failures Robot]

34 failures in 3186 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #23 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 27
• mozilla-central: 7

Platform and build breakdown:

• macosx1015-64-qr: 33
  • debug: 33
• macosx1100-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1735482&startday=2021-10-18&endday=2021-10-24&tree=all

Comment 5

[Paul Bone [:pbone]]

it sure looks like it started after that patch. I don't see how (yet) but I'll investigate.

Comment 6

[Iulian Moraru]

Update:
There have been 34 failures within the last 7 days:
• 33 failures on OS X 10.15 WebRender debug
• 1 failures on OS X 11 WebRender Shippable opt
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=356468830&repo=autoland&lineNumber=3906

[task 2021-10-29T19:59:27.778Z] 19:59:27     INFO - TEST-START | dom/canvas/test/webgl-conf/generated/test_2_conformance2__textures__misc__generate-mipmap-with-large-base-level.html
[task 2021-10-29T19:59:27.980Z] 19:59:27     INFO - GECKO(8699) | [Child 8701, Main Thread] WARNING: Not resolving response because actor is dead.: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:935
[task 2021-10-29T19:59:27.981Z] 19:59:27     INFO - GECKO(8699) | [Child 8701, Main Thread] WARNING: IPDL resolver dropped without being called!: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:963
[task 2021-10-29T19:59:27.981Z] 19:59:27     INFO - GECKO(8699) | [Child 8701, Main Thread] WARNING: Not resolving response because actor is dead.: file /builds/worker/checkouts/gecko/ipc/glue/ProtocolUtils.cpp:935
[task 2021-10-29T19:59:28.109Z] 19:59:28     INFO - GECKO(8699) | [gl:0x123c34d00] void mozilla::gl::GLContext::fGenerateMipmap(GLenum): Generated unexpected GL_OUT_OF_MEMORY error.
[task 2021-10-29T19:59:28.110Z] 19:59:28     INFO - GECKO(8699) | [GFX1]: void mozilla::gl::GLContext::fGenerateMipmap(GLenum): Generated unexpected GL_OUT_OF_MEMORY error
[task 2021-10-29T19:59:28.110Z] 19:59:28     INFO - GECKO(8699) | Assertion failure: [GFX1]: void mozilla::gl::GLContext::fGenerateMipmap(GLenum): Generated unexpected GL_OUT_OF_MEMORY error, at /builds/worker/checkouts/gecko/gfx/2d/Logging.h:751
[task 2021-10-29T19:59:28.112Z] 19:59:28     INFO -  Initializing stack-fixing for the first stack frame, this may take a while...
[task 2021-10-29T19:59:36.468Z] 19:59:36     INFO - GECKO(8699) | #01: mozilla::gfx::Log<1, mozilla::gfx::CriticalLogger>::WriteLog(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) [gfx/2d/Logging.h:752]
[task 2021-10-29T19:59:36.468Z] 19:59:36     INFO - GECKO(8699) | #02: mozilla::gfx::Log<1, mozilla::gfx::CriticalLogger>::Flush() [gfx/2d/Logging.h:279]
[task 2021-10-29T19:59:36.469Z] 19:59:36     INFO - GECKO(8699) | #03: mozilla::gfx::Log<1, mozilla::gfx::CriticalLogger>::~Log() [gfx/2d/Logging.h:270]
[task 2021-10-29T19:59:36.469Z] 19:59:36     INFO - GECKO(8699) | #04: mozilla::gl::GLContext::AfterGLCall_Debug(char const*) const [gfx/gl/GLContext.cpp:2593]
[task 2021-10-29T19:59:36.469Z] 19:59:36     INFO - GECKO(8699) | #05: mozilla::WebGLTexture::GenerateMipmap() [dom/canvas/WebGLTexture.cpp:811]
[task 2021-10-29T19:59:36.470Z] 19:59:36     INFO - GECKO(8699) | #06: auto bool mozilla::MethodDispatcher<mozilla::WebGLMethodDispatcher, (unsigned long)71, void (mozilla::HostWebGLContext::*)(unsigned int) const, &mozilla::HostWebGLContext(unsigned int)::GenerateMipmap const>::DispatchCommand<mozilla::HostWebGLContext>(mozilla::HostWebGLContext&, unsigned long, mozilla::webgl::RangeConsumerView&)::{lambda(auto:1&...)#1}::operator()<unsigned int>(unsigned int&) const [dom/canvas/WebGLCommandQueue.h:248]
[task 2021-10-29T19:59:36.471Z] 19:59:36     INFO - GECKO(8699) | #07: bool mozilla::MethodDispatcher<mozilla::WebGLMethodDispatcher, (unsigned long)68, void (mozilla::HostWebGLContext::*)(unsigned long long, unsigned int, unsigned int, unsigned int, unsigned int) const, &mozilla::HostWebGLContext(unsigned long long, unsigned int, unsigned int, unsigned int, unsigned int)::RenderbufferStorageMultisample const>::DispatchCommand<mozilla::HostWebGLContext>(mozilla::HostWebGLContext&, unsigned long, mozilla::webgl::RangeConsumerView&) [dom/canvas/WebGLCommandQueue.h:253]
[task 2021-10-29T19:59:36.471Z] 19:59:36     INFO - GECKO(8699) | #08: mozilla::dom::WebGLParent::RecvDispatchCommands(mozilla::ipc::Shmem&&, unsigned long long) [dom/canvas/WebGLParent.cpp:64]
[task 2021-10-29T19:59:36.472Z] 19:59:36     INFO - GECKO(8699) | #09: mozilla::dom::PWebGLParent::OnMessageReceived(IPC::Message const&) [s3:gecko-generated-sources:83f0fc8eea21f87305e8cc086ef4153440dc37f971f4413ebe3f2a4c59321a74e3fd17541f686ebe0a260c7d829200964a90982bb078963403c3b4cc97890558/ipc/ipdl/PWebGLParent.cpp::222]
[task 2021-10-29T19:59:36.472Z] 19:59:36     INFO - GECKO(8699) | #10: mozilla::layers::PCompositorManagerParent::OnMessageReceived(IPC::Message const&) [s3:gecko-generated-sources:77156d68f09aef8f3cd0023ea3ee1a1c027d49330011da475987387a5487f29b44c54641053e2ddbe46e91372c785a50d6458aedf3a39253cf435168863b24e9/ipc/ipdl/PCompositorManagerParent.cpp::195]
[task 2021-10-29T19:59:36.473Z] 19:59:36     INFO - GECKO(8699) | #11: mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) [ipc/glue/MessageChannel.cpp:2043]
[task 2021-10-29T19:59:36.473Z] 19:59:36     INFO - GECKO(8699) | #12: mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) [ipc/glue/MessageChannel.cpp:1968]
[task 2021-10-29T19:59:36.474Z] 19:59:36     INFO - GECKO(8699) | #13: mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) [ipc/glue/MessageChannel.cpp:1827]
[task 2021-10-29T19:59:36.474Z] 19:59:36     INFO - GECKO(8699) | #14: mozilla::ipc::MessageChannel::MessageTask::Run() [ipc/glue/MessageChannel.cpp:1857]
[task 2021-10-29T19:59:36.474Z] 19:59:36     INFO - GECKO(8699) | #15: nsThread::ProcessNextEvent(bool, bool*) [xpcom/threads/nsThread.cpp:1169]
[task 2021-10-29T19:59:36.475Z] 19:59:36     INFO - GECKO(8699) | #16: NS_ProcessNextEvent(nsIThread*, bool) [xpcom/threads/nsThreadUtils.cpp:467]
[task 2021-10-29T19:59:36.475Z] 19:59:36     INFO - GECKO(8699) | #17: mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) [ipc/glue/MessagePump.cpp:330]
[task 2021-10-29T19:59:36.476Z] 19:59:36     INFO - GECKO(8699) | #18: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:307]
[task 2021-10-29T19:59:36.476Z] 19:59:36     INFO - GECKO(8699) | #19: nsThread::ThreadFunc(void*) [xpcom/threads/nsThread.cpp:393]
[task 2021-10-29T19:59:36.672Z] 19:59:36     INFO - GECKO(8699) | #20: _pt_root [nsprpub/pr/src/pthreads/ptthread.c:204]
[task 2021-10-29T19:59:36.673Z] 19:59:36     INFO - GECKO(8699) | #21: _pthread_start [/usr/lib/system/libsystem_pthread.dylib + 0x6109]
[task 2021-10-29T19:59:36.673Z] 19:59:36     INFO - GECKO(8699) | Hit MOZ_CRASH(GFX: An assert from the graphics logger) at /builds/worker/checkouts/gecko/gfx/2d/Logging.h:752

Comment 7

[Iulian Moraru]

Hi Paul! Any updates on this?

Comment 8

[Intermittent Failures Robot]

35 failures in 2845 pushes (0.012 failures/push) were associated with this bug in the last 7 days.

This is the #20 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 27
• mozilla-central: 7
• try: 1

Platform and build breakdown:

• macosx1015-64-qr: 33
  • debug: 33
• macosx1100-64-shippable-qr: 1
  • opt: 1
• macosx1015-64-devedition-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1735482&startday=2021-10-25&endday=2021-10-31&tree=all

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1669392

Comment 10

[Paul Bone [:pbone]]

(In reply to Iulian Moraru from comment #7)

Hi Paul! Any updates on this?

I now have access to a MacOS machine so I can begin trying to reproduce it.

Comment 11

[Intermittent Failures Robot]

37 failures in 3030 pushes (0.012 failures/push) were associated with this bug in the last 7 days.

This is the #31 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 12
• autoland: 24
• try: 1

Platform and build breakdown:

• macosx1015-64-qr: 36
  • debug: 36
• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1735482&startday=2021-11-01&endday=2021-11-07&tree=all

Comment 12

[Intermittent Failures Robot]

22 failures in 2723 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

This is the #45 most frequent failure this week.

Repository breakdown:

• autoland: 17
• mozilla-central: 4
• mozilla-beta: 1

Platform and build breakdown:

• macosx1015-64-qr: 21
  • debug: 20
  • opt: 1
• macosx1015-64-devedition-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1735482&startday=2021-11-08&endday=2021-11-14&tree=all

Comment 13

[Paul Bone [:pbone]]

Attachment: Bug 1735482 - Don't use a QuantumWide size class on MacOS r=glandium

Comment 14

[Paul Bone [:pbone]]

This is most likely a memory safety error so I'm going to mark it as secure.

Comment 15

[Paul Bone [:pbone]]

Comment on attachment 9251498 [details]
Bug 1735482 - Don't use a QuantumWide size class on MacOS r=glandium

Security Approval Request

• How easily could an exploit be constructed based on the patch?: Not easily, the fault appears to be in the OpenGL library on MacOS whose source code is closed. My patch works around it.
• Do comments in the patch, the check-in comment, or tests included in the patch paint a bulls-eye on the security problem?: No
• Which older supported branches are affected by this flaw?: beta
• If not all supported branches, which bug introduced the flaw?: Bug 1669392
• Do you have backports for the affected branches?: No
• If not, how different, hard to create, and risky will they be?: The patch will probably apply cleanly. Not risky, it reverts jemalloc to it's previous behaviour for MacOS.
• How likely is this patch to cause regressions; how much testing does it need?: Not risky, it reverts jemalloc to it's previous behaviour for MacOS. This code is tested every time Firefox runs and this was the previous behaviour for a long time.

Comment 17

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Comment on attachment 9251498 [details]
Bug 1735482 - Don't use a QuantumWide size class on MacOS r=glandium

Approved to land and request uplift

Comment 18

[Ryan VanderMeulen [:RyanVM]]

Landed on autoland. This grafts cleanly to Beta, so go ahead and request Beta approval at your earliest convenience.
https://hg.mozilla.org/integration/autoland/rev/85fdbf7e368a2084bc53ed1d4433206b8f982391

Comment 19

[Iulian Moraru]

Backed out for causing OS X build bustages on mozjemalloc.cpp.
This failed on OS X Cross Compiled opt and debug.

Push with failures

Failure log

Backout link

[task 2021-11-22T16:48:37.367Z] 16:48:37     INFO -  gmake[4]: Entering directory '/builds/worker/workspace/obj-build/memory/build'
[task 2021-11-22T16:48:37.369Z] 16:48:37     INFO -  /builds/worker/fetches/sccache/sccache /builds/worker/fetches/clang/bin/clang++ --sysroot /builds/worker/fetches/MacOSX10.12.sdk -mmacosx-version-min=10.12 -stdlib=libc++ -std=gnu++17 --target=x86_64-apple-darwin -o Unified_cpp_memory_build0.o -c  -fvisibility=hidden -fvisibility-inlines-hidden -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -ftrivial-auto-var-init=pattern -DDEBUG=1 -DMOZ_MEMORY_IMPL -DMOZ_PHC -DMOZ_REPLACE_MALLOC_STATIC -DMOZ_HAS_MOZGLUE -DIMPL_MFBT -I/builds/worker/checkouts/gecko/memory/build -I/builds/worker/workspace/obj-build/memory/build -I/builds/worker/workspace/obj-build/dist/include -I/builds/worker/workspace/obj-build/dist/include/nspr -I/builds/worker/workspace/obj-build/dist/include/nss -DMOZILLA_CLIENT -include /builds/worker/workspace/obj-build/mozilla-config.h -Qunused-arguments -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wdeprecated-this-capture -Wempty-body -Wformat-type-confusion -Wignored-qualifiers -Wpointer-arith -Wshadow-field-in-constructor-modified -Wshadow-uncaptured-local -Wsign-compare -Wtype-limits -Wno-error=tautological-type-limit-compare -Wunreachable-code -Wunreachable-code-return -Wunused-but-set-parameter -Wno-invalid-offsetof -Wclass-varargs -Wempty-init-stmt -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wno-range-loop-analysis -Wc++2a-compat -Wcomma -Wenum-compare-conditional -Wimplicit-fallthrough -Werror=non-literal-null-conversion -Wstring-conversion -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=backend-plugin -Wno-error=free-nonheap-object -Wno-error=return-std-move -Wno-error=atomic-alignment -Wno-error=deprecated-copy -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Wno-psabi -Wno-unknown-warning-option -fno-sized-deallocation -fno-aligned-new -fcrash-diagnostics-dir=/builds/worker/artifacts -fno-exceptions -fno-strict-aliasing -fPIC -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -g -Xclang -load -Xclang /builds/worker/workspace/obj-build/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O3 -fno-omit-frame-pointer -funwind-tables -Werror  -MD -MP -MF .deps/Unified_cpp_memory_build0.o.pp   Unified_cpp_memory_build0.cpp
[task 2021-11-22T16:48:37.369Z] 16:48:37     INFO -  In file included from Unified_cpp_memory_build0.cpp:2:
[task 2021-11-22T16:48:37.369Z] 16:48:37     INFO -  In file included from /builds/worker/checkouts/gecko/memory/build/mozjemalloc.cpp:166:
[task 2021-11-22T16:48:37.370Z] 16:48:37  WARNING -  /builds/worker/checkouts/gecko/memory/build/Mutex.h:27:3: warning: 'OSSpinLock' is deprecated: first deprecated in macOS 10.12 - Use os_unfair_lock() from <os/lock.h> instead [-Wdeprecated-declarations]
[task 2021-11-22T16:48:37.370Z] 16:48:37     INFO -    OSSpinLock mMutex;
[task 2021-11-22T16:48:37.370Z] 16:48:37     INFO -    ^
[task 2021-11-22T16:48:37.370Z] 16:48:37     INFO -  /builds/worker/fetches/MacOSX10.12.sdk/usr/include/libkern/OSSpinLockDeprecated.h:79:17: note: 'OSSpinLock' has been explicitly marked deprecated here
[task 2021-11-22T16:48:37.371Z] 16:48:37     INFO -  typedef int32_t OSSpinLock OSSPINLOCK_DEPRECATED_REPLACE_WITH(os_unfair_lock);
[task 2021-11-22T16:48:37.371Z] 16:48:37     INFO -                  ^
[task 2021-11-22T16:48:37.371Z] 16:48:37     INFO -  In file included from Unified_cpp_memory_build0.cpp:2:
[task 2021-11-22T16:48:37.371Z] 16:48:37     INFO -  In file included from /builds/worker/checkouts/gecko/memory/build/mozjemalloc.cpp:166:
[task 2021-11-22T16:48:37.371Z] 16:48:37  WARNING -  /builds/worker/checkouts/gecko/memory/build/Mutex.h:63:5: warning: 'OSSpinLockLock' is deprecated: first deprecated in macOS 10.12 - Use os_unfair_lock_lock() from <os/lock.h> instead [-Wdeprecated-declarations]
[task 2021-11-22T16:48:37.371Z] 16:48:37     INFO -      OSSpinLockLock(&mMutex);
[task 2021-11-22T16:48:37.372Z] 16:48:37     INFO -      ^
[task 2021-11-22T16:48:37.372Z] 16:48:37     INFO -  /builds/worker/fetches/MacOSX10.12.sdk/usr/include/libkern/OSSpinLockDeprecated.h:99:9: note: 'OSSpinLockLock' has been explicitly marked deprecated here
[task 2021-11-22T16:48:37.372Z] 16:48:37     INFO -  void    OSSpinLockLock( volatile OSSpinLock *__lock );
[task 2021-11-22T16:48:37.372Z] 16:48:37     INFO -          ^
[task 2021-11-22T16:48:37.372Z] 16:48:37     INFO -  In file included from Unified_cpp_memory_build0.cpp:2:
[task 2021-11-22T16:48:37.373Z] 16:48:37     INFO -  In file included from /builds/worker/checkouts/gecko/memory/build/mozjemalloc.cpp:166:
[task 2021-11-22T16:48:37.373Z] 16:48:37  WARNING -  /builds/worker/checkouts/gecko/memory/build/Mutex.h:73:5: warning: 'OSSpinLockUnlock' is deprecated: first deprecated in macOS 10.12 - Use os_unfair_lock_unlock() from <os/lock.h> instead [-Wdeprecated-declarations]
[task 2021-11-22T16:48:37.373Z] 16:48:37     INFO -      OSSpinLockUnlock(&mMutex);
[task 2021-11-22T16:48:37.373Z] 16:48:37     INFO -      ^
[task 2021-11-22T16:48:37.373Z] 16:48:37     INFO -  /builds/worker/fetches/MacOSX10.12.sdk/usr/include/libkern/OSSpinLockDeprecated.h:105:9: note: 'OSSpinLockUnlock' has been explicitly marked deprecated here
[task 2021-11-22T16:48:37.374Z] 16:48:37     INFO -  void    OSSpinLockUnlock( volatile OSSpinLock *__lock );
[task 2021-11-22T16:48:37.374Z] 16:48:37     INFO -          ^
[task 2021-11-22T16:48:37.374Z] 16:48:37     INFO -  In file included from Unified_cpp_memory_build0.cpp:2:
[task 2021-11-22T16:48:37.374Z] 16:48:37    ERROR -  /builds/worker/checkouts/gecko/memory/build/mozjemalloc.cpp:2265:3: error: static_assert failed due to requirement 'max_divisor > min_divisor'
[task 2021-11-22T16:48:37.374Z] 16:48:37     INFO -    static_assert(max_divisor > min_divisor);
[task 2021-11-22T16:48:37.375Z] 16:48:37     INFO -    ^             ~~~~~~~~~~~~~~~~~~~~~~~~~
[task 2021-11-22T16:48:37.375Z] 16:48:37     INFO -  /builds/worker/checkouts/gecko/memory/build/mozjemalloc.cpp:2321:13: note: in instantiation of template class 'FastDivide<256, 256>' requested here
[task 2021-11-22T16:48:37.375Z] 16:48:37     INFO -              FastDivide<kQuantumWide, kMaxQuantumWideClass>::divide(diff, size);
[task 2021-11-22T16:48:37.375Z] 16:48:37     INFO -              ^
[task 2021-11-22T16:48:37.376Z] 16:48:37     INFO -  3 warnings and 1 error generated.
[task 2021-11-22T16:48:37.376Z] 16:48:37    ERROR -  gmake[4]: *** [/builds/worker/checkouts/gecko/config/rules.mk:662: Unified_cpp_memory_build0.o] Error 1
[task 2021-11-22T16:48:37.376Z] 16:48:37     INFO -  gmake[4]: Leaving directory '/builds/worker/workspace/obj-build/memory/build'
[task 2021-11-22T16:48:37.376Z] 16:48:37    ERROR -  gmake[3]: *** [/builds/worker/checkouts/gecko/config/recurse.mk:72: memory/build/target-objects] Error 2
[task 2021-11-22T16:48:37.376Z] 16:48:37     INFO -  gmake[3]: *** Waiting for unfinished jobs....
[task 2021-11-22T16:48:37.377Z] 16:48:37     INFO -  gmake[4]: Entering directory '/builds/worker/workspace/obj-build/security/nss/lib/pk11wrap/pk11wrap_pk11wrap'

Comment 20

[Paul Bone [:pbone]]

Ah, it collided with Bug 1738240.

Comment 21

[Paul Bone [:pbone]]

Attachment: Bug 1735482 - Don't use a QuantumWide size class on MacOS r=glandium

Comment 22

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Don't use a QuantumWide size class on MacOS r=glandium
https://hg.mozilla.org/integration/autoland/rev/71332992f78f548b219f6405ca970828bd22c071
https://hg.mozilla.org/mozilla-central/rev/71332992f78f

Comment 23

[Paul Bone [:pbone]]

Comment on attachment 9251498 [details]
Bug 1735482 - Don't use a QuantumWide size class on MacOS r=glandium

Beta/Release Uplift Approval Request

• User impact if declined: A crash in CI, possible visual problems. A suspected memory error affecting MacOS OpenGL.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This change reverts jemalloc's behaviour to that of Firefox 94.
• String changes made/needed: None.

BTW, I rebased my existing patch onto beta, fixed the problems and did moz-phab submit to re-upload it, rather than add a new patch. I'm sorry if this isn't normal/expected. It was hard to to find procedure documentation for this and different people seem to disagree.

Comment 24

[Pascal Chevrel:pascalc (PTO until April 26)]

Comment on attachment 9251498 [details]
Bug 1735482 - Don't use a QuantumWide size class on MacOS r=glandium

Approved for out last 95 beta, thanks.

Comment 25

[Pascal Chevrel:pascalc (PTO until April 26)]

https://hg.mozilla.org/releases/mozilla-beta/rev/9ecfcec976f6