Bug 1347075 (CVE-2017-5464)

negative size memmove in mozilla::a11y::Accessible::InsertChildAt

RESOLVED FIXED in Firefox -esr45
(NeedInfo from)

Status

()

defect
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: nils, Assigned: surkov, NeedInfo)

Tracking

(Blocks 2 bugs, {csectype-intoverflow, sec-high})

unspecified
mozilla55
Points:
---
Dependency tree / graph
Bug Flags:
sec-bounty +

Firefox Tracking Flags

(firefox-esr4553+ fixed, firefox52 wontfix, firefox-esr5253+ fixed, firefox53+ fixed, firefox54+ fixed, firefox55+ fixed)

Details

(Whiteboard: [post-critsmash-triage][adv-main53+][adv-esr45.9+][adv-esr52.1+])

Attachments

(2 attachments)

(Reporter)

Description

2 years ago
The following testcase crashes the latest ASAN build of Firefox (BuildID=20170313233726).

<script>
function start() {
	fuzzPriv.enableAccessibility();
	o2=document.createElement('iframe');
	document.body.appendChild(o2);
	o51=document.body;
	o151=document.createElement('iframe');
	o233=(new DOMParser()).parseFromString("<div><div>","text/html");
	o236=o233.all[2];
	o256=o233.all[3];
	o256.appendChild(o51);
	document.replaceChild(o233.documentElement,document.documentElement);
	document.documentElement.appendChild(o151);
	o151.contentWindow.onresize=fun0;
	document.documentElement.replaceWith(o236,"x");
}
function fun0 () {
	document.documentElement.appendChild(o256);
	o51.innerHTML="<marquee>><p><ul id='id56'>";
	document.documentElement.setAttribute('aria-owns','id56');
	setTimeout("location.reload()", 100);
}
</script>
<body onload="start()"></body>

ASAN output:
=================================================================
==1262==ERROR: AddressSanitizer: negative-size-param: (size=-8)
    #0 0x49beeb in __asan_memmove /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:425:3
    #1 0x7f9e4c384d50 in MoveOverlappingRegion /home/worker/workspace/build/src/obj-firefox/dist/include/nsTArray.h:620:5
    #2 0x7f9e4c384d50 in ShiftData<nsTArrayInfallibleAllocator> /home/worker/workspace/build/src/obj-firefox/dist/include/nsTArray-inl.h:272
    #3 0x7f9e4c384d50 in InsertElementAt<mozilla::a11y::Accessible *&, nsTArrayInfallibleAllocator> /home/worker/workspace/build/src/obj-firefox/dist/include/nsTArray.h:2064
    #4 0x7f9e4c384d50 in mozilla::a11y::Accessible::InsertChildAt(unsigned int, mozilla::a11y::Accessible*) /home/worker/workspace/build/src/accessible/generic/Accessible.cpp:2113
    #5 0x7f9e4c3a6d42 in mozilla::a11y::DocAccessible::MoveChild(mozilla::a11y::Accessible*, mozilla::a11y::Accessible*, int) /home/worker/workspace/build/src/accessible/generic/DocAccessible.cpp:2239:3
    #6 0x7f9e4c3a563b in mozilla::a11y::DocAccessible::PutChildrenBack(nsTArray<RefPtr<mozilla::a11y::Accessible> >*, unsigned int) /home/worker/workspace/build/src/accessible/generic/DocAccessible.cpp:2184:5
    #7 0x7f9e4c3a50cc in mozilla::a11y::DocAccessible::ValidateARIAOwned() /home/worker/workspace/build/src/accessible/generic/DocAccessible.cpp:2027:7
    #8 0x7f9e4c30d0ab in mozilla::a11y::NotificationController::WillRefresh(mozilla::TimeStamp) /home/worker/workspace/build/src/accessible/base/NotificationController.cpp:790:3
    #9 0x7f9e49ba72e8 in nsRefreshDriver::Tick(long, mozilla::TimeStamp) /home/worker/workspace/build/src/layout/base/nsRefreshDriver.cpp:1863:7
    #10 0x7f9e49bb17fa in nsRefreshDriver::FinishedWaitingForTransaction() /home/worker/workspace/build/src/layout/base/nsRefreshDriver.cpp:2159:5
    #11 0x7f9e45305eb7 in mozilla::layers::ClientLayerManager::DidComposite(unsigned long, mozilla::TimeStamp const&, mozilla::TimeStamp const&) /home/worker/workspace/build/src/gfx/layers/client/ClientLayerManager.cpp:495:5
    #12 0x7f9e453edc80 in mozilla::layers::CompositorBridgeChild::RecvDidComposite(unsigned long const&, unsigned long const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&) /home/worker/workspace/build/src/gfx/layers/ipc/CompositorBridgeChild.cpp:588:7
    #13 0x7f9e4472cfeb in mozilla::layers::PCompositorBridgeChild::OnMessageReceived(IPC::Message const&) /home/worker/workspace/build/src/obj-firefox/ipc/ipdl/PCompositorBridgeChild.cpp:1547:20
    #14 0x7f9e4406cdb0 in mozilla::ipc::MessageChannel::DispatchAsyncMessage(IPC::Message const&) /home/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:1795:14
    #15 0x7f9e440692a9 in mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) /home/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:1730:17
    #16 0x7f9e4406b924 in mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) /home/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:1603:5
    #17 0x7f9e4406bf6e in mozilla::ipc::MessageChannel::MessageTask::Run() /home/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:1636:5
    #18 0x7f9e432661b2 in nsThread::ProcessNextEvent(bool, bool*) /home/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1264:7
    #19 0x7f9e43262a60 in NS_ProcessNextEvent(nsIThread*, bool) /home/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:389:10
    #20 0x7f9e44074cbf in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /home/worker/workspace/build/src/ipc/glue/MessagePump.cpp:96:21
    #21 0x7f9e43fe68c8 in RunInternal /home/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:238:3
    #22 0x7f9e43fe68c8 in RunHandler /home/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:231
    #23 0x7f9e43fe68c8 in MessageLoop::Run() /home/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:211
    #24 0x7f9e494d2bcf in nsBaseAppShell::Run() /home/worker/workspace/build/src/widget/nsBaseAppShell.cpp:156:3
    #25 0x7f9e4cd13467 in XRE_RunAppShell() /home/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:854:12
    #26 0x7f9e43fe68c8 in RunInternal /home/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:238:3
    #27 0x7f9e43fe68c8 in RunHandler /home/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:231
    #28 0x7f9e43fe68c8 in MessageLoop::Run() /home/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:211
    #29 0x7f9e4cd12e69 in XRE_InitChildProcess(int, char**, XREChildData const*) /home/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:686:7
    #30 0x4e01b6 in content_process_main /home/worker/workspace/build/src/browser/app/../../ipc/contentproc/plugin-container.cpp:64:19
    #31 0x4e01b6 in main /home/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:286
    #32 0x7f9e5e72082f in __libc_start_main /build/glibc-t3gR2i/glibc-2.23/csu/../csu/libc-start.c:291
    #33 0x41c3d8 in _start (/home/nils/fuzzer3/firefox/firefox+0x41c3d8)

0x6030002fc788 is located 24 bytes inside of 32-byte region [0x6030002fc770,0x6030002fc790)
allocated by thread T0 (Web Content) here:
    #0 0x4b319e in realloc /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:71:3
    #1 0x4e136d in moz_xrealloc /home/worker/workspace/build/src/memory/mozalloc/mozalloc.cpp:105:20
    #2 0x7f9e4302fa16 in Realloc /home/worker/workspace/build/src/obj-firefox/dist/include/nsTArray.h:211:12
    #3 0x7f9e4302fa16 in nsTArrayInfallibleAllocator::ResultTypeProxy nsTArray_base<nsTArrayInfallibleAllocator, nsTArray_CopyWithMemutils>::EnsureCapacity<nsTArrayInfallibleAllocator>(unsigned long, unsigned long) /home/worker/workspace/build/src/obj-firefox/dist/include/nsTArray-inl.h:183
    #4 0x7f9e4c384c5b in AppendElement<mozilla::a11y::Accessible *&, nsTArrayInfallibleAllocator> /home/worker/workspace/build/src/obj-firefox/dist/include/nsTArray.h:2116:32
    #5 0x7f9e4c384c5b in mozilla::a11y::Accessible::InsertChildAt(unsigned int, mozilla::a11y::Accessible*) /home/worker/workspace/build/src/accessible/generic/Accessible.cpp:2109
    #6 0x7f9e4c3a6d42 in mozilla::a11y::DocAccessible::MoveChild(mozilla::a11y::Accessible*, mozilla::a11y::Accessible*, int) /home/worker/workspace/build/src/accessible/generic/DocAccessible.cpp:2239:3
    #7 0x7f9e4c3a563b in mozilla::a11y::DocAccessible::PutChildrenBack(nsTArray<RefPtr<mozilla::a11y::Accessible> >*, unsigned int) /home/worker/workspace/build/src/accessible/generic/DocAccessible.cpp:2184:5
    #8 0x7f9e4c3a50cc in mozilla::a11y::DocAccessible::ValidateARIAOwned() /home/worker/workspace/build/src/accessible/generic/DocAccessible.cpp:2027:7
    #9 0x7f9e4c30d0ab in mozilla::a11y::NotificationController::WillRefresh(mozilla::TimeStamp) /home/worker/workspace/build/src/accessible/base/NotificationController.cpp:790:3
    #10 0x7f9e49ba72e8 in nsRefreshDriver::Tick(long, mozilla::TimeStamp) /home/worker/workspace/build/src/layout/base/nsRefreshDriver.cpp:1863:7
    #11 0x7f9e49bb17fa in nsRefreshDriver::FinishedWaitingForTransaction() /home/worker/workspace/build/src/layout/base/nsRefreshDriver.cpp:2159:5
    #12 0x7f9e45305eb7 in mozilla::layers::ClientLayerManager::DidComposite(unsigned long, mozilla::TimeStamp const&, mozilla::TimeStamp const&) /home/worker/workspace/build/src/gfx/layers/client/ClientLayerManager.cpp:495:5
    #13 0x7f9e453edc80 in mozilla::layers::CompositorBridgeChild::RecvDidComposite(unsigned long const&, unsigned long const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&) /home/worker/workspace/build/src/gfx/layers/ipc/CompositorBridgeChild.cpp:588:7
    #14 0x7f9e4472cfeb in mozilla::layers::PCompositorBridgeChild::OnMessageReceived(IPC::Message const&) /home/worker/workspace/build/src/obj-firefox/ipc/ipdl/PCompositorBridgeChild.cpp:1547:20
    #15 0x7f9e4406cdb0 in mozilla::ipc::MessageChannel::DispatchAsyncMessage(IPC::Message const&) /home/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:1795:14
    #16 0x7f9e440692a9 in mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) /home/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:1730:17
    #17 0x7f9e4406b924 in mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) /home/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:1603:5
    #18 0x7f9e4406bf6e in mozilla::ipc::MessageChannel::MessageTask::Run() /home/worker/workspace/build/src/ipc/glue/MessageChannel.cpp:1636:5
    #19 0x7f9e432661b2 in nsThread::ProcessNextEvent(bool, bool*) /home/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1264:7
    #20 0x7f9e43262a60 in NS_ProcessNextEvent(nsIThread*, bool) /home/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:389:10
    #21 0x7f9e44074cbf in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /home/worker/workspace/build/src/ipc/glue/MessagePump.cpp:96:21
    #22 0x7f9e43fe68c8 in RunInternal /home/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:238:3
    #23 0x7f9e43fe68c8 in RunHandler /home/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:231
    #24 0x7f9e43fe68c8 in MessageLoop::Run() /home/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:211
    #25 0x7f9e494d2bcf in nsBaseAppShell::Run() /home/worker/workspace/build/src/widget/nsBaseAppShell.cpp:156:3
    #26 0x7f9e4cd13467 in XRE_RunAppShell() /home/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:854:12
    #27 0x7f9e43fe68c8 in RunInternal /home/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:238:3
    #28 0x7f9e43fe68c8 in RunHandler /home/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:231
    #29 0x7f9e43fe68c8 in MessageLoop::Run() /home/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:211
    #30 0x7f9e4cd12e69 in XRE_InitChildProcess(int, char**, XREChildData const*) /home/worker/workspace/build/src/toolkit/xre/nsEmbedFunctions.cpp:686:7
    #31 0x4e01b6 in content_process_main /home/worker/workspace/build/src/browser/app/../../ipc/contentproc/plugin-container.cpp:64:19
    #32 0x4e01b6 in main /home/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:286
    #33 0x7f9e5e72082f in __libc_start_main /build/glibc-t3gR2i/glibc-2.23/csu/../csu/libc-start.c:291

SUMMARY: AddressSanitizer: negative-size-param /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:425:3 in __asan_memmove
==1262==ABORTING
Flags: sec-bounty?
Alex maybe a regression from your recent work?
Flags: needinfo?(surkov.alexander)
Group: core-security → dom-core-security
(Assignee)

Comment 2

2 years ago
(In reply to David Bolter [:davidb] from comment #1)
> Alex maybe a regression from your recent work?

from what I can tell it is another broken tree problem, so no - answering your original question. So not yet clear, whether this is indexes problem or parent-child relashinship.
Flags: needinfo?(surkov.alexander)
(Assignee)

Comment 3

2 years ago
Posted patch stop bleedingSplinter Review
A broken tree problem: accessible tree is out of sync with DOM tree, i.e. 'origContainer' contains 'child' and 'prevChild' on DOM level as children, but 'prevChild' has parents before 'origContainer'. 

More debugging is required to figure out how we've got a broken tree. In the meantime we can have a crash stopper fix, which can be safely backported.
Assignee: nobody → surkov.alexander
Attachment #8850658 - Flags: review?(yzenevich)
(Assignee)

Updated

2 years ago
Comment on attachment 8850658 [details] [diff] [review]
stop bleeding

Review of attachment 8850658 [details] [diff] [review]:
-----------------------------------------------------------------

thanks
Attachment #8850658 - Flags: review?(yzenevich) → review+
(Assignee)

Comment 5

2 years ago
Comment on attachment 8850658 [details] [diff] [review]
stop bleeding

[Security approval request comment]
How easily could an exploit be constructed based on the patch? not easy, quite likely the fuzzing is needed

Do comments in the patch, the check-in comment, or tests included in the patch paint a bulls-eye on the security problem? no tests, commit message is "validate insertion index for a child move"

Which older supported branches are affected by this flaw? no info, presumably the problem is old enough to affect all branches

Do you have backports for the affected branches? If not, how different, hard to create, and risky will they be? same patch should work

How likely is this patch to cause regressions; how much testing does it need? not likely, just a correction of broken accessible tree, it shouldn't make thing worse
Attachment #8850658 - Flags: sec-approval?
This needs a security rating before it can receive sec-approval. I don't understand the security implications of the problem. Can you explain them at all?
(Assignee)

Comment 7

2 years ago
who can answer comment #6?
Flags: needinfo?(dveditz)
This sounds bad, so I'll mark it sec-high.
Flags: needinfo?(dveditz)
Going with Andrew here. Giving sec-approval+ for trunk. Please nominate for Aurora, Beta, ESR52, and ESR45 (or make new patches and nominate them if necessary) after it lands on trunk.
Attachment #8850658 - Flags: sec-approval? → sec-approval+
https://hg.mozilla.org/mozilla-central/rev/140d845ace14
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
(Assignee)

Comment 13

2 years ago
The landed patch is a bleed stopper only, a follow-up fix is required to fix the root issue. How is it better to proceed? Should I file a follow up bug for easier this sec bug tracking?
(In reply to alexander :surkov from comment #13)
> The landed patch is a bleed stopper only, a follow-up fix is required to fix
> the root issue. How is it better to proceed? Should I file a follow up bug
> for easier this sec bug tracking?

Yes, please.
Flags: sec-bounty? → sec-bounty+
Group: dom-core-security → core-security-release
(In reply to Al Billings [:abillings] from comment #9)
> Going with Andrew here. Giving sec-approval+ for trunk. Please nominate for
> Aurora, Beta, ESR52, and ESR45 (or make new patches and nominate them if
> necessary) after it lands on trunk.

This didn't happen. It looks like this is going to miss the Firefox 53 release (and associated ESR branches). Is that on purpose?
Flags: needinfo?(surkov.alexander)
(Assignee)

Comment 16

2 years ago
Comment on attachment 8850658 [details] [diff] [review]
stop bleeding

Approval Request Comment
[Feature/Bug causing the regression]:unknown
[User impact if declined]:sec/crash issue
[Is this code covered by automated tests?]:the code is tested overall, not this specific case though
[Has the fix been verified in Nightly?]:yes
[Needs manual test from QE? If yes, steps to reproduce]: shouldn't be necessary, str in the bug description
[List of other uplifts needed for the feature/fix]:no
[Is the change risky?]:low
[Why is the change risky/not risky?]: validity checks/fixes only
[String changes made/needed]:no
Flags: needinfo?(surkov.alexander)
Attachment #8850658 - Flags: approval-mozilla-beta?
Attachment #8850658 - Flags: approval-mozilla-aurora?
Attachment #8850658 - Flags: approval-mozilla-esr52?
Attachment #8850658 - Flags: approval-mozilla-esr45?
Comment on attachment 8850658 [details] [diff] [review]
stop bleeding

Fix for sec-high issue, ok to uplift to aurora and beta. This should land for the beta 10 build tomorrow.
Attachment #8850658 - Flags: approval-mozilla-beta?
Attachment #8850658 - Flags: approval-mozilla-beta+
Attachment #8850658 - Flags: approval-mozilla-aurora?
Attachment #8850658 - Flags: approval-mozilla-aurora+
(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #17)
> Comment on attachment 8850658 [details] [diff] [review]
> stop bleeding
> 
> Fix for sec-high issue, ok to uplift to aurora and beta. This should land
> for the beta 10 build tomorrow.

What about ESR45 and ESR52?
Flags: needinfo?(rkothari)
Flags: needinfo?(lhenry)
Flags: needinfo?(jcristau)
Comment on attachment 8850658 [details] [diff] [review]
stop bleeding

take this in both esr branches too
Flags: needinfo?(rkothari)
Flags: needinfo?(lhenry)
Flags: needinfo?(jcristau)
Attachment #8850658 - Flags: approval-mozilla-esr52?
Attachment #8850658 - Flags: approval-mozilla-esr52+
Attachment #8850658 - Flags: approval-mozilla-esr45?
Attachment #8850658 - Flags: approval-mozilla-esr45+
This needs rebasing for ESR45.
Flags: needinfo?(surkov.alexander)
Whiteboard: [post-critsmash-triage]
(Assignee)

Comment 23

2 years ago
Posted patch esr45 pathSplinter Review
Flags: needinfo?(surkov.alexander)
(Assignee)

Updated

2 years ago
Blocks: 1355488
(Assignee)

Comment 25

2 years ago
(In reply to Andrew McCreight [:mccr8] from comment #14)
> (In reply to alexander :surkov from comment #13)
> > The landed patch is a bleed stopper only, a follow-up fix is required to fix
> > the root issue. How is it better to proceed? Should I file a follow up bug
> > for easier this sec bug tracking?
> 
> Yes, please.

filed, bug 1355488
Alias: CVE-2017-5464
Whiteboard: [post-critsmash-triage] → [post-critsmash-triage][adv-main53+][adv-esr45.9+][adv-esr52.1+]
Reproduced the initial issue using the build and the script given in comment 0 and having domfuzz_helper extension installed. The crash is still reproducible on 55.0a1 (20170418213532) and 54.0a2 (20170418134549) latest linux64-asan-debug builds, using Ubuntu 16.04 x64. Also, using 54.0b1 (20170418132012) and 53.0 (20170418101543) latest linux64-asan-debug builds, the crash isn't reproducible anymore, but Firefox becomes totally unresponsive.
I wasn't able to verify the fix on firefox-esr45 and firefox-esr52 asan builds, because domfuzz_helper extension is not compatible with these builds.
Flags: needinfo?(nils)
See Also: → 1385372
(Assignee)

Comment 27

2 years ago
(In reply to Iulia Cristescu, QA [:JuliaC] from comment #26)
> Reproduced the initial issue using the build and the script given in comment
> 0 and having domfuzz_helper extension installed. The crash is still
> reproducible on 55.0a1 (20170418213532) and 54.0a2 (20170418134549) latest
> linux64-asan-debug builds, using Ubuntu 16.04 x64.

weird, did you have a chance to test the latest nightlies (20170418 seems out of date)? Is the stack same? Could you give a link to crash stats please?

btw, what bug 1385372 (a 'see also' from the previous comment) is about? it seems I'm not authorized for it.
(In reply to alexander :surkov from comment #27)
> (In reply to Iulia Cristescu, QA [:JuliaC] from comment #26)
> > Reproduced the initial issue using the build and the script given in comment
> > 0 and having domfuzz_helper extension installed. The crash is still
> > reproducible on 55.0a1 (20170418213532) and 54.0a2 (20170418134549) latest
> > linux64-asan-debug builds, using Ubuntu 16.04 x64.
> 
> weird, did you have a chance to test the latest nightlies (20170418 seems
> out of date)? Is the stack same? Could you give a link to crash stats please?
> 
> btw, what bug 1385372 (a 'see also' from the previous comment) is about? it
> seems I'm not authorized for it.

Investigated again this issue using 
- 52.3.0esr (20170803170140) linux64-asan-debug build with domfuzz_helper extension installed
- 55.0 (20170803173024) linux64-asan-debug build with domfuzz_helper extension installed
- 56.0b1 (20170804101649) linux64-asan-debug build with domfuzz_helper extension installed
- 57.0a1 (20170803031846) linux64-fuzzing-asan-opt build 
on Ubuntu 16.04 x64
and the crash scenario from the comment 0 is not to reproducible anymore. But Firefox becomes unresponsive on the esr build. 
Any thoughts about these?
Flags: needinfo?(surkov.alexander)
(Assignee)

Comment 29

2 years ago
(In reply to Iulia Cristescu, QA [:JuliaC] from comment #28)

> and the crash scenario from the comment 0 is not to reproducible anymore.
> But Firefox becomes unresponsive on the esr build. 
> Any thoughts about these?

It is weird that esr only is affected. Any chance to make a perf profile?
Flags: needinfo?(surkov.alexander)
(In reply to alexander :surkov from comment #29)
>  Any chance to make a perf profile?

Back with more investigation details. Used Ubuntu 16.04 x64 and:
- 52.4.1esr (20170928162237) linux64-asan-debug build with domfuzz_helper extension installed 
- 58.0a1 (20170929094722) linux64-fuzzing-asan-opt build
- 57.0b4 (20170929070024) linux64-fuzzing-asan-opt build
- 56.0.1 (20170928154249) linux64-asan-debug build with domfuzz_helper extension installed
Following the scenario from comment 0, these are the results:
- the esr build crashes, but I am unable to access the about:crashes page to get the crash report details and also, the performance profile recording couldn't be saved
- the Nightly and the Beta builds work ok 
- the Release build gets unresponsive, but I was still able so save the performance recording profile.
Here you have the profile recordings for Nightly, Beta and esr builds https://goo.gl/UmPyW7. 
Hope that these will help.
Flags: needinfo?(surkov.alexander)
(Assignee)

Comment 31

2 years ago
I managed to open perf profiles in devtools, but they didn't provide much: no stacks, all I can see is which module takes what time. Was that Gecko perf profiling (https://perf-html.io/)?

On the other hand, if esr and release builds are effected only, then anyways we might be not allowed to land a patch (we landed a whole bunch of changes into 57 and most of them are perf related). We probably should get a pre thumbs up from the risk management team, before digging into the problem any further.

I wonder whether the esr52 crash is a crash of this bug or something else, or even somehow connected with the hangs you observe. Is my understanding correct that the about:crashes is empty when you reopen the browser?
Flags: needinfo?(surkov.alexander)
(In reply to alexander :surkov from comment #31)
> I managed to open perf profiles in devtools, but they didn't provide much:
> no stacks, all I can see is which module takes what time. Was that Gecko
> perf profiling (https://perf-html.io/)?

I used the Firefox Developer Tools Performance tool https://developer.mozilla.org/son/docs/Tools/Performance, but I will start over again the investigation and use the tool that you indicated. 

> On the other hand, if esr and release builds are effected only, then anyways
> we might be not allowed to land a patch (we landed a whole bunch of changes
> into 57 and most of them are perf related). We probably should get a pre
> thumbs up from the risk management team, before digging into the problem any
> further.
> 
> I wonder whether the esr52 crash is a crash of this bug or something else,
> or even somehow connected with the hangs you observe. Is my understanding
> correct that the about:crashes is empty when you reopen the browser?

No, simply I am unable to access the about:crashes page using the esr build indicated in the comment 30 (see https://goo.gl/KBxCe2).
(In reply to Iulia Cristescu, QA [:JuliaC] from comment #32)
> (In reply to alexander :surkov from comment #31)
> > I managed to open perf profiles in devtools, but they didn't provide much:
> > no stacks, all I can see is which module takes what time. Was that Gecko
> > perf profiling (https://perf-html.io/)?
> 
> I used the Firefox Developer Tools Performance tool
> https://developer.mozilla.org/son/docs/Tools/Performance, but I will start
> over again the investigation and use the tool that you indicated. 
 
Repeated the investigation for the problematic builds: 
- 56.0.1 (20171004100747) linux64-asan-debug build with domfuzz_helper extension installed
- 52.4.1esr (20171004174042) linux64-asan-debug build with domfuzz_helper extension installed
The esr build crashes right after the Gecko perf profiling add-on installation is finished. And the about:crashes page is still unreachable. 
Here is the performance recording profile for the release build https://goo.gl/t6mU25.
Flags: needinfo?(surkov.alexander)
(Assignee)

Updated

2 years ago
Blocks: ariaowns
Flags: needinfo?(surkov.alexander)
(Assignee)

Comment 34

2 years ago
(In reply to Iulia Cristescu, QA [:JuliaC] from comment #32)

> No, simply I am unable to access the about:crashes page using the esr build
> indicated in the comment 30 (see https://goo.gl/KBxCe2).

ouch, it's weird, you may want to poke around to see if anyone has ideas on it

(In reply to Iulia Cristescu, QA [:JuliaC] from comment #33)

> Repeated the investigation for the problematic builds: 
> - 56.0.1 (20171004100747) linux64-asan-debug build with domfuzz_helper
> extension installed
> - 52.4.1esr (20171004174042) linux64-asan-debug build with domfuzz_helper
> extension installed
> The esr build crashes right after the Gecko perf profiling add-on
> installation is finished. And the about:crashes page is still unreachable. 
> Here is the performance recording profile for the release build
> https://goo.gl/t6mU25.

Thank you. Btw, the most convenient way for both of us is to share a link on a profile, see [1] for instructions, but before we go any further, we probably should doublecheck whether this is something worth to continue to investigate, since release build is affected only, and I have doubts that a patch will be accepted even if we had it. We still are faced to huge performance problems on Nightly.

[1] https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Reporting_a_Performance_Problem
Group: core-security-release
You need to log in before you can comment on or make changes to this bug.