Closed Bug 1359624 Opened 7 years ago Closed 7 years ago

Crash in libsystem_kernel.dylib@0x19d42 called from nsAuthSambaNTLM::SpawnNTLMAuthHelper() with Mac Sierra. Need to disable ntlm_auth

Categories

(Core :: Networking, defect)

53 Branch
Unspecified
macOS
defect
Not set
critical

Tracking

()

VERIFIED FIXED
mozilla57
Tracking Status
thunderbird_esr52 56+ fixed
firefox-esr52 --- wontfix
firefox53 + wontfix
firefox54 + wontfix
firefox55 + wontfix
firefox56 + fixed
firefox57 + fixed

People

(Reporter: jesup, Assigned: bagder)

References

Details

(Keywords: crash, regression, topcrash-thunderbird, Whiteboard: [ntlm][necko-backlog][startupcrash][tbird topcrash])

Crash Data

Attachments

(1 file, 1 obsolete file)

This bug was filed from the Socorro interface and is 
report bp-48488ba5-c600-444a-9cdf-bf92a0170425.
=============================================================

This is for the subset of the signature where nsAuthSambaNTLM::SpawnNTLMAuthHelper() (in extensions/) was on the stack.  Last call outside the kernel was to fork() in _MD_CreateUnixProcess().

Lots of crashes in the last week - ~450 -- all in 52 and newer, including a bunch in Thunderbird 52.
No idea who should get this, but the crash has spiked and is recent
Flags: needinfo?(mozillamarcia.knous)
Flags: needinfo?(jduell.mcbugs)
This is spiking on 52.0.2 _and_ on 45.8.0 (~800 crashes in 6 months).  Probably one of the [NTLM] fixes landed on or around 52 causes this.
Assignee: nobody → honzab.moz
Flags: needinfo?(mozillamarcia.knous)
Flags: needinfo?(jduell.mcbugs)
Summary: Crash in libsystem_kernel.dylib@0x19d42 called from nsAuthSambaNTLM::SpawnNTLMAuthHelper() → [ntlm] Crash in libsystem_kernel.dylib@0x19d42 called from nsAuthSambaNTLM::SpawnNTLMAuthHelper()
Summary: [ntlm] Crash in libsystem_kernel.dylib@0x19d42 called from nsAuthSambaNTLM::SpawnNTLMAuthHelper() → Crash in libsystem_kernel.dylib@0x19d42 called from nsAuthSambaNTLM::SpawnNTLMAuthHelper()
Whiteboard: [ntlm]
Whiteboard: [ntlm] → [ntlm][necko-active]
[Tracking Requested - why for this release]:
this issue is a sizeable crash on 53.0 where it's around 10% of browser crashes for users on macos.

the issue seems to affect only users on mac os 10.12. it also looks like a regression in firefox 53 (as there may have been a regeneration of crash signatures since the bug was created - the signature of the report in comment #0 apparently changed), crashes with version 52 seem to all come from thunderbird:
https://crash-stats.mozilla.com/search/?proto_signature=~nsAuthSambaNTLM%3A%3ASpawnNTLMAuthHelper&product=Firefox&platform=Mac%20OS%20X&date=%3E%3D2016-11-04T10%3A54%3A00.000Z&_facets=signature&_facets=version&_facets=user_comments&_facets=install_time&_facets=platform_version#facet-version

the issue triggers a lot of user comments indicating that it is a severe (& probably reproducible) issue for affected users, many of them tying the crash to being connected through a proxy/vpn/corporate firewall:
https://crash-stats.mozilla.com/signature/?product=Firefox&proto_signature=~nsAuthSambaNTLM%3A%3ASpawnNTLMAuthHelper&signature=__pthread_kill&date=%3E%3D2017-03-01T00%3A00%3A00.000Z#comments
Crash Signature: [@ libsystem_kernel.dylib@0x19d42] → [@ libsystem_kernel.dylib@0x19d42] [@ __pthread_kill] [@ libsystem_kernel.dylib@0x19dd6] [@ libsystem_kernel.dylib@0x19dda]
Keywords: regression
Version: unspecified → 53 Branch
Makes sense to track this for 54/55, for investigation. Not sure we would be able to do much about 53.

What are the next steps here?
This looks pretty bad for Mac users. This seems to be the most common signature, for MacOS 10.12 https://crash-stats.mozilla.com/signature/?signature=__pthread_kill

If we fix this it could be a ridealong on 53 if we end up with another desktop dot release. No plans for one yet, though.
Crash Signature: [@ libsystem_kernel.dylib@0x19d42] [@ __pthread_kill] [@ libsystem_kernel.dylib@0x19dd6] [@ libsystem_kernel.dylib@0x19dda] → [@ libsystem_kernel.dylib@0x19d42] [@ __pthread_kill] [@ libsystem_kernel.dylib@0x19dd6] [@ libsystem_kernel.dylib@0x19dda] [@ __pthread_kill | abort | free | _arc4_fork_child ]
Hi Selena, I pinged you on IRC about this top crasher. It came up in the channel meeting today. We would like some dev help here to investigate and potentially fix this top crasher. Thanks!
Flags: needinfo?(sdeckelmann)
It would be great if someone could try to reproduce this.  I don't currently own any OSX running machine and don't have an easy way to install a virtual.

I can *try* to provide a public test page to trigger this issue (no promises it would be reliable to reproduce, tho).  Other way is to setup IIS server on a windows machine (7-10 Pro) and setup Windows Integrated authentication on some folder exposed via HTTP.  It's relatively easy, no need for domain controllers at all.
Assignee: honzab.moz → nobody
Whiteboard: [ntlm][necko-active] → [ntlm]
So, the page you may try to reproduce the crash with is at:

https://janbambas.cz/moz/ntlm/
So :mhowell just tried to reproduce with both a successful and unsuccessful login in win10 Pro and could not. I sent email to pi-request for additional help.
To clarify, I was running IIS on Win10 Pro and logging into it from a macOS 10.12 system.
(In reply to Honza Bambas (:mayhemer) from comment #9)
> So, the page you may try to reproduce the crash with is at:
> 
> https://janbambas.cz/moz/ntlm/

What's needed to reproduce? I tried this site both with "cancel" and entering blank or dummy creds and didnt see a crash. Im on OSX 10.12.24 with Firefox 53 (release) and nightly (buildid: 20170516122050)
Whiteboard: [ntlm] → [ntlm][necko-backlog]
(In reply to Paul Theriault [:pauljt] from comment #12)
> (In reply to Honza Bambas (:mayhemer) from comment #9)
> > So, the page you may try to reproduce the crash with is at:
> > 
> > https://janbambas.cz/moz/ntlm/
> 
> What's needed to reproduce? I tried this site both with "cancel" and
> entering blank or dummy creds and didnt see a crash. Im on OSX 10.12.24 with
> Firefox 53 (release) and nightly (buildid: 20170516122050)

Thanks for looking into this.

Try adding "https://janbambas.cz" to "network.automatic-ntlm-auth.trusted-uris" pref.  Looking into the code, that's the necessary condition to trigger the crashing code path.
Flags: needinfo?(ptheriault)
I made that change, and got a system but not a firefox crash. I'll pass it along out of band.
Sorry Honza, I first assumed that Selena had given you all you needed, but maybe that wasnt the case. ANyways, I just tested, setting the pref as you said in comment 13 and I'm not able to reproduce on OSX (nightly)

I tried both https://janbambas.cz/moz/ntlm/ and https://janbambas.cz/moz/ntlm/?non-sticky, both with and without creds, and also cancelling the prompt for creds.
Flags: needinfo?(ptheriault)
(In reply to Paul Theriault [:pauljt] from comment #15)
> Sorry Honza, I first assumed that Selena had given you all you needed, but
> maybe that wasnt the case. ANyways, I just tested, setting the pref as you
> said in comment 13 and I'm not able to reproduce on OSX (nightly)
> 
> I tried both https://janbambas.cz/moz/ntlm/ and
> https://janbambas.cz/moz/ntlm/?non-sticky, both with and without creds, and
> also cancelling the prompt for creds.

She has sent me a "system crash" with not more data to gain from than from regular crash reporter records.  Also, OSX is not my development platform at all and this is an OS or near-OS problem, not a networking one as it appears. 

The first build id having this crash is 
Firefox 20170413192749, version 53.0. (m-r@d345b657d381)
Thunderbird 20170413214957, version 52.0.1

Hence, quite broad.

The range 52.0 Firefox to 53.0 Firefox is m-r@9a3506a37e59 - m-r@d345b657d381.  There is no change to NSPR in 53.0.  Only change I found close to this code between 52 and 53 is bug 1344305, but it's off the table, since we have a TB 52.0.1 crash based on esr52 _not_ containing that fix.

Hence, I really don't know what more to do.  Someone needs to debug this and see if the the handle we are forking is invalid and understand exactly what the crash is - is it a access violation or is it a system assertion failure or something else?  Also I don't understand the NSPR code being in fault here and can't say who would be the best person to debug that part of the code and give more clues :(
Too late at this point for 53. And probably for 54 unless we find someone else who digs into this and a way to test better.
Jim -- is this something your team might be able to dig into?
Flags: needinfo?(sdeckelmann) → needinfo?(jmathies)
Assignee: nobody → daniel
Selena, just FYI, Daniel is willing to try to reproduce this bug and if successful, debug what's going on.
On my mac mini running the latest macOS (10.12.5), I tried the https://janbambas.cz/moz/ntlm/ link (after editing network.automatic-ntlm-auth.trusted-uris as comment #13 suggests).

I just tested with Firefox 53, Firefox Nightly 55 and my own local build straight off github/gecko-dev this morning.

Cannot reproduce! =(
You may ask Selena or maybe you can dig something out from her email I forwarded to you.
Tracy, can you try to repo this please? If not, ping spohl, see if he can. 

- running osx 10.12, fx 53
- add "https://janbambas.cz" to "network.automatic-ntlm-auth.trusted-uris" pref
- visit https://janbambas.cz/moz/ntlm/
Flags: needinfo?(twalker)
No crash here with Firefox 53.0.3 on macOSX 10.12.5.

I get Authorization: NTLM Type3 -> OK, you successfully authenticated via NTLM with a dummy password.
Flags: needinfo?(twalker)
(In reply to Tracy Walker [:tracy] from comment #23)
> No crash here with Firefox 53.0.3 on macOSX 10.12.5.
> 
> I get Authorization: NTLM Type3 -> OK, you successfully authenticated via
> NTLM with a dummy password.

Exactly the same environment/result here.
Flags: needinfo?(jmathies)
See Also: → 1362238
Since 54 was released, mark 54 won't fix.
The above crash reports seem to never occur on versions >54 but still keeps happening on 54. (Or it changed signature enough to not look like the same anymore.)
__pthread_kill | abort | free | _arc4_fork_child is #4 crash for Macs in TB52.2.1

Signature in 54.0b3 is libsystem_kernel.dylib@0x19d42  bp-434b2515-7a48-4195-92e3-a92d50170709
Whiteboard: [ntlm][necko-backlog] → [ntlm][necko-backlog][startupcrash][tbird topcrash]
Blocks: 1362238
See Also: 1362238
Summary: Crash in libsystem_kernel.dylib@0x19d42 called from nsAuthSambaNTLM::SpawnNTLMAuthHelper() → Crash in libsystem_kernel.dylib@0x19d42 called from nsAuthSambaNTLM::SpawnNTLMAuthHelper() with Mac Sierra
No crash on Firefox 54.0.1 in macOS 10.12.5.

Removing myself from the CC list. needinfo-me if another test is needed.

:)
Still crashing in Firefox 54.0.1 on macOS 10.12.5 (16F73).

I see 744 crashes (link below) matching the following:

  * "signature" is "__pthread_kill | abort | free | _arc4_fork_child"
  * "proto signature" contains: "SpawnNTLMAuthHelper"
  * "product" has terms: "Firefox"
  * "version" has terms: "54.0.1"

All of the crashes appear to have occurred on macOS 10.12.

https://crash-stats.mozilla.com/signature/?product=Firefox&proto_signature=~SpawnNTLMAuthHelper&version=54.0.1&signature=__pthread_kill%20%7C%20abort%20%7C%20free%20%7C%20_arc4_fork_child&date=%3E%3D2017-06-18T18%3A13%3A12.000Z&date=%3C2017-12-18T18%3A13%3A00.000Z
my organization uses an internal app that reliably reproduces this crash.  let me know if I can help.
hi tquan, if you can easily reproduce the crash on a particular website or with a certain sequence of inputs, it would be great if you could run the mozregression tool from http://mozilla.github.io/mozregression/install on the affected device - it will automatically download & run various firefox versions and ask you after each one if it contained the bug or not until it ends up at a particular patch/change in the code that triggered this problem.

you can start the tool with "mozregression --good 52" and then work your way through it until it provides you with a pushlog or regressing bug. that would be the crucial piece of information to move this bug forward. thank you!

http://mozilla.github.io/mozregression/documentation/usage.html
Here's how you can reproduce it yourself

https://bugzilla.mozilla.org/show_bug.cgi?id=1359624#c13 is not correct.  That's not the right syntax for the "network.automatic-ntlm-auth.trusted-uris" pref (or at least not the one that triggers the bug)

to trigger the bug:

1.  for "network.automatic-ntlm-auth.trusted-uris" pref set it to simply "janbambas.cz" (without the quotes)
2.  visit the previously created test site https://janbambas.cz/moz/ntlm/
3.  Crash reporter appears

our organization used "network.automatic-ntlm-auth.trusted-uris" pref of the above format and it stopped working in some recent Firefox release on MacOS 10.12.  if I get some cycles I will try mozregression later today to figure out where the bug was introduced.  the documentation for this parameter does say that a simple domain (not a URL) is allowed as a value for this preference.  https://developer.mozilla.org/en-US/docs/Mozilla/Integrated_authentication says 

site-list = "mydomain.com, https://myotherdomain.com"

I'm on Firefox 54.0.1, MacOS 10.12.6.
btw, you might have to restart the browser after changing network.automatic-ntlm-auth.trusted-uris pref.
(In reply to [:philipp] from comment #32)
> hi tquan, if you can easily reproduce the crash on a particular website or
> with a certain sequence of inputs, it would be great if you could run the
> mozregression tool from http://mozilla.github.io/mozregression/install on
> the affected device - it will automatically download & run various firefox
> versions and ask you after each one if it contained the bug or not until it
> ends up at a particular patch/change in the code that triggered this problem.
> 
> you can start the tool with "mozregression --good 52" and then work your way
> through it until it provides you with a pushlog or regressing bug. that
> would be the crucial piece of information to move this bug forward. thank
> you!
> 
> http://mozilla.github.io/mozregression/documentation/usage.html

philipp, I couldn't reproduce the bug using mozregression, even using the profile that I used to reproduce the bug in https://bugzilla.mozilla.org/show_bug.cgi?id=1359624#c33 .  I'm not sure why it doesn't reproduce in mozregression but does reproduce with the actual browser.  Can you verify that you're able to reproduce the bug that way?  At least it's a start. As mentioned, NTLM is used in corporate environments quite a bit so it would be good to nail this one down.
sorry, i don't have any system with macos available, so i cannot try to reproduce it myself...
Could please somebody with a deeper OSX development experience sit down and find out what this crash actually is?  I believe the real problem is in the ntlm binary and not firefox.  OSX update probably exposes a bug in the ntlm binary or there is an additional check the ntlm binary doesn't pass anymore or something like that...

Adding some samba folks.
is it possible that the ntlm binary isn't included with mozregression, or that mozregression cannot "catch" crashes that happen with that binary?
I doubt you will find ntlm_auth on a MacOS box, they ditched Samba years ago (GPLv3), so this whole thing should be disabled, the internal NTLM support will have to do, and there won't be any cached passwords anyway (mac AD login never used winbind).
(In reply to Andrew Bartlett from comment #39)
> I doubt you will find ntlm_auth on a MacOS box, they ditched Samba years ago
> (GPLv3), so this whole thing should be disabled, the internal NTLM support
> will have to do, and there won't be any cached passwords anyway (mac AD
> login never used winbind).

Thanks, that's a good point.  So, it looks like we are facing an OSX bug here after all.  No idea why they are crashing instead of reporting a runtime error.

Selena, do you have ntlm_auth binary on the system you experienced the crash in comment 14?  If not, I think we have to check its existence on the system (I don't want to break anyone who potentially installed it manually by completely disabling this feature.)

Thanks.
Flags: needinfo?(sdeckelmann)
(In reply to Honza Bambas (:mayhemer) from comment #40)
> (In reply to Andrew Bartlett from comment #39)
> > I doubt you will find ntlm_auth on a MacOS box, they ditched Samba years ago
> > (GPLv3), so this whole thing should be disabled, the internal NTLM support
> > will have to do, and there won't be any cached passwords anyway (mac AD
> > login never used winbind).
> 
> Thanks, that's a good point.  So, it looks like we are facing an OSX bug
> here after all.  No idea why they are crashing instead of reporting a
> runtime error.
> 
> Selena, do you have ntlm_auth binary on the system you experienced the crash
> in comment 14?  If not, I think we have to check its existence on the system
> (I don't want to break anyone who potentially installed it manually by
> completely disabling this feature.)
> 
> Thanks.

Yup, Apple removed Samba from MacOS in all versions 10.7 and higher: http://appleinsider.com/articles/11/03/23/inside_mac_os_x_10_7_lion_server_apple_replaces_samba_for_windows_networking_services.html

I'm on MacOS 10.12, can reproduce the crash, and do *not* have the ntlm_auth binary on the system.
I can easily reproduce this with the steps in comment #33. I'm on Firefox 54.0.1, MacOS 10.12.5.

I observed some weird things. First, I can only reproduce this when I lunch firefox with Launchpad, but never reproduce this when launching firefox by running ./firefox-bin with my shell. Second, when crash reporter shows up, my firefox is still working. It looks like that the crashed process is another one but not firefox. Not sure if it's the same situation as commentt #14 described.

BTW, I can't reproduce this using nightly with the same profile, neither lunching firefox by Launchpad nor shell.

Let me know if there is anything I can help. Thanks.
(In reply to Kershaw Chang [:kershaw] from comment #42)
> Let me know if there is anything I can help. Thanks.

Do you have the ntlm_auth binary on that system?
Flags: needinfo?(kechang)
(In reply to Honza Bambas (:mayhemer) from comment #43)
> (In reply to Kershaw Chang [:kershaw] from comment #42)
> > Let me know if there is anything I can help. Thanks.
> 
> Do you have the ntlm_auth binary on that system?

No, I can't find this file on my system.
Flags: needinfo?(kechang)
(In reply to Kershaw Chang [:kershaw] from comment #42)
> I can easily reproduce this with the steps in comment #33. I'm on Firefox
> 54.0.1, MacOS 10.12.5.
> 
> I observed some weird things. First, I can only reproduce this when I lunch
> firefox with Launchpad, but never reproduce this when launching firefox by
> running ./firefox-bin with my shell. Second, when crash reporter shows up,
> my firefox is still working. It looks like that the crashed process is
> another one but not firefox. Not sure if it's the same situation as commentt
> #14 described.
> 
> BTW, I can't reproduce this using nightly with the same profile, neither
> lunching firefox by Launchpad nor shell.
> 
> Let me know if there is anything I can help. Thanks.

All of the above are true for me as well and other folks in my organization who can reproduce the bug. None of us have the ntlm_auth binary.

It does appear that this functionality likely never worked on Macs anyway as mentioned in comment #39. Even if we had the ntlm_auth binary the Mac just doesn't have this functionality. If that is the case can we simply disable the usage of ntlm_auth on Macs as the author of that comment suggests?
(In reply to Honza Bambas (:mayhemer) from comment #40)

> Selena, do you have ntlm_auth binary on the system you experienced the crash
> in comment 14?  If not, I think we have to check its existence on the system
> (I don't want to break anyone who potentially installed it manually by
> completely disabling this feature.)


I don't have that binary on my system
Flags: needinfo?(sdeckelmann)
Too late for 55 but happy to take a fix for 56
Daniel, can you work up a patch?

Being a startup crash makes this pretty bad.  Sample of half the comments
 Crashes every time I start Thunderbird since the upgrade :-( :-(
 This new update is UNUSABLE! Please, please test this crap BEFORE you publish it. Looks like I'm going to have to find a new Email Client. Nice work folks.
 When I upgraded to this version during starting this message shows. The thunderbird is started and when I click exit the window with thunderbird is opened and it works. But it hangs a lot. It seams to me that it starts two thunderbirds and the second now is crushing.
Aparece esta ventana de informe de fallo cada vez que se abre Thunderbird. Gracias
Flags: needinfo?(daniel)
Summary: Crash in libsystem_kernel.dylib@0x19d42 called from nsAuthSambaNTLM::SpawnNTLMAuthHelper() with Mac Sierra → Crash in libsystem_kernel.dylib@0x19d42 called from nsAuthSambaNTLM::SpawnNTLMAuthHelper() with Mac Sierra. Need to disable ntlm_auth
I've never been able to reproduce, and the crash stats from way before in this bug were only up to Firefox 54. I presume because the signature changed? Comment #47 implies this still happens in 56 and 57?
Assignee: daniel → nobody
Flags: needinfo?(daniel)
(oops unassigned by mistake, but its not a bad idea for someone who can reproduce to take this)
Assignee: nobody → daniel
Daniel, the plan is to disable the nsAuthSambaNTLM module on OSX.  You can either do it in nsHttpNTLMAuth::ChallengeReceived (preferred - we already have some Windows specific #ifdefs there) or conditionally remove it at [1] from the factory list.


[1] https://dxr.mozilla.org/mozilla-central/rev/56188620cce00b19700fbb8efaafea65e6ca8c61/extensions/auth/nsAuthFactory.cpp#213
Honza, you means simply like this?
Attachment #8901678 - Flags: review?(honzab.moz)
Comment on attachment 8901678 [details] [diff] [review]
0001-Bug-1359624-disable-nsAuthSambaNTLM-module-on-OSX-r-.patch

Review of attachment 8901678 [details] [diff] [review]:
-----------------------------------------------------------------

let's try it.
Attachment #8901678 - Flags: review?(honzab.moz) → review+
BTW this still reproduces in Firefox 55, but instead of a crash the browser hangs/becomes unresponsive.

I'm happy to test the fix once a build that has it is available...I assume some upcoming nightly build will get it?
(In reply to Tony Quan from comment #54)
> BTW this still reproduces in Firefox 55, but instead of a crash the browser
> hangs/becomes unresponsive.
> 
> I'm happy to test the fix once a build that has it is available...I assume
> some upcoming nightly build will get it?

Yes, the day after you see messages that Daniel/bot has landed it on mozilla-centra.
Keywords: checkin-needed
But ok, let's do a try first.
Keywords: checkin-needed
Keywords: checkin-needed
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/3b924184c333
Disable nsAuthSambaNTLM module on OSX. r=mayhemer
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/3b924184c333
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla57
Jorg, 
I could be wrong, but I don't foresee Firefox taking this on esr.
do you see a need for this to go through beta before us taking for 52.x?
Flags: needinfo?(jorgk)
"A need", hmm, I don't watch crash stats, I watch C-C treeherder ;-)
I'll take your advice on taking that one-line change.
Flags: needinfo?(jorgk)
I was able to confirm with Nightly that this bug is fixed (no further crash/hang on NTLM authentication on MacOS Sierra).  the issue still reproduces for me on Firefox 55 with the same profile.
Please request Beta/ESR52 approval on this when you get a chance.
Flags: needinfo?(daniel)
Comment on attachment 8906530 [details] [diff] [review]
v2-0001-Bug-1359624-disable-nsAuthSambaNTLM-module-on-OSX.patch

[Approval Request Comment]
If this is not a sec:{high,crit} bug, please state case for ESR consideration:
User impact if declined:

- It is a crashing bug that will keep annoying users (on mac).

Fix Landed on Version:

- in nightly/57

Risk to taking this patch (and alternatives if risky): 

- I'd call it very low risk since it doesn't change any actual code, just disables/removes a part.

String or UUID changes made by this patch: 

- none

See https://wiki.mozilla.org/Release_Management/ESR_Landing_Process for more info.
Flags: needinfo?(daniel)
Attachment #8906530 - Flags: approval-mozilla-esr52?
Attachment #8906530 - Flags: approval-mozilla-beta?
(In reply to Daniel Stenberg [:bagder] from comment #64)
> Comment on attachment 8906530 [details] [diff] [review]
> v2-0001-Bug-1359624-disable-nsAuthSambaNTLM-module-on-OSX.patch
> 
> [Approval Request Comment]
> If this is not a sec:{high,crit} bug, please state case for ESR
> consideration:
> User impact if declined:
> 
> - It is a crashing bug that will keep annoying users (on mac).

I know you don't mean in the sense of "merely annoying".  So to clarify the impact from the Thunderbird perspective, this is a topcrash for Mac (3rd only behind bug 1365600 and bug 1365136), made worse because >60% of users crash on startup which effectively makes the current version of Thunderbird totally unusable for some class of Mac users. (The crash volume has gone down some each of the last few months, but that's probably from users going back to a prior version or abandoning Thunderbird)
(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #65)

> > - It is a crashing bug that will keep annoying users (on mac).
> 
> I know you don't mean in the sense of "merely annoying".

I mean it in the exact sense I wrote it. It crashes Firefox, which annoys users. What else do you think I should say is the impact of leaving out this fix? Isn't that an accurate description? I don't know the current crash frequency so I didn't include that.

> So to clarify the impact from the Thunderbird perspective

I didn't ask for this on behalf of Thunderbird, but for Firefox. I actually don't have many clues how this impacts Thunderbird, other than what you're telling me here. My comments above is about what the impact on Firefox would be if we didn't merge the fix into those branches. I believe I'm following the correct procedure.
Comment on attachment 8906530 [details] [diff] [review]
v2-0001-Bug-1359624-disable-nsAuthSambaNTLM-module-on-OSX.patch

This looks like a low volume crash on 56, but it might become worse in release. Fix verified on nightly in comment 62. Let's uplift this for beta 12.
Attachment #8906530 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #67)
> This looks like a low volume crash on 56...

I haven't tried a 56 build, but in 55.0.3, the whole Firefox UI simply hangs instead of crashing the process, and the app must be killed.  This was first mentioned in comment 54; it might explain the drop in crash volume.
badger, thank you for clarifying - I stand corrected. 

Unrelated - there can be other crash signatures. For example  EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER per bp-8559bf21-5ef4-4d54-ad50-a37d70170907 user comment "changing network" (same user sees  __pthread_kill | abort | free | _arc4_fork_child per bp-1ccd7603-5467-4e89-9985-9629f0170907 )
Comment on attachment 8906530 [details] [diff] [review]
v2-0001-Bug-1359624-disable-nsAuthSambaNTLM-module-on-OSX.patch

this doesn't seem to be badly affecting esr52 so I'll skip this for esr52.4
Attachment #8906530 - Flags: approval-mozilla-esr52? → approval-mozilla-esr52-
(In reply to Julien Cristau [:jcristau] from comment #71)
> Comment on attachment 8906530 [details] [diff] [review]
> v2-0001-Bug-1359624-disable-nsAuthSambaNTLM-module-on-OSX.patch
> 
> this doesn't seem to be badly affecting esr52 so I'll skip this for esr52.4

I tested this bug on esr52 and it doesn't reproduce there using the same NTLM configuration where it reproduces on Firefox 55. So I think you're fine.
There is a very significant reduction in the vicinity of TB52.4.0 per graph for 
* libsystem_kernel.dylib@0x19dd6  - https://crash-stats.mozilla.com/signature/?product=Thunderbird&signature=libsystem_kernel.dylib%400x19dd6&date=%3E%3D2017-08-30T04%3A00%3A04.000Z&date=%3C2017-11-30T02%3A00%3A04.000Z&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_columns=install_time&_sort=-date&page=1#graphs
* __pthread_kill | abort | free | _arc4_fork_child - https://crash-stats.mozilla.com/signature/?signature=__pthread_kill%20%7C%20abort%20%7C%20free%20%7C%20_arc4_fork_child&date=%3E%3D2017-05-30T09%3A54%3A59.000Z&date=%3C2017-11-30T08%3A54%3A59.000Z#graphs
so the patch is fairly successful for Thunderbird

However, neither signature is totally gone.  __pthread_kill | abort | free | _arc4_fork_child for example ranks ~18 in Thunderbird Mac crashes, with an equal split between 10.12 and 10.13 users.   https://crash-stats.mozilla.com/signature/?product=Thunderbird&platform=Mac%20OS%20X&version=52.4.0&version=52.5.0&signature=__pthread_kill%20%7C%20abort%20%7C%20free%20%7C%20_arc4_fork_child&date=%3E%3D2017-08-30T05%3A59%3A01.000Z&date=%3C2017-11-30T03%3A59%3A01.000Z&_columns=date&_columns=product&_columns=version&_columns=reason&_columns=address&_columns=user_comments&_sort=-date&page=1  ...

for example bp-100cb3f2-e0b3-4ce2-9dae-296250171129

So might file some new bugs. I didn't check the Firefox side. Additional thoughts welcome
Status: RESOLVED → VERIFIED
(In reply to Wayne Mery (:wsmwk) from comment #75)
> However, neither signature is totally gone.  __pthread_kill | abort | free |
> _arc4_fork_child for example ranks ~18 in Thunderbird Mac crashes, with an
> equal split between 10.12 and 10.13 users.  
> https://crash-stats.mozilla.com/signature/
> ?product=Thunderbird&platform=Mac%20OS%20X&version=52.4.0&version=52.5.
> 0&signature=__pthread_kill%20%7C%20abort%20%7C%20free%20%7C%20_arc4_fork_chil
> d&date=%3E%3D2017-08-30T05%3A59%3A01.000Z&date=%3C2017-11-30T03%3A59%3A01.
> 000Z&_columns=date&_columns=product&_columns=version&_columns=reason&_columns
> =address&_columns=user_comments&_sort=-date&page=1  ...
> 
> for example bp-100cb3f2-e0b3-4ce2-9dae-296250171129
> 
> So might file some new bugs. I didn't check the Firefox side. Additional
> thoughts welcome
Flags: needinfo?(honzab.moz)
I'm quite certain that this bug is gone in Firefox 56 and higher, as I had an internal corporate website that I use daily and which I couldn't use with Firefox until the bug was fixed. Unfortunately I don't have a repro scenario for Thunderbird so can't say why the bug might be lingering there.
I looked at the crash search in comment 75 and there is nothing related to this particular ntlm bug.  If you feel a need to file new bugs, then feel free to do so, but I'm not even sure about a component to file them at.  All reports show just a weird stack:

https://crash-stats.mozilla.com/report/index/c6226076-003b-42fa-a79a-d94e90180109

Thread (0)

__pthread_kill
abort
free
_arc4_fork_child
_libc_fork_child
libSystem_atfork_child
fork
HIS_XPC_SetNetworkLocation
___ZL14SwitchLocationP13OpaqueMenuReft_block_invoke
_dispatch_call_block_and_release
_dispatch_client_callout
_dispatch_queue_override_invoke
_dispatch_root_queue_drain
_dispatch_worker_thread3
_pthread_wqthread
start_wqthread
dispatch_source_set_cancel_handler

all other threads are already gone.  according the uptime, this looks like a shutdown crash.
Flags: needinfo?(honzab.moz)
You need to log in before you can comment on or make changes to this bug.