1404298 - Crashes with read-access content sandboxing triggered by mounted volumes

Status: RESOLVED FIXED

(Core :: Security: Process Sandboxing, defect)

Description

[charlie.siegel]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:57.0) Gecko/20100101 Firefox/57.0
Build ID: 20170925150345

Steps to reproduce:

Open both Firefox Stable (v56) and Firefox Developer Edition (v57).  Use both browsers for a while.  Eventually I will get the "Gah your tab crashed" message on one or both of the bowers.

I am running MacOS Sierra 10.12.6.


Actual results:

Once the issue occurs, tabs crash and I can't get any web pages to load.

The issue remains until I reboot my machine.  I had previously tried deleting profiles and starting fresh, but that doesn't resolve the issue.  Only a reboot solves it.


Expected results:

I should have been able to run both browsers simultaneously without issue.  I had also previously seen this same issue on Firefox Nightly (57) when launching a separate instance of nightly with the following command:

/Applications/FirefoxNightly.app/Contents/MacOS/firefox -P --no-remote 

and then selecting a separate dedicate profile for the Nightly version.

I had assumed that I was doing something wrong, but the same thing occurs with the developer edition which is meant to do exactly this.

Comment 1

[YF (Yang)]

https://developer.mozilla.org/en-US/docs/How_to_get_a_stacktrace_for_a_bug_report

Comment 2

[charlie.siegel]

Here you go:

Report ID 	Date Submitted
bp-49f717d4-1c49-4ad3-9b3e-ea14f0170929
	9/29/2017	7:54 AM
bp-ad43a7ec-7d35-46cb-8134-b2bef1170929
	9/29/2017	7:54 AM
bp-d7bb593e-bb57-4c91-a85f-65b7b0170929
	9/29/2017	7:54 AM
bp-f8ea40cb-92ec-45c8-ac57-4b29e0170929
	9/29/2017	7:54 AM
bp-253634a1-03bf-462d-a21a-18fa80170929
	9/29/2017	7:54 AM
bp-9043ec29-edbd-405d-a790-d29500170929
	9/29/2017	7:54 AM
bp-93e9d548-5f62-4a75-a996-f65d41170929
	9/29/2017	7:54 AM
bp-2287c567-d35f-48f0-af40-c06760170929
	9/29/2017	7:54 AM
bp-6663a979-ca99-4db9-8da4-345460170929
	9/29/2017	7:54 AM
bp-c825af19-e249-4a6b-a758-ba8430170929
	9/29/2017	7:54 AM
bp-dc7bb508-d2ca-4a2f-91c1-b098e0170929
	9/29/2017	7:54 AM
bp-d67bff97-2649-4f40-86db-f86d20170929
	9/29/2017	7:54 AM


There's a few more that haven't yet been submitted (it did it again when I opened it up):

Report ID 	Date Crashed
6A4999D1-96EA-482C-83AA-FD96908CD59C
	9/29/2017	10:49 AM
B2E4DF55-488B-46DC-A78B-BD91B2E86F32
	9/29/2017	10:49 AM

Comment 3

[Julien Cristau [:jcristau]]

Lots of crashes with this signature from very few installations on beta.  A few on release as well.

Comment 4

[charlie.siegel]

This reminds me.  I actually started seeing this issue while only running 56 (never having launched any other instance of Firefox) after a fresh reboot.  It's very odd, because I never had this issue with 55 or below before.

Comment 5

[Marcia Knous [:marcia]]

(In reply to charlie.siegel from comment #4)
> This reminds me.  I actually started seeing this issue while only running 56
> (never having launched any other instance of Firefox) after a fresh reboot. 
> It's very odd, because I never had this issue with 55 or below before.

Hello Charlie: I noticed in your reports you are using UBlock - can you reproduce it with 

* a clean profile, no extensions?
* Does it only happen when you are running two browsers at once, or can you reproduce the issue running only 57 Developer Edition?

Right now the latest 57b5 shows only 3 installs, so this doesn't appear to be an extremely widespread issue at the moment.

Comment 6

[Chris Peterson [:cpeterson]]

gcp, is this bug a dupe of "Whitelist extensions dir" bug 1385891?

This bug's crash signature matches ErrorLoadingSheet bug 1384446, which was resolved as a dupe of bug 1385891. 1385891 which was fixed in Firefox 57 and supposedly didn't affect 56, but both this bug and bug 1384446 affect 56.

Comment 7

[Gian-Carlo Pascutto [:gcp]]

(In reply to Chris Peterson [:cpeterson] from comment #6)
> gcp, is this bug a dupe of "Whitelist extensions dir" bug 1385891?

Reporter is on MacOS, not Linux.

Comment 8

[charlie.siegel]

(In reply to Marcia Knous [:marcia - use ni] from comment #5)
> (In reply to charlie.siegel from comment #4)
> > This reminds me.  I actually started seeing this issue while only running 56
> > (never having launched any other instance of Firefox) after a fresh reboot. 
> > It's very odd, because I never had this issue with 55 or below before.
> 
> Hello Charlie: I noticed in your reports you are using UBlock - can you
> reproduce it with 
> 
> * a clean profile, no extensions?
> * Does it only happen when you are running two browsers at once, or can you
> reproduce the issue running only 57 Developer Edition?
> 
> Right now the latest 57b5 shows only 3 installs, so this doesn't appear to
> be an extremely widespread issue at the moment.


Hello,

I haven't run Firefox all day.  I opened it up to load a site in a different browser so I wasn't authenticated.  I'm seeing the 'gah your tab crashed' issue again.

I closed it, wiped out ~/library/application support/firefox

I'm still getting the same thing.  This is a clean profile, no extensions installed on 56.

I also get the same thing with the latest Developer version (57.0b6) (which also has a clean profile)

The pending crash reports for 56 are:

92D6F32C-54B1-478E-9AD4-558E8FCDE057
AFAA9A62-549B-4CFC-8EEB-980588469C59

The pending crash reports for 57 are:
0E48A106-C48A-4AA8-B351-66D02617D54C
22FBC4F8-0642-4388-A7F5-023C522AA0D9

I'll get these submitted the next time I reboot my computer.

Comment 9

[charlie.siegel]

They're submitted.

Comment 10

[Marcia Knous [:marcia]]

From reading the comments is - http://bit.ly/2hzn8nH - there are a handful of people experiencing this running 56 and 57. Some of the reports I looked at have no third party extensions installed, so hard maybe to blame an addon here. So far I haven't see an reports from 58 Nightly.

Comment 11

[charlie.siegel]

I saw that a new beta of 57 came out.  I'm going to try an experiment today.  I've wiped all Firefox / Mozilla Cache/Application Setting / Preferences files and am running 57.0b7 (not the developer version) today for my personal browsing.  I'm keeping basically all settings at default (just changed search provider) and not installing any plugins or signing into sync.

I'll see how this goes and report back here.

Comment 12

[charlie.siegel]

(In reply to charlie.siegel from comment #11)
> I saw that a new beta of 57 came out.  I'm going to try an experiment today.
> I've wiped all Firefox / Mozilla Cache/Application Setting / Preferences
> files and am running 57.0b7 (not the developer version) today for my
> personal browsing.  I'm keeping basically all settings at default (just
> changed search provider) and not installing any plugins or signing into sync.
> 
> I'll see how this goes and report back here.

I've got results from experiment.

I ran Firefox 57.0b7 all morning.  Again, completely fresh profile, no extensions installed, all settings default except switching the search provider.  It ran fine all morning, no issues.

I decided to push it a little bit more after lunch and reinstalled the latest Developer version.  I used this along side the beta for probably about 45 minutes and then I got another "gah, your tab has crashed."  It began happening on both browsers.

Here are a few of the submitted crash reports:

bp-174d185a-4815-43f9-9ce1-95b950171011
	10/11/2017	2:20 PM
bp-17fa97b3-1e83-4604-8547-4a1fc0171011
	10/11/2017	2:20 PM
bp-b6a30a49-4ebc-47c9-bc6e-241230171011
	10/11/2017	2:19 PM
bp-180915ae-3c3b-4859-bb27-d3eaa0171011
	10/11/2017	2:19 PM

Something is definitely wrong here.  I'm not sure what it is, I've isolated this as far as I can on my end.  I won't be able to do any further testing though, I can't spend anymore time on this.  I'll be using different browsers until I see that this is resolved.

Comment 13

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

While this may be a valid bug, I don't consider this a severe issue affecting a significant # of users, pushed tracking nom to 58.

Comment 14

[charlie.siegel]

(In reply to Ritu Kothari (:ritu) from comment #13)
> While this may be a valid bug, I don't consider this a severe issue
> affecting a significant # of users, pushed tracking nom to 58.

FWIW, One of my colleagues who is just running the stable 56 and has never run a beta or multiple instances of Firefox simultaneously had this issue yesterday. He got the gah your tab crashed and it was broken until he rebooted. 

He also is running the latest patch level of MacOS Sierra. 

It may not be affecting large numbers of people, but maybe it's constrained to Sierra?

What your effectively telling me is I won't be able to use Firefox until 2018. Very disappointing.

Comment 15

[Gian-Carlo Pascutto [:gcp]]

(In reply to charlie.siegel from comment #14)
> FWIW, One of my colleagues who is just running the stable 56 and has never
> run a beta or multiple instances of Firefox simultaneously had this issue
> yesterday. 

If he is *not* running multiple versions simultaneously, then he is not suffering from *this* issue, which is that it starts crashing if you *DO* run multiple versions simultaneously...

Please file a new bug.

Comment 16

[Gian-Carlo Pascutto [:gcp]]

Hmm, I notice now you are the original reporter. So maybe the issue isn't related to running multiple versions together at all? Do you both happen to run some similar third party software?

Comment 17

[charlie.siegel]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #16)
> Hmm, I notice now you are the original reporter. So maybe the issue isn't
> related to running multiple versions together at all? Do you both happen to
> run some similar third party software?

I have similar thoughts.  The one thing that comes to mind is we have Sophos Antivirus installed.  Are you able to find out if there are any issues with Firefox when running Sophos?

I haven't experienced this on my home pc, which isn't running Sophos (but is also now running High Sierra).

Comment 18

[charlie.siegel]

It appears Sophos Hitman Pro is likely the culprit: https://bugzilla.mozilla.org/show_bug.cgi?id=1346244#c20

Do you see anything in the crashdump pointing at hmpalert.dll like is referenced in that bug?

Do you have any further insight into that bug or the referenced bugs or can you point me in the right direction?  It looked like they had spoke to someone at Sophos and they were going to fix it, but it appears that it's not completely fixed.

Comment 19

[Gian-Carlo Pascutto [:gcp]]

Given that hmpalert.dll is a Windows DLL, it's probably called something else on Mac OS X. I didn't immediately find any reference to Sophos in the crash stacks, but I have no good idea what to look for.

We might be able to verify this by locally though by having someone obtain a copy. I assume you can't test this yourself (disable/uninstall antivirus) on your work PCs.

Comment 20

[charlie.siegel]

Oh ya(In reply to Gian-Carlo Pascutto [:gcp] from comment #19)
> Given that hmpalert.dll is a Windows DLL, it's probably called something
> else on Mac OS X. I didn't immediately find any reference to Sophos in the
> crash stacks, but I have no good idea what to look for.
> 
> We might be able to verify this by locally though by having someone obtain a
> copy. I assume you can't test this yourself (disable/uninstall antivirus) on
> your work PCs.

Duh, ya good point, no .dll's on Mac :) .  I can find out what the component is on Mac when I get into work this morning.

I actually do have some ability to control our antivirus at work.  I'll see what I can do.  What exactly do you need?

I looked this up a little bit and they've actually been targeting fixes to their product specifically for Firefox for a while: 

https://www.wilderssecurity.com/threads/hitmanpro-alert-support-and-discussion-thread.324841/page-383#post-2583777

One of the posts is saying the CPU is much better with this version.  In addition to the crashes I've been experiencing, I have also noticed extremely high CPU usage by Firefox, this could be the culprit.

Comment 21

[Marco Castelluccio [:marco]]

(In reply to charlie.siegel from comment #20)
> I actually do have some ability to control our antivirus at work.  I'll see
> what I can do.  What exactly do you need?

Could you test if the issues (the crash and the high CPU usage) are reproducible with the antivirus disabled?
We can contact Sophos about this issue if you can't reproduce with the antivirus disabled. Are you OK with me sharing your contact details with them, so they can contact you directly should they need more information?

Comment 22

[charlie.siegel]

(In reply to Marco Castelluccio [:marco] from comment #21)
> (In reply to charlie.siegel from comment #20)
> > I actually do have some ability to control our antivirus at work.  I'll see
> > what I can do.  What exactly do you need?
> 
> Could you test if the issues (the crash and the high CPU usage) are
> reproducible with the antivirus disabled?
> We can contact Sophos about this issue if you can't reproduce with the
> antivirus disabled. Are you OK with me sharing your contact details with
> them, so they can contact you directly should they need more information?

Well, I don't think it's Sophos.  I excluded the following processes from the realtime scanner:

FirefoxDeveloperEdition
FirefoxDeveloperEditionCP Web Content

Firefox
FirefoxCP Web Content

And it just happened again, in the exact same way.  I was using Firefox 56 and the Development version together and it worked fine for a while, but then the tabs started crashing in both instances.

I don't know.  Something isn't right, I have no idea what though.  We don't have any other 3rd party software that would be causing this, so I think this is an issue with Firefox.

Comment 23

[Haik Aftandilian [:haik]]

(In reply to charlie.siegel from comment #22)
> (In reply to Marco Castelluccio [:marco] from comment #21)
> > (In reply to charlie.siegel from comment #20)
> > > I actually do have some ability to control our antivirus at work.  I'll see
> > > what I can do.  What exactly do you need?
> > 
> > Could you test if the issues (the crash and the high CPU usage) are
> > reproducible with the antivirus disabled?
> > We can contact Sophos about this issue if you can't reproduce with the
> > antivirus disabled. Are you OK with me sharing your contact details with
> > them, so they can contact you directly should they need more information?
> 
> Well, I don't think it's Sophos.  I excluded the following processes from
> the realtime scanner:
> 
> FirefoxDeveloperEdition
> FirefoxDeveloperEditionCP Web Content
> 
> Firefox
> FirefoxCP Web Content

Could you also try disabling any process named "plugin-container"? Some Firefox processes end up with that generic name.

Completely disabling the antivirus would be a good test if that's possible.

Thanks!

Comment 24

[charlie.siegel]

(In reply to Haik Aftandilian [:haik] from comment #23)
> (In reply to charlie.siegel from comment #22)
> > (In reply to Marco Castelluccio [:marco] from comment #21)
> > > (In reply to charlie.siegel from comment #20)
> > > > I actually do have some ability to control our antivirus at work.  I'll see
> > > > what I can do.  What exactly do you need?
> > > 
> > > Could you test if the issues (the crash and the high CPU usage) are
> > > reproducible with the antivirus disabled?
> > > We can contact Sophos about this issue if you can't reproduce with the
> > > antivirus disabled. Are you OK with me sharing your contact details with
> > > them, so they can contact you directly should they need more information?
> > 
> > Well, I don't think it's Sophos.  I excluded the following processes from
> > the realtime scanner:
> > 
> > FirefoxDeveloperEdition
> > FirefoxDeveloperEditionCP Web Content
> > 
> > Firefox
> > FirefoxCP Web Content
> 
> Could you also try disabling any process named "plugin-container"? Some
> Firefox processes end up with that generic name.
> 
> Completely disabling the antivirus would be a good test if that's possible.
> 
> Thanks!


When I launch Firefox now, it crashes right away (it will do this until I reboot).  There is no process running named "plugin-container" when this happens, so I don't think this it.  When I attempt to launch Firefox again, all I have running is "Firefox".

I just turned Sophos off completely and attempted to relaunch Firefox / Firefox Development and I get the same thing - 'gah your tab crashed.'

I think we can rule Sophos out now.

Comment 25

[Haik Aftandilian [:haik]]

OK. I can think of a few more things to try.

Turn on sandbox violation logging and check the Mac Console for errors related to loading a style sheet. There's a few steps for this:
1) In about:config, set "security.sandbox.logging.enabled" to true and then quit the browser.
2) Before restarting, open the Mac Console application (from /Applications/Utilities/Console) and enter plugin-container in the Search field. 
3) Click "Clear" in Console
4) Then launch Firefox again using the same profile
5) You should start to see some violation logging as soon as you launch Firefox.

Any sandbox violations that show up in Console around the time of the crash or that relate to loading a stylesheet could be related. It's also possible in Console to use Edit->Select All, Edit->Copy, and then paste in a text edit and provide that to help us debug.

--

And something else to test is to disable the sandbox. One of the changes in build 56 relates to sandboxing. To disable the sandbox, you use about:config and change "security.sandbox.content.level" to 0 and then restart the browser. If the problem is not reproducible with security.sandbox.content.level=0, that tells us it's probably sandboxing related. (You have to restart the browser for the change to take effect.)

Comment 26

[charlie.siegel]

(In reply to Haik Aftandilian [:haik] from comment #25)
> OK. I can think of a few more things to try.
> 
> Turn on sandbox violation logging and check the Mac Console for errors
> related to loading a style sheet. There's a few steps for this:
> 1) In about:config, set "security.sandbox.logging.enabled" to true and then
> quit the browser.
> 2) Before restarting, open the Mac Console application (from
> /Applications/Utilities/Console) and enter plugin-container in the Search
> field. 
> 3) Click "Clear" in Console
> 4) Then launch Firefox again using the same profile
> 5) You should start to see some violation logging as soon as you launch
> Firefox.
> 
> Any sandbox violations that show up in Console around the time of the crash
> or that relate to loading a stylesheet could be related. It's also possible
> in Console to use Edit->Select All, Edit->Copy, and then paste in a text
> edit and provide that to help us debug.
> 
> --
> 
> And something else to test is to disable the sandbox. One of the changes in
> build 56 relates to sandboxing. To disable the sandbox, you use about:config
> and change "security.sandbox.content.level" to 0 and then restart the
> browser. If the problem is not reproducible with
> security.sandbox.content.level=0, that tells us it's probably sandboxing
> related. (You have to restart the browser for the change to take effect.)

You're a Genius!  Disabling sandboxing fixed it, pages are loading again.  I obviously don't want to run without the sandbox enabled, but hopefully this will provide you with an idea of what is going on.

I am uploading two separate text files with the console output you requested:

1. sandboxing enabled.txt 
2. sandboxing disabled.txt


This is encouraging.  Thanks.

Comment 27

[charlie.siegel]

Attachment: sandboxing enabled.txt

Comment 28

[charlie.siegel]

Attachment: sandboxing disabled.txt

Comment 29

[Haik Aftandilian [:haik]]

(In reply to charlie.siegel from comment #26)
> You're a Genius!  Disabling sandboxing fixed it, pages are loading again.  I
> obviously don't want to run without the sandbox enabled, but hopefully this
> will provide you with an idea of what is going on.
> 
> I am uploading two separate text files with the console output you requested:
> 
> 1. sandboxing enabled.txt 
> 2. sandboxing disabled.txt
> 
> This is encouraging.  Thanks.

Thanks. Glad we could make some progress!

Setting security.sandbox.content.level=1 will turn the sandbox on, but the restrictions won't be as strong as what's in 56. It's better than disabling it for now and I suspect that will also work around the problem.

I think we'll have to reproduce this one locally to figure out what's going on. I'm not seeing anything in the logs that explains the crash, but could try a few things. Thanks for helping with the debugging.

Comment 30

[charlie.siegel]

One other thing I wanted to add about this. We are on an Active Directory domain. I'm not sure if it's relevant, but it's the only other thing I can think of that is different about my setup at work vs home. Also, this happened on the latest 57 beta today without running a second instances of Firefox.

Comment 31

[Haik Aftandilian [:haik]]

(In reply to charlie.siegel from comment #30)
> One other thing I wanted to add about this. We are on an Active Directory
> domain. I'm not sure if it's relevant, but it's the only other thing I can
> think of that is different about my setup at work vs home. Also, this
> happened on the latest 57 beta today without running a second instances of
> Firefox.

OK, thanks. When you're logged in with Active Directory, what is the path to your home directory? Opening up /Applications/Utilities/Terminal and running "echo $HOME" (without the quotes) should print the path. I'm wondering if your home directory ends up in a non-standard location and that is causing a sandbox issue, but I don't have any explanation as to why that would happen.

Comment 32

[charlie.siegel]

(In reply to Haik Aftandilian [:haik] from comment #31)
> (In reply to charlie.siegel from comment #30)
> > One other thing I wanted to add about this. We are on an Active Directory
> > domain. I'm not sure if it's relevant, but it's the only other thing I can
> > think of that is different about my setup at work vs home. Also, this
> > happened on the latest 57 beta today without running a second instances of
> > Firefox.
> 
> OK, thanks. When you're logged in with Active Directory, what is the path to
> your home directory? Opening up /Applications/Utilities/Terminal and running
> "echo $HOME" (without the quotes) should print the path. I'm wondering if
> your home directory ends up in a non-standard location and that is causing a
> sandbox issue, but I don't have any explanation as to why that would happen.

I checked that and it's correct and as far as I know it's in the standard place:

/Users/<username>

One other data point, I didn't reboot my machine last night and when I came in this morning Firefox is working again.

Comment 33

[charlie.siegel]

Two more data points:

1. Just to be absolutely positive it isn't Sophos, yesterday I tried excluding 'plugin-container' in addition to 'Firefox' and 'FirefoxCP Web Content' from Sophos.  I hadn't directly tested excluding plugin-container previously, I just assumed it wasn't the issue because it wasn't running.

This didn't resolve the issue though.  I still ran into the same thing.

2. I tried running the latest Nightly.  It still occurs.


I think at this point I have to throw up my hands.  If there's anything else you need from me, please let me know.  Thanks.

Comment 34

[charlie.siegel]

Oh, I forgot to add one more thing.

When it was in a crashed state yesterday, I switched MacOS accounts to a different local account I have on this machine.  It still wouldn't work.

In my mind this proves two things:

1. AD is not related to this issue.  This was a local account.

2. It's not something with my user profile or most likely even wrong with my install of MacOS - I was wondering if I might need to rebuild my machine to fix this issue, but it would appear no.

Comment 35

[Haik Aftandilian [:haik]]

Adding needinfo to remind myself to do some more debugging on this one.

Comment 36

[charlie.siegel]

One thing I found on this is that when I run up against this issue, even changing security.sandbox.content.level to 2 allows it to work again.  It definitely appears to be something in the strongest level of sandboxing protection that is clashing with MacOS.

Comment 37

[charlie.siegel]

I'm not sure if this will help or not, but I was curious if any errors were thrown when launching Firefox from the CLI.  Below are the outputs with sandbox levels set to 2/3.  With it set to 2, it does launch and work.  Right now with it set to 3, the tab crashes.




With security.sandbox.content.level=2

$ /Applications/Firefox.app/Contents/MacOS/firefox -P --no-remote
$ 2017-10-26 15:00:40.910 plugin-container[65320:856279] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x9f2b, name = 'com.apple.tsm.portname'
See /usr/include/servers/bootstrap_defs.h for the error codes.





 
With security.sandbox.content.level=3

$ /Applications/Firefox.app/Contents/MacOS/firefox -P --no-remote

###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost

$ [Child 65345, Main Thread] ###!!! ABORT: LoadSheetSync failed with error 80520012 loading built-in stylesheet 'resource://gre-resources/counterstyles.css': file /builds/worker/workspace/build/src/layout/style/nsLayoutStylesheetCache.cpp, line 776
[Child 65345, Main Thread] ###!!! ABORT: LoadSheetSync failed with error 80520012 loading built-in stylesheet 'resource://gre-resources/counterstyles.css': file /builds/worker/workspace/build/src/layout/style/nsLayoutStylesheetCache.cpp, line 776

###!!! [Parent][MessageChannel] Error: (msgtype=0x2400FA,name=PContent::Msg_AsyncMessage) Channel error: cannot send/recv


###!!! [Parent][MessageChannel] Error: (msgtype=0x150080,name=PBrowser::Msg_LoadRemoteScript) Channel error: cannot send/recv


###!!! [Parent][MessageChannel] Error: (msgtype=0x150083,name=PBrowser::Msg_Destroy) Channel error: cannot send/recv

Comment 38

[Haik Aftandilian [:haik]]

The counterstyles.css error 

  [Child 65345, Main Thread] ###!!! ABORT: LoadSheetSync failed with error 80520012 loading built-in stylesheet
  'resource://gre-resources/counterstyles.css': file /builds/worker/workspace/build/src/layout/style/nsLayoutStylesheetCache.cpp, line 776

is consistent with the crash reports, but knowing the file is resource://gre-resources/counterstyles.css may help.

resources://gre-resources/counterstyles.css should refer to /Applications/Firefox.app/Contents/Resources/omni.jar - chrome/res/counterstyles.css and I think the path is derived from NS_GRE_DIR which should be "/Applications/Firefox.app/Contents/Resources". If NS_GRE_DIR is somehow wrong, that might explain why countstyles.css fails to load. I don't know how NS_GRE_DIR could end up being wrong.

--

Now that you have it in the crashing state. Could you try using Console.app again? First load up Console.app (/Applications/Utilities/Console) and put in plugin-container in the filter box and then hit Clear. Then run

  $ MOZ_SANDBOX_LOGGING=1 /Applications/Firefox.app/Contents/MacOS/firefox -P --no-remote

If we get lucky, you'll see a Sandbox violation for the path that Firefox is erroneously trying to read from. You'll see other sandbox violations such as "Sandbox: plugin-container(5389) deny(1) file-read-metadata /Applications", so anything like that would be useful. You'll see entries in Console for the crash, but look for deny file-read* violations.

Comment 39

[charlie.siegel]

Is NS_GRE_DIR something I should be able to echo on the terminal?  If so it doesn't look like it's set.

I do see in the correct place.


I ran Firefox from the terminal as you asked.  I'll attach the full output to the case, but I think this is what you're after:

Violation:       deny file-read-metadata /Applications 
Process:         plugin-container [73640]
Path:            /Applications/Firefox.app/Contents/MacOS/plugin-container.app/Contents/MacOS/plugin-container
Load Address:    0x104dab000
Identifier:      org.mozilla.plugincontainer
Version:         1.0 (???)
Code Type:       x86_64 (Native)
Parent Process:  firefox [73636]
Responsible:     /Applications/Utilities/Terminal.app/Contents/MacOS/Terminal [5960]
User ID:         617400580

Date/Time:       2017-10-26 17:37:00.775 EDT
OS Version:      Mac OS X 10.12.6 (16G29)
Report Version:  8


It looks like possibly the sandbox isn't allowing /Applications/Firefox.app/Contents/Resources to be read from?

Comment 40

[charlie.siegel]

Attachment: sandbox violoations.txt

Comment 41

[Haik Aftandilian [:haik]]

(In reply to charlie.siegel from comment #39)
> Is NS_GRE_DIR something I should be able to echo on the terminal?  If so it
> doesn't look like it's set.

No, it's an internal Firefox name that should refer to the Resources directory.

> I do see in the correct place.
> 
> I ran Firefox from the terminal as you asked.  I'll attach the full output
> to the case, but I think this is what you're after:
> 
> Violation:       deny file-read-metadata /Applications 
> Process:         plugin-container [73640]
> Path:           
> /Applications/Firefox.app/Contents/MacOS/plugin-container.app/Contents/MacOS/
> plugin-container
> Load Address:    0x104dab000
> Identifier:      org.mozilla.plugincontainer
> Version:         1.0 (???)
> Code Type:       x86_64 (Native)
> Parent Process:  firefox [73636]
> Responsible:    
> /Applications/Utilities/Terminal.app/Contents/MacOS/Terminal [5960]
> User ID:         617400580
> 
> Date/Time:       2017-10-26 17:37:00.775 EDT
> OS Version:      Mac OS X 10.12.6 (16G29)
> Report Version:  8

The "deny file-read-metadata /Applications" is expected and is probably unrelated to this problem because we see that when things are working. We've looked into those a bit in bug 1378968.

> It looks like possibly the sandbox isn't allowing
> /Applications/Firefox.app/Contents/Resources to be read from?

I can build a version of Firefox that prints details about the sandbox policy on the console if you wouldn't mind giving that a try. I'll post a link once it's ready.

Comment 42

[charlie.siegel]

(In reply to Haik Aftandilian [:haik] from comment #41)
> (In reply to charlie.siegel from comment #39)
> > Is NS_GRE_DIR something I should be able to echo on the terminal?  If so it
> > doesn't look like it's set.
> 
> No, it's an internal Firefox name that should refer to the Resources
> directory.
> 
> > I do see in the correct place.
> > 
> > I ran Firefox from the terminal as you asked.  I'll attach the full output
> > to the case, but I think this is what you're after:
> > 
> > Violation:       deny file-read-metadata /Applications 
> > Process:         plugin-container [73640]
> > Path:           
> > /Applications/Firefox.app/Contents/MacOS/plugin-container.app/Contents/MacOS/
> > plugin-container
> > Load Address:    0x104dab000
> > Identifier:      org.mozilla.plugincontainer
> > Version:         1.0 (???)
> > Code Type:       x86_64 (Native)
> > Parent Process:  firefox [73636]
> > Responsible:    
> > /Applications/Utilities/Terminal.app/Contents/MacOS/Terminal [5960]
> > User ID:         617400580
> > 
> > Date/Time:       2017-10-26 17:37:00.775 EDT
> > OS Version:      Mac OS X 10.12.6 (16G29)
> > Report Version:  8
> 
> The "deny file-read-metadata /Applications" is expected and is probably
> unrelated to this problem because we see that when things are working. We've
> looked into those a bit in bug 1378968.
> 
> > It looks like possibly the sandbox isn't allowing
> > /Applications/Firefox.app/Contents/Resources to be read from?
> 
> I can build a version of Firefox that prints details about the sandbox
> policy on the console if you wouldn't mind giving that a try. I'll post a
> link once it's ready.

Sure, I'll run that.  Just let me know.

Comment 43

[Haik Aftandilian [:haik]]

Sorry for the delay on this. Here's a build for Firefox56 that prints out the sandbox policy on the command line and a some other path information. (It's build 56, but is named Nightly in the dmg.) It will print a few hundred lines each time a content process starts up.

Non-debug build:
  https://queue.taskcluster.net/v1/task/CWgpgOtkTX2DXqkgQpnVig/runs/0/artifacts/public/build/target.dmg

Debug build:
  https://queue.taskcluster.net/v1/task/YiwEqFKORaiDMLo5HOWNNA/runs/0/artifacts/public/build/target.dmg

I'm hoping that will tell us something when you run it when you're in the failing state. I'd recommend trying the non-debug build first. Debug builds are really slow and print out loads of data that isn't going to be relevant. It would be good to try in the failing state too though.

You can get to these dmg links from https://treeherder.mozilla.org/#/jobs?repo=try&revision=3676866987f2ca7c0f904578a3b65e1c5d887f97&selectedJob=141173637 where you can inspect the source code changes if you like.

Comment 44

[charlie.siegel]

No problem.

You're in luck, I was already in a crashed state today.  I ran both the non-debug and debug builds.  In both cases I instantly get the gah crashed tab message.

I'll attach the output from each.

Comment 45

[charlie.siegel]

Attachment: Test Build Output.txt

Comment 46

[charlie.siegel]

Attachment: Test Build Output - Debug.txt

Comment 47

[Haik Aftandilian [:haik]]

The sandbox profile looks right to me, but I'll look some more in more detail. From the debug build output, this appears to be the first set of errors related to the problem. I don't know what's going on here, but I'll keep debugging.

[Child 51979] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80520012: file /home/worker/workspace/build/src/js/xpconnect/loader/mozJSComponentLoader.cpp, line 667
[Child 51979] WARNING: NS_ENSURE_SUCCESS_VOID(rv) failed with result 0x80520012: file /home/worker/workspace/build/src/dom/base/nsFrameMessageManager.cpp, line 1625
[Child 51979] WARNING: NS_ENSURE_SUCCESS_VOID(rv) failed with result 0x80520012: file /home/worker/workspace/build/src/dom/base/nsFrameMessageManager.cpp, line 1625
[Child 51979] WARNING: NS_ENSURE_SUCCESS_VOID(rv) failed with result 0x80520012: file /home/worker/workspace/build/src/dom/base/nsFrameMessageManager.cpp, line 1625
[Child 51979] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80520012: file /home/worker/workspace/build/src/js/xpconnect/loader/mozJSComponentLoader.cpp, line 667
JavaScript error: chrome://global/content/browser-child.js, line 13: NS_ERROR_FILE_NOT_FOUND: Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIXPCComponents_Utils.import]

Comment 48

[Haik Aftandilian [:haik]]

One other thing to check: in the Sophos "history" for the affected machine, do you see any events that could be related to Firefox? I'm testing with the free version of Sophos and on the "Sohpos Home" page there is a history tab that lists device events.

Comment 49

[charlie.siegel]

I just checked into this.  Wishing Sophos Endpoint I enabled "Write threat and error events to system log."  These messages go to the MacOS Console.  Firefox just went into a crashed state, but nothing was logged to this.

Comment 50

[charlie.siegel]

So(In reply to charlie.siegel from comment #49)
> I just checked into this.  Wishing Sophos Endpoint I enabled "Write threat
> and error events to system log."  These messages go to the MacOS Console. 
> Firefox just went into a crashed state, but nothing was logged to this.


Sorry, I meant "Within Sophos..."

Comment 51

[charlie.siegel]

No activity in this for a while.  A couple of questions / thoughts on this:

1. Are there any more correlating crash / bug reports on this issue since 57 went stable that might help?

2. Have you been able to find anything else on your end?

3. I had updated to MacOS High Sierra and then had to downgrade to Sierra because of a show stopping OS bug.  I did a clean install.  I actually actively avoided installing Firefox for 2-3 weeks because of this issue, but due to Apple's recent software issues I'm becoming less comfortable with using Safari as my only browser, so I gave Firefox a try again.

I only installed few extensions (no ad blockers) and it ran fine for the rest of the day. 

I left my computer on over night (closing Firefox).  The next day I went to start Firefox and - Gah, your tab crashed...


So, is there any hope on this issue?  My guess is it is likely Sophos.  Is there anyway to coordinate something with Sophos to further troubleshoot this or at least see if they're seeing reports of this on their end?

Thanks

Comment 52

[Haik Aftandilian [:haik]]

(In reply to charlie.siegel from comment #51)
> No activity in this for a while.  A couple of questions / thoughts on this:
> 
> 1. Are there any more correlating crash / bug reports on this issue since 57
> went stable that might help?

There are other crashes that look similar. It might be unrelated, but I see some user-submitted crash report comments mentioning network-mounted volumes. You mentioned before you had your home directory on an Active Directory share. Is it still true that you've never hit the crash without Active Directory?

> 2. Have you been able to find anything else on your end?

I haven't made any more progress on this. I've been doing some testing with Sophos and multiple versions of Firefox running, but I haven't reproduced the crash so far. I have a smbfs share mounted, but I haven't tested with Active Directory.

> 3. I had updated to MacOS High Sierra and then had to downgrade to Sierra
> because of a show stopping OS bug.  I did a clean install.  I actually
> actively avoided installing Firefox for 2-3 weeks because of this issue, but
> due to Apple's recent software issues I'm becoming less comfortable with
> using Safari as my only browser, so I gave Firefox a try again.
> 
> I only installed few extensions (no ad blockers) and it ran fine for the
> rest of the day. 
> 
> I left my computer on over night (closing Firefox).  The next day I went to
> start Firefox and - Gah, your tab crashed...

When you did the clean install, did you reinstall Sophos before using Firefox? Is that machine also using Active Directory?

> So, is there any hope on this issue?  My guess is it is likely Sophos.  Is
> there anyway to coordinate something with Sophos to further troubleshoot
> this or at least see if they're seeing reports of this on their end?

Yes, we can. If possible, could you file a bug or service record with Sophos? I will try to contact them too.

Comment 53

[charlie.siegel]

(In reply to Haik Aftandilian [:haik] from comment #52)
> (In reply to charlie.siegel from comment #51)
> > No activity in this for a while.  A couple of questions / thoughts on this:
> > 
> > 1. Are there any more correlating crash / bug reports on this issue since 57
> > went stable that might help?
> 
> There are other crashes that look similar. It might be unrelated, but I see
> some user-submitted crash report comments mentioning network-mounted
> volumes. You mentioned before you had your home directory on an Active
> Directory share. Is it still true that you've never hit the crash without
> Active Directory?

Yes, that's correct.

> 
> > 2. Have you been able to find anything else on your end?
> 
> I haven't made any more progress on this. I've been doing some testing with
> Sophos and multiple versions of Firefox running, but I haven't reproduced
> the crash so far. I have a smbfs share mounted, but I haven't tested with
> Active Directory.
> 
> > 3. I had updated to MacOS High Sierra and then had to downgrade to Sierra
> > because of a show stopping OS bug.  I did a clean install.  I actually
> > actively avoided installing Firefox for 2-3 weeks because of this issue, but
> > due to Apple's recent software issues I'm becoming less comfortable with
> > using Safari as my only browser, so I gave Firefox a try again.
> > 
> > I only installed few extensions (no ad blockers) and it ran fine for the
> > rest of the day. 
> > 
> > I left my computer on over night (closing Firefox).  The next day I went to
> > start Firefox and - Gah, your tab crashed...
> 
> When you did the clean install, did you reinstall Sophos before using
> Firefox? Is that machine also using Active Directory?

Yes, I installed Sophos prior to Firefox.  The machine is also on AD.  Both of these are requirements for my work machine.

> 
> > So, is there any hope on this issue?  My guess is it is likely Sophos.  Is
> > there anyway to coordinate something with Sophos to further troubleshoot
> > this or at least see if they're seeing reports of this on their end?
> 
> Yes, we can. If possible, could you file a bug or service record with
> Sophos? I will try to contact them too.


I will look into this this week.


---

I may have another lead here.  I ran into these two bugs:

https://bugzilla.mozilla.org/show_bug.cgi?id=1404042
https://bugzilla.mozilla.org/show_bug.cgi?id=1422090

These are dealing with high CPU / GPU usage.  Both of which I've experienced frequently with Firefox.  I kind of just thought that was par for the course, but it appears it's not supposed to be as bad as it is.

The one thing that really perked up my ears is that it appears to be happening more often when you have a mac with scaled retina resolutions, which is something I do have on my work machine:

- I scaled the resolution up to the highest setting on the internal montior
- I have an external 4K display, set scaled to the equivalent to 2560x1440 - which is up I think a few notches from default

Another reason this makes sense to me, the colleague who I'd previously mentioned has experienced this issue, but it's much less frequent than me.  But I'm pretty sure he doesn't scale his internal display up as far as I do, and his external display is 1080P, not 4K - so there is a lot less stress on his video card.

What are your thoughts?  I've posed the question on the second bug if anyone has also experienced tab crashes.

Thanks

Comment 54

[charlie.siegel]

(In reply to Haik Aftandilian [:haik] from comment #52)
> (In reply to charlie.siegel from comment #51)
> > No activity in this for a while.  A couple of questions / thoughts on this:
> > 
> > 1. Are there any more correlating crash / bug reports on this issue since 57
> > went stable that might help?
> 
> There are other crashes that look similar. It might be unrelated, but I see
> some user-submitted crash report comments mentioning network-mounted
> volumes. You mentioned before you had your home directory on an Active
> Directory share. Is it still true that you've never hit the crash without
> Active Directory?
> 

I FIGURED IT OUT!!! :-) 

It is related to network drive mounting.  If I have more than one network drive mounted it happens.  I can repeat this on demand.  Mount 2nd drive, tabs start crashing, unmount second drive, tabs work again.  I don't even have to quit firefox.

This explains why a logout fixed the problem.

I have a script that mounts 3 drives, which is why I was seeing this very frequently.  For now I've commented out all but the most important drive, so I'm good for the time being.

Are there any ideas as to why mounted drives are interacting with Firefox?  Are there any other related bugs already open?

Thanks

Comment 55

[Haik Aftandilian [:haik]]

(In reply to charlie.siegel from comment #54)
> (In reply to Haik Aftandilian [:haik] from comment #52)
> > (In reply to charlie.siegel from comment #51)
> > > No activity in this for a while.  A couple of questions / thoughts on this:
> > > 
> > > 1. Are there any more correlating crash / bug reports on this issue since 57
> > > went stable that might help?
> > 
> > There are other crashes that look similar. It might be unrelated, but I see
> > some user-submitted crash report comments mentioning network-mounted
> > volumes. You mentioned before you had your home directory on an Active
> > Directory share. Is it still true that you've never hit the crash without
> > Active Directory?
> > 
> 
> I FIGURED IT OUT!!! :-) 

Great work! Thanks!

> It is related to network drive mounting.  If I have more than one network
> drive mounted it happens.  I can repeat this on demand.  Mount 2nd drive,
> tabs start crashing, unmount second drive, tabs work again.  I don't even
> have to quit firefox.
> 
> This explains why a logout fixed the problem.
> 
> I have a script that mounts 3 drives, which is why I was seeing this very
> frequently.  For now I've commented out all but the most important drive, so
> I'm good for the time being.

Could you provide output from the mount command (/sbin/mount run from the Terminal app) before and after you mount the 2nd drive which triggers the crashes? If I can reproduce the problem, that will help immensely with debugging. I'm curious what type of network drive is being mounted (smbfs, ftp, etc.) and where on your filesystem it's being mounted. Any data that would help me reproduce your configuration would help. Feel free to email me directly if you prefer not to post that data on the bug.

> Are there any ideas as to why mounted drives are interacting with Firefox?

security.sandbox.content.level=3 blocks access to /Volumes from web content processes and I suspect that is somehow related.

> Are there any other related bugs already open?

Nothing I've been able to find.

Comment 56

[charlie.siegel]

(In reply to Haik Aftandilian [:haik] from comment #55)
> (In reply to charlie.siegel from comment #54)
> > (In reply to Haik Aftandilian [:haik] from comment #52)
> > > (In reply to charlie.siegel from comment #51)
> > > > No activity in this for a while.  A couple of questions / thoughts on this:
> > > > 
> > > > 1. Are there any more correlating crash / bug reports on this issue since 57
> > > > went stable that might help?
> > > 
> > > There are other crashes that look similar. It might be unrelated, but I see
> > > some user-submitted crash report comments mentioning network-mounted
> > > volumes. You mentioned before you had your home directory on an Active
> > > Directory share. Is it still true that you've never hit the crash without
> > > Active Directory?
> > > 
> > 
> > I FIGURED IT OUT!!! :-) 
> 
> Great work! Thanks!
> 
> > It is related to network drive mounting.  If I have more than one network
> > drive mounted it happens.  I can repeat this on demand.  Mount 2nd drive,
> > tabs start crashing, unmount second drive, tabs work again.  I don't even
> > have to quit firefox.
> > 
> > This explains why a logout fixed the problem.
> > 
> > I have a script that mounts 3 drives, which is why I was seeing this very
> > frequently.  For now I've commented out all but the most important drive, so
> > I'm good for the time being.
> 
> Could you provide output from the mount command (/sbin/mount run from the
> Terminal app) before and after you mount the 2nd drive which triggers the
> crashes? If I can reproduce the problem, that will help immensely with
> debugging. I'm curious what type of network drive is being mounted (smbfs,
> ftp, etc.) and where on your filesystem it's being mounted. Any data that
> would help me reproduce your configuration would help. Feel free to email me
> directly if you prefer not to post that data on the bug.
> 
> > Are there any ideas as to why mounted drives are interacting with Firefox?
> 
> security.sandbox.content.level=3 blocks access to /Volumes from web content
> processes and I suspect that is somehow related.
> 
> > Are there any other related bugs already open?
> 
> Nothing I've been able to find.

These are SMB shares from a Windows Server.  They get mounted in /Volumes/<sharename> by MacOS.

I typically either mount these shares through the GUI or via my AppleScript.

When I'm mounting in the GUI I click on the server in Finder and then open the folder share I want to mount - this automatically mounts the share and then makes it available as a mounted network drive in the GUI.

My AppleScript uses a command like this for each share:

mount volume "smb://<servername>/<sharename>"


I tried manually mounting these shares with /sbin/mount.  I was able to mount them after I manually made the correct folder in /Volumes, but:

- The problem doesn't get reproduced when I do this.  I think the reason is that it doesn't get presented to the GUI - there's no visual indicator in the OS that the drives are mounted this way, they're only available at that path.  I'm assuming this means it's not triggering whatever condition Firefox is choking on with this.  My assumption is that it's keeping track of mounted drives so that they're available for sending data to and from them when needed.
- There was no output of really any kind in the terminal.  Definitely not any error messages.


I looked in the console to see if there were any messages when the drives are mounted, but I don't see any.

I *think* though, if you just try to mount two different SMB shares (which I would assume could source from anything that can share over SMB) you should be able to recreate this.

Comment 57

[charlie.siegel]

Two things that may further help with reproducing this:

1. Sometimes it fails on 2 shares, sometimes 3, so you may need 3.

2. It typically doesn't start crashing until after you quit Firefox and then reopen it.

Comment 58

[Haik Aftandilian [:haik]]

Thanks for the info. Update on this: I've been able to reproduce the problem when running Firefox out of my home directory and with a shared directory mounted from Windows 10. I don't have a full root cause yet, but am fairly certain problem is related to some code we have in net_GetFileFromURLSpec() from nsURLHelperOSX.cpp which tries to deal with old-style HFS paths that start with the volume name. In my case, the Windows share gets mounted on /Volumes/Users. Then, during browser startup, reading some support files from /Users/haik/... breaks because net_GetFileFromURLSpec() prepends "/Volumes" resulting in an incorrect path /Volumes/Users/haik/. I'll update once I have more information.

Comment 59

[Haik Aftandilian [:haik]]

The problem is due to read-access sandboxing breaking our Mac implementation of net_GetFileFromURLSpec() in nsURLHelperOSX.cpp. In net_GetFileFromURLSpec(), we attempt to handle POSIX paths as well as HFS paths of the form file:///volume-name/path-name. i.e., where instead of /Volumes/MyMountPoint/foo, we have a path like file:///MyMountPoint/foo.

net_GetFileFromURLSpec() disambiguates between these path formats by testing if the directory exists at the top level. For example, if we have a mounted volume named MyMountPoint, if /MyMountPoint is an actual directory, we'll use /MyMountPoint/foo as the POSIX path, but if the directory doesn't exist, we convert the path to /Volumes/MyMountPoint/foo. This breaks with level=3 content sandboxing because the check for /MyMountPoint always fails because the content process isn't allowed to access those paths. So, if we have a mount such as /Volumes/Applications, resolving a file URL such as /Applications/Firefox.app will fail because the actual path used will be /Volumes/Applications/Firefox.app.

We use FSGetVolumeInfo() to get the list of volumes and, in my tests, manually mounted volumes weren't included in the list. That explains why the problem doesn't occur for manually mounted volumes. The Apple documentation for FSGetVolumeInfo() doesn't include any specifics -- it's deprecated.

I was unsure if we still need to support these older HFS volume paths. Checked with :spohl and he's seen these in the URL bar. I suspect other applications might still be using them and could result in Firefox needing to open such a file URL.

The fix I'm testing changes the Mac sandbox rules to include read-metadata access to the top-level directories:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3825f52d0a13e6687b41bf0653e90973b75e3c81

Comment 60

[Haik Aftandilian [:haik]]

Charlie, would you be able to test this build and confirm if it solves the problem for you? Specifically I'd like to get it tested when run from /Applications (assuming that's where you install Firefox) because the paths used can affect whether or not the crash happens.

  https://queue.taskcluster.net/v1/task/bwUdRXagSc-PB6vvjtLhMg/runs/0/artifacts/public/build/target.dmg

Thanks!

Comment 61

[charlie.siegel]

(In reply to Haik Aftandilian [:haik] from comment #60)
> Charlie, would you be able to test this build and confirm if it solves the
> problem for you? Specifically I'd like to get it tested when run from
> /Applications (assuming that's where you install Firefox) because the paths
> used can affect whether or not the crash happens.
> 
>  
> https://queue.taskcluster.net/v1/task/bwUdRXagSc-PB6vvjtLhMg/runs/0/
> artifacts/public/build/target.dmg
> 
> Thanks!

This fixes the problem.  I confirmed this by mounting 3 volumes.  The new build continues to load pages, but FireFox 57's tab's crash.

Thank you for your work on this.

Just curious, will the fix for this be included in 58?

Comment 62

[Haik Aftandilian [:haik]]

Attachment: Bug 1404298 - Crashes with read-access content sandboxing triggered by mounted volumes.

Allow read-metadata access to top-level directory entries.

Review commit: https://reviewboard.mozilla.org/r/208454/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/208454/

Comment 63

[Haik Aftandilian [:haik]]

(In reply to charlie.siegel from comment #61)
> This fixes the problem.  I confirmed this by mounting 3 volumes.  The new
> build continues to load pages, but FireFox 57's tab's crash.
> 
> Thank you for your work on this.
> 
> Just curious, will the fix for this be included in 58?

And thank you for all the help and testing. I appreciate it. We should be able to get this 58. The current fix is low risk so that shouldn't be a problem at all. 58 is scheduled to be released on 2018-01-23, but Beta builds with the fix would be available sooner than that. I'll have to talk to release engineering about whether or not this would be acceptable for a future 57 update. But first this has to go through code review and integrate into Nightly.

Comment 64

[Alex Gaynor [:Alex_Gaynor]]

Comment on attachment 8937853 [details]
Bug 1404298 - Crashes with read-access content sandboxing triggered by mounted volumes.

https://reviewboard.mozilla.org/r/208454/#review214440

::: security/sandbox/test/browser_content_sandbox_fs.js:690
(Diff revision 1)
> +    // ensure the file/dir exists before we ask a content process to stat
> +    // it so we know a failure is not due to a nonexistent file/dir
> +    if (test.func === statPath) {
> +      ok(test.file.exists(), `${test.file.path} exists`);
> +    }

I think `exists` is just implemented as a `stat` and checking for an error. Does this get us anything?

Comment 65

[Haik Aftandilian [:haik]]

Comment on attachment 8937853 [details]
Bug 1404298 - Crashes with read-access content sandboxing triggered by mounted volumes.

https://reviewboard.mozilla.org/r/208454/#review214450

::: security/sandbox/test/browser_content_sandbox_fs.js:690
(Diff revision 1)
> +    // ensure the file/dir exists before we ask a content process to stat
> +    // it so we know a failure is not due to a nonexistent file/dir
> +    if (test.func === statPath) {
> +      ok(test.file.exists(), `${test.file.path} exists`);
> +    }

It's for the tests that validate content doesn't have access. My thinking was that if we are trying to test that content sandboxing is blocking a stat() call, we want to know content failed because it doesn't have access, not because the file we're testing got removed. For example, if Apple moved /private/etc.

Comment 66

[charlie.siegel]

(In reply to Haik Aftandilian [:haik] from comment #63)
> (In reply to charlie.siegel from comment #61)
> > This fixes the problem.  I confirmed this by mounting 3 volumes.  The new
> > build continues to load pages, but FireFox 57's tab's crash.
> > 
> > Thank you for your work on this.
> > 
> > Just curious, will the fix for this be included in 58?
> 
> And thank you for all the help and testing. I appreciate it. We should be
> able to get this 58. The current fix is low risk so that shouldn't be a
> problem at all. 58 is scheduled to be released on 2018-01-23, but Beta
> builds with the fix would be available sooner than that. I'll have to talk
> to release engineering about whether or not this would be acceptable for a
> future 57 update. But first this has to go through code review and integrate
> into Nightly.

Sounds great, thank you.

Comment 67

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bda5959fad21
Crashes with read-access content sandboxing triggered by mounted volumes. r=Alex_Gaynor

Comment 68

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/bda5959fad21

Comment 69

[Julien Cristau [:jcristau]]

Please request beta uplift when you're comfortable doing so.

Comment 70

[Haik Aftandilian [:haik]]

Comment on attachment 8937853 [details]
Bug 1404298 - Crashes with read-access content sandboxing triggered by mounted volumes.

Approval Request Comment

[Feature/Bug causing the regression]:
Bug 1332190 - [Mac] Enable level 3 Mac content sandbox, removing filesystem read access

[User impact if declined]:
The browser is unusable due to repeated crashes on startup. More specifically, a Mac user with mounted volumes may suddenly start to see repeated browser crashes every time the browser is launched. Whether or not the crashes occur depends on the name of the mount in /Volumes and whether or not they were mounted manually using the mount(8) command.

[Is this code covered by automated tests?]:
The changed sandboxing code is executed by automated tests since it is run every time a content process is started. The patch includes additional automated tests. We don't have tests for the /Volumes interaction.

[Has the fix been verified in Nightly?]:
Yes

[Needs manual test from QE? If yes, steps to reproduce]: 
No

[List of other uplifts needed for the feature/fix]:
None

[Is the change risky?]:
No

[Why is the change risky/not risky?]:
The change only adds filesystem read-metadata file access to the Mac sandbox rules for top-level filesystem entries. Adding read-metadata permissions is low risk because it's unlikely that we have code that depends on not having such access.

[String changes made/needed]:
None

Comment 71

[Gerry Chang [:gchang]]

Comment on attachment 8937853 [details]
Bug 1404298 - Crashes with read-access content sandboxing triggered by mounted volumes.

Fix a crash. Beta58+.

Comment 72

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/releases/mozilla-beta/rev/d71f0c5729d4