Closed Bug 1391186 Opened 7 years ago Closed 7 years ago

Thunderbird loses setting as default email client when "mailto" triggered by Firefox 56/57

Categories

(Firefox :: File Handling, defect, P1)

56 Branch
x86
macOS
defect

Tracking

()

VERIFIED FIXED
Firefox 57
Tracking Status
firefox-esr52 --- unaffected
firefox55 --- unaffected
firefox56 + verified
firefox57 + fixed

People

(Reporter: alberts, Assigned: haik)

References

()

Details

(Keywords: regression)

Attachments

(6 files)

There seems to be some weirdness going on between Thunderbird and Firefox on macOS. When triggering a mailto link from Firefox 56/57 [1] Thunderbird before set as default email client looses these privilege. This does not happen when using other browsers (or FF 55, see below).


TB version TB 56.0b2, build ID 20170809184428
macOS version 10.11.6


1) Make sure TB is set as default email client [2]
2) open test pen [3] with Opera (or Chrome, or Safari)
3) Make sure TB composer window opens on click
4) open test pen [3] with Firefox 56 or 57 [1]


Currently for me the link does not do anything and when I switch to Thunderbird and check the "default mail client" setting all three checkboxes [1] are unticked.
In the attached screenshot you can see the settings modal with checkboxes unticked and a modal from Opera/Chrome asking to set a default email client, after I clicked the "mailto" with FF 57.


[1] tested Firefox versions:
    Good FF 54.0.1 build ID 20170628075643
    Good FF 55.0.2 build ID 20170814073321
    Bad  FF 56.0b3 build ID 20170815141045 (also checked with add-ons disabled)
    Bad  FF 57.0a1 build ID20170816100153 (also checked web and browser console on click => nothing)
[2] Go to "TB > Preferences > Advanced > General > Check Now..." and make sure
    all three checkboxes (E-Mail, Newsgroups, Feeds) are ticked
[3] https://codepen.io/scheinercc/pen/ayBMEX


Note: Not sure if this should be raised against TB or FF tbh!?
Let's try Firefox :-(
Component: OS Integration → Untriaged
Product: Thunderbird → Firefox
Component: Untriaged → File Handling
Brindusa, can someone from QA help find a narrower regression range?
Flags: needinfo?(brindusa.tot)
Started searching for regression range (just (finally) installed mozregression, first time used, it's great!). Clearing NI for Brindusa, hope that's ok.

Last good revision: ba8db0fbc00605d6d097dde0b7e034297f55c1ec
First bad revision: 6b101438c684bff925471edbfe593e500bcb3a03

Pushlog: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=ba8db0fbc00605d6d097dde0b7e034297f55c1ec&tochange=6b101438c684bff925471edbfe593e500bcb3a03

Need anything else?
Flags: needinfo?(brindusa.tot)
I don't think so, thanks!!

Well, looking at the change, it's one line:
https://hg.mozilla.org/integration/autoland/rev/6b101438c684#l1.12

So what happens if you set security.sandbox.content.level to 1 or 2?
Blocks: 1332190
Flags: needinfo?(haftandilian)
Flags: needinfo?(agaynor)
See Also: → 1391555
Albert, thanks for reporting this. Leaving the needinfo for myself to come back and debug this.

In the meantime, if you want to debug further, could you try the following steps to get more information?

1) Set the pref security.sandbox.logging.enabled=true and quit Firefox.
2) Start the Console app (/Applications/Utilities/Console).
3) In the search field, enter "plugin-container" no quotes.
4) Click the "Clear" icon.
5) Start Firefox and reproduce the problem.
6) In the Console app, you'll see several messages about SandboxViolations, but look for anything that may be related to mail or starting Thunderbird. In Console, you can do Edit->Select All, Edit->Copy, and then paste in a text editor to get a full list. If you attach that list or send it to me, I'll go through it.

When done, remember to revert security.sandbox.logging.enabled to false. (There's another bug that can cause performance issues if you leave that set to true: bug 1338651)

Thanks.
Flags: needinfo?(agaynor)
Attached file 3765769_ff-nightly.log
Got a couple of messages and log files. Will attach the first and last TB log and the one FF log. Hope they'll help.

19/08/2017 10:22:42.429 sandboxd[92249]: ([34351]) plugin-container(34351) deny file-read-metadata /Applications/Thunderbird.app
19/08/2017 10:22:42.432 sandboxd[92249]: ([34351]) plugin-container(34351) deny file-read-metadata /Applications/Thunderbird.app
19/08/2017 10:22:42.435 sandboxd[92249]: ([34351]) plugin-container(34351) deny file-read-metadata /Applications/Thunderbird.app
19/08/2017 10:22:42.438 sandboxd[92249]: ([34351]) plugin-container(34351) deny file-read-metadata /Applications/Thunderbird.app
19/08/2017 10:22:42.441 sandboxd[92249]: ([34351]) plugin-container(34351) deny file-read-metadata /Applications/Thunderbird.app
19/08/2017 10:22:42.000 kernel[0]: Sandbox: plugin-container(34351) deny(1) file-read-metadata /Applications/Thunderbird.app
19/08/2017 10:22:42.000 kernel[0]: Sandbox: plugin-container(34351) deny(1) file-read-metadata /Applications/Thunderbird.app
19/08/2017 10:22:42.000 kernel[0]: Sandbox: plugin-container(34351) deny(1) file-read-metadata /Applications/Thunderbird.app
19/08/2017 10:22:42.000 kernel[0]: Sandbox: plugin-container(34351) deny(1) file-read-metadata /Applications
19/08/2017 10:22:42.444 sandboxd[92249]: ([34351]) plugin-container(34351) deny file-read-metadata /Applications
19/08/2017 10:22:42.447 sandboxd[92249]: ([34351]) plugin-container(34351) deny file-read-metadata /Applications/Thunderbird.app
19/08/2017 10:22:42.450 sandboxd[92249]: ([34351]) plugin-container(34351) deny file-read-metadata /Applications/Thunderbird.app
Attached file 3765758_tb-beta.log
Attached file 3765777_tb-beta.log
Thanks! That's really helpful.

The stacks tell us that the Mac implmementation of nsExternalHelperAppService::GetApplicationDescription() needs to read from the helper Application's bundle, but reading from /Applications is blocked in the content process sandbox which prevents this from working and causes nsExternalHelperAppService::GetProtocolHandlerInfo to fail. nsExternalHelperAppService() has already remoted nsExternalHelperAppService::LoadURI() so that loading the URI is done by the parent. (On Mac, starting with build 52, the content process sandbox prevents content processes from directly executing applications.) But we'll have to remote GetApplicationDescription() and possibly other parts of nsExternalHelperAppService(). Needs more investigation.

Alternatively, we could allow reading from /Applications in the content process sandbox which should allow this to work for apps that reside under /Applications. Allowing read access to /Applications has a limited security impact because /Applications typically wouldn't contain user private data. If a user has configured the browser to use a helper app in their home directory, that wouldn't work.

I'm looking into how we could remote the GetApplicationDescription() call.
Flags: needinfo?(haftandilian)
Flags: needinfo?(haftandilian)
[Tracking Requested - why for this release]:
Regression in "mailto:" protocol handling caused by enabling the content sandbox in bug 1332190.
Priority: -- → P1
This problem is not reproducible for me on OS X 10.12 (Sierra) with Thunderbird 52.3.0 (64-bit) and Firefox Nightly 57.0a1 (2017-08-21) (64-bit). I do see the Sandbox violations messages, but mailto: links work as expected.

With OS X 10.11 (El Capitan), the problem is reproducible for me. On 10.11, if the mailto handler is set in the _Firefox_ preferences to be a specific client such as Thunderbird or Mail, mailto links work as expected.

However, on 10.11, if the handler is set to be "<Client> (default)", mailto links do not work. That's where Firefox uses the OS default and displays it in the list next "(default)".
Assignee: nobody → haftandilian
Track 56+/57+ as regression in "mailto:" protocol handling.
Update: still debugging, but I've noticed some strange behavior while testing this on my 10.11 system which makes me wonder if I'm hitting the same problem as Albert.

On 10.11, when I'm in a situation where I hit the problem every time, the parent process' call to LSOpenCFURLRef(myURLRef, NULL) from nsMIMEInfoMac::LoadUriInternal() fails with kLSApplicationNotFoundErr. I've reproduced that with the sandbox level disabled or set to 1, 2, and 3.

If I manually open the mail client first and then quit it, mailto links start working in Firefox.

I've reproduced the problem as far back as build 50.

Albert used mozregression to identify Bug 1332190 in comment 3 and I can't explain why that triggered the failure in his case.
Flags: needinfo?(haftandilian)
Albert, when you're hitting the problem, could you take a screenshot of your Firefox settings for the mailto: handler?

Specifically, in Firefox preferences, under Applications, what is Content Type mailto set to?

Thanks.
Flags: needinfo?(albert)
Before triggering the bug (clicking on a mailto link) it is set to:
"mailto" - "Use Thunderbird (default)"

when clickling a mailto link and refreshing Preferences it changes to:
"mailto" - "Use (default)"

... literally with this blank space. When opening the options dropdown this text is gone and only standard options missing a default are presented. See screenshots
Flags: needinfo?(albert)
Made some progress on this and have an explanation for what's happening.

It was already clear that calling LSGetApplicationForURL() in our sandboxed content process fails. In testing a simple app, I've found that the call fails if the process doesn't have read access to /Applications and access to the com.apple.lsd.mapdb service. (I've only tested with mail clients installed in /Applications). This seems reasonable because the OS may want to check the application exists before returning it as the handler.

But, in addition to failing, calling LSGetApplicationForURL() in our content process has the unexpected side effect of breaking the default email client association requiring it to be reset before LSGetApplicationForURL() works again from any app. This must be an 10.11 OS X bug. On OS X, one sets/views the default mail client association in the Mail.app preferences. To confirm this, I set a breakpoint on LSGetApplicationForURL() in the content process and checked that the email client was Thunderbird. At a breakpoint immediately after LSGetApplicationForURL(), the Thunderbird association as reported in Mail.app's preferences was lost.

After LSGetApplicationForURL() is called in content, future calls to LSGetApplicationForURL() fail and future calls to launch the client with LSOpenCFURLRef("mailto:...") (even in the parent process) also fail until the default email client is reset in the Mail.app preferences. This is why Thunderbird is failing to launch despite it being launched from the parent process--the call to LSOpenCFURLRef() fails because the default email handler setting in the OS is in some undefined state. This is what I encountered in comment 13.

We shouldn't need to call LSGetApplicationForURL() from content. We already launch the email client from the parent process. I'm looking into the best way to fix this.
Summary: Thunderbird looses setting as default email client when "mailto" triggered by Firefox 56/57 → Thunderbird loses setting as default email client when "mailto" triggered by Firefox 56/57 (OS X 10.11)
The LSGetApplicationForURL() failure/flakiness described in comment 16 is also probably responsible for the flakey behavior of the "mailto" setting in the Firefox settings, but I haven't confirmed that yet.
Can do Haik. What is the best/easiest way to test/install it on my system? I haven't dealt with try-builds before ...
Flags: needinfo?(haftandilian)
Thanks. You just need the dmg file (but I included the try link because it can be used to view the source code changes in case you wanted to see what you were installing.)

  https://queue.taskcluster.net/v1/task/EzgICI1FQWiaqFGOsy9LBQ/runs/0/artifacts/public/build/target.dmg

That target.dmg will look like a standard Firefox download for Mac, but it will include Nightly. Once you open the .dmg, you can drag Nightly to your /Applications folder or your ~/Desktop and run it there (like a typical Mac app.)
Flags: needinfo?(haftandilian)
Tested and verified! FF Nightly and TB like each other again with this fix in place ;)
Flags: needinfo?(albert)
Posting a minimal fix that avoids calling the Mac nsOSHelperAppService::GetApplicationDescription() from nsOSHelperAppService::GetProtocolHandlerInfoFromOS() if we are executing in the child process on Mac. The result of this is that in the child, the nsMIMEInfoImpl will not have a description of the default application. I haven't found any code that executes in the child and depends on having the description string for the default app. See comment 16 for a description of the problem.
Comment on attachment 8903344 [details]
Bug 1391186 - Thunderbird loses setting as default email client when "mailto" triggered by Firefox 56/57 (OS X 10.11).

https://reviewboard.mozilla.org/r/175142/#review180372

I cannot review this patch. I'm not familiar with uriloader on mac and the implication of this change.
Attachment #8903344 - Flags: review?(amarchesini)
Stephen, are you comfortable reviewing this Mac-specific change? Context in comment 16 and comment 22.
Flags: needinfo?(spohl.mozilla.bugs)
Comment on attachment 8903344 [details]
Bug 1391186 - Thunderbird loses setting as default email client when "mailto" triggered by Firefox 56/57 (OS X 10.11).

https://reviewboard.mozilla.org/r/175142/#review182010

This looks good to me overall. One comment:

(In reply to Haik Aftandilian [:haik] from comment #22)
> Posting a minimal fix that avoids calling the Mac
> nsOSHelperAppService::GetApplicationDescription() from
> nsOSHelperAppService::GetProtocolHandlerInfoFromOS() if we are executing in
> the child process on Mac. The result of this is that in the child, the
> nsMIMEInfoImpl will not have a description of the default application. I
> haven't found any code that executes in the child and depends on having the
> description string for the default app. See comment 16 for a description of
> the problem.

Since this appears to be fixed in macOS 10.12, is there value in keeping the existing behavior if we're on 10.12 or above? It would be great to confirm on 10.13 that the issue continues to be fixed. Or are you 100% confident that no code in the child will need the description string for the default app? By using nsCocoaFeatures::OnSierraOrLater()[1] we could limit possible regressions to 10.11 and below.

There appear to be some failures on try. Please make sure that these are known intermittents and not new failures introduced by this patch.

[1] https://dxr.mozilla.org/mozilla-central/rev/c959327c6b75cd4930a6ea087583c38b805e7524/widget/cocoa/nsCocoaFeatures.h#23
Attachment #8903344 - Flags: review+
Comment on attachment 8903344 [details]
Bug 1391186 - Thunderbird loses setting as default email client when "mailto" triggered by Firefox 56/57 (OS X 10.11).

https://reviewboard.mozilla.org/r/175142/#review182010

Thanks for the review.

I had assumed the problem was fixed, but it's reproducible on the latest 10.13 Beta (17A360a). I still can't reproduce this on 10.12.

I'll make another pass to make sure no code in the child is going to need the description string, and will make sure the test failures are unrelated before landing.
Summary: Thunderbird loses setting as default email client when "mailto" triggered by Firefox 56/57 (OS X 10.11) → Thunderbird loses setting as default email client when "mailto" triggered by Firefox 56/57
Flags: needinfo?(spohl.mozilla.bugs)
Pushed by haftandilian@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1a56faf6adf5
Thunderbird loses setting as default email client when "mailto" triggered by Firefox 56/57 (OS X 10.11). r=spohl
https://hg.mozilla.org/mozilla-central/rev/1a56faf6adf5
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 57
Comment on attachment 8903344 [details]
Bug 1391186 - Thunderbird loses setting as default email client when "mailto" triggered by Firefox 56/57 (OS X 10.11).

Approval Request Comment
[Feature/Bug causing the regression]:
Mac OS content sandboxing in build 56 exposes this problem.

[User impact if declined]:
Clicking mailto links or other external protocol links may fail to open the application (e.g., Thunderbird).

[Is this code covered by automated tests?]:
No

[Has the fix been verified in Nightly?]:
I manually verified the fix in Nightly on OS X 10.13 High Sierra.

[Needs manual test from QE? If yes, steps to reproduce]: 
[List of other uplifts needed for the feature/fix]:
None

[Is the change risky?]:
Medium

[Why is the change risky/not risky?]:
The change is small, but it is in an area that isn't covered well by automated tests.

[String changes made/needed]:
None
Attachment #8903344 - Flags: approval-mozilla-beta?
Blocks: highsierra
Comment on attachment 8903344 [details]
Bug 1391186 - Thunderbird loses setting as default email client when "mailto" triggered by Firefox 56/57 (OS X 10.11).

From discussion with haik it sounds like this affects any mac external application defaults. While this is last minute in beta, I think we need to not break how Firefox handles external app defaults. Let's uplift this for beta 12. 

I think this should have a bit of exploratory testing (on 56) and I'm asking haik to suggest some ways to test.
Flags: needinfo?(andrei.vaida)
Attachment #8903344 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Flags: qe-verify+
I managed to reproduce the bug using an older version of Nightly (2017-08-17) on Mac OS x10.11. If the handler is set to be 
"<Client> (default", Firefox only opens a black new tab and nothing else happens.

I retested using the latest Beta 56.0b12 on the same platform, but the bug is not reproducing anymore. I would like to mention that I used both Thunderbird and another email client to test this issue. They both work fine on latest Beta. And even after I restarted the browser, the settings remained as the ones that I selected. 

I tried to reproduce the bug with other external application defaults, like calendar, pdf, video and so on, but I couldn't. As far as I can tell only the email client was affected.
Updating flags based on Comment 34, thank you Oana!
Status: RESOLVED → VERIFIED
Flags: qe-verify+
Flags: needinfo?(andrei.vaida)
See Also: → 1400803
You need to log in before you can comment on or make changes to this bug.