Closed Bug 1197007 Opened 4 years ago Closed 4 years ago

[EME] Sandbox prevents eme-adobe.dll loading on Win10 when Firefox profile on FAT-formatted SD card

Categories

(Core :: Audio/Video: Playback, defect, P1)

Unspecified
Windows 8.1
defect

Tracking

()

RESOLVED FIXED
mozilla43
Tracking Status
firefox41 --- affected
firefox42 --- affected
firefox43 --- fixed

People

(Reporter: cpearce, Assigned: bobowen)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

I keep getting GMP process crashes on startup when the sandbox is enabled on Win10 when my Firefox profile is on an SD card. I don't see the crashes when my Firefox profile is on the main disk and the sandbox is enabled.

It looks like there's something preventing the LoadLibrary call, as my crash reports don't have an eme-adobe.dll loaded, and from the NSPR GMP log I can see that the child process gets to mozilla::gmp::GMPChild::RecvStartPlugin before crashing.

https://crash-stats.mozilla.com/report/index/a77272f1-be44-402d-94f6-59ce82150820

We'll need this fixed and uplifted to Beta41 pretty promptly.

STR:
* Get Windows 10.
* Install Nightly.
* Create a Firefox profile on an SD card
* Run Firefox using profile from SD card
* Open Netflix.com, try to play a video.
* Note "plugin crashed" notification box drops down from browser chrome
Bob: Can you take a look at this? If you're going on PTO, can you find someone else to look?
Flags: needinfo?(bobowen.code)
Sticking the Firefox profile on a USB stick also doesn't crash. I also tried on my Win8.1 desktop loading the firefox profile on my secondary (non-System) hard drive, and the problem didn't reproduce.
The SD card is FAT format. I tried partitioning my hard drive to create a FAT and FAT32 partition, and the bug didn't repro when my firefox profile was located on those partitions. So the bug isn't caused by the partition type.
A profile on an SD card (micro in adapter) works for me on Windows 7 laptop.

I did a test of sorts on my laptop with Windows 10 (running in VirtualBox in linux host).

The only problem is that VirutalBox doesn't do SD cards so you have to use VBoxManage commands to create a vmdk from the raw disk.
So, I suppose this again shows that there isn't a problem with the underlying file system.

Unfortunately, I don't think I'm going to have time to set-up a physical install of Windows 10 on my laptop, before I leave.

Tim - would you have time to look into this?
If not maybe aklotz or bbondy could help.
Flags: needinfo?(bobowen.code) → needinfo?(tabraldes)
Upgrade didn't take too long and I can reproduce.

The sandbox seems to be allowing the file access OK, but in debug it looked like I was getting some hangs, possibly because the sandbox makes a sync IPC call of its own to open the file.

If I format the SD card as NTFS it works fine, although it crashed the very first time I tried it.

From copying the profile back to the SD card, FAT seems to be really slow, much slower that NTFS, so maybe it is some sort of timing issue.
I've just tried a really old SD Card (which I think was originally used for a PS2), that worked OK when formatted as NTFS.

So it looks like this is an SD Card and FAT specific problem.

Doubt I'll be able to look into this further now.
Netflix reports that they have seen (seven instances of) a new error F7392 (bug 1197909) in their recent beta test. The error only seems to affect Windows 10. Could these bugs be related?
OS: Unspecified → Windows 8.1
Summary: [EME] Sandbox prevents eme-adobe.dll loading on Win10 when Firefox profile on SD card → [EME] Sandbox prevents eme-adobe.dll loading on Win10 when Firefox profile on FAT-formatted SD card
Sorry for the delayed reply! I was out of town last Tuesday through yesterday.

> Tim - would you have time to look into this?
> If not maybe aklotz or bbondy could help.

I don't have a Windows 10 machine and I've been out-of-touch with this codebase for a while now; setting needinfo for the other people mentioned.

I could maybe install some dev tools on my wife's machine if no one else is able to take a look at this, but no guarantees.
Flags: needinfo?(tabraldes)
Flags: needinfo?(netzen)
Flags: needinfo?(aklotz)
Sorry I'm in startup mode at the moment so have been really busy, won't have time to look anytime soon. Hopefully aklotz.
Flags: needinfo?(netzen)
If I drop the sandbox to USER_LIMITED it works.
So, I think this is failing on Windows 10 (and not others) for the same reason USER_LOCKDOWN was failing, i.e. the creation of the activation context before lowering the sandbox isn't working.
I'm not sure but I think this might be to do with code optimisation.

Chris, is there a way that I can test changes to plugin-container.exe, without checking things in, given the voucher?
Flags: needinfo?(aklotz) → needinfo?(cpearce)
(In reply to Bob Owen (:bobowen) from comment #10)
> Chris, is there a way that I can test changes to plugin-container.exe,
> without checking things in, given the voucher?

No.

You could try asking relman to spin and sign a build for you. What I did when I had to test something in plugin-container is landed a behaviour change that only took effect when an environment variable was set.
Flags: needinfo?(cpearce)
(In reply to Chris Pearce (:cpearce) from comment #11)
> (In reply to Bob Owen (:bobowen) from comment #10)
> > Chris, is there a way that I can test changes to plugin-container.exe,
> > without checking things in, given the voucher?
> 
> No.
> 
> You could try asking relman to spin and sign a build for you. What I did
> when I had to test something in plugin-container is landed a behaviour
> change that only took effect when an environment variable was set.

Hmm, my first plan is to turn off optimisation for the Load function, so I can't do that with an env var.

Given some of the memory clearing that is going on in that function, I wonder if turning off optimisation might not be a bad idea anyway.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e094bf1b60c1
Bug 1197007: Turn off optimization for GMPLoaderImpl::Load. r?cpearce
Attachment #8658315 - Flags: review?(cpearce)
This will see if my thought that optimisation is causing a problem here.

There may be a better way to prevent the optimisation though.
Assignee: nobody → bobowen.code
Comment on attachment 8658315 [details]
MozReview Request: Bug 1197007: Turn off optimization for GMPLoaderImpl::Load. r?cpearce

https://reviewboard.mozilla.org/r/18575/#review16595
Attachment #8658315 - Flags: review?(cpearce) → review+
Thanks Chris.

Try push in comment 12, thanks.
Keywords: checkin-needed
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/b5add456ce27
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla43
Damn, that didn't work.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
I've managed to reproduce this for clearkey by adding the following line to its moz.build:
LDFLAGS += ['-MANIFEST:EMBED,ID=2']

This just adds a default manifest into the DLL, which is what the Adobe DLL has.

Haven't found a fix yet, but at least I can test things and run a debug build.

Chris - As far as I can see the Adobe manifest is just the default one and doesn't specify any DLL versions, so I wonder if they actually need it or whether it is just the default in Visual Studio.
Flags: needinfo?(cpearce)
This fix should be coming in our next drop.
Flags: needinfo?(cpearce)
This is fixed for me with CDM v14, which doesn't have the side-by-side manifest any more.

Also, I tried with USER_LOCKDOWN and that seems to work as well (as expected).
Flags: needinfo?(cpearce)
Thanks Bob. If you want to tighten the sandboxing, go ahead.
Status: REOPENED → RESOLVED
Closed: 4 years ago4 years ago
Flags: needinfo?(cpearce)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.