1257375 - no video acceleration with wireless display on Windows 10

Status: RESOLVED FIXED

(Core :: Graphics, defect)

Description

[Paul Blinzer]

User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0
Build ID: 20160303134406

Steps to reproduce:

Repro steps:
Use a system with Wireless display capability (Miracast, etc) 
Connect a display with a physical connector (VGA, displayport, HDMI). 
play back video through Firefox e.g. from http://www.youtube.com
note that video playback is accelerated.

Connect with wireless display only on Windows10. 
play back video through Firefox e.g. from http://www.youtube.com
note that video playback is not accelerated. 




Actual results:

The video playback in the second scenario is non-accelerated. 

Investigation into this revealed that on Windows10 the Miracast driver reports the display adapter device as 0x0000 which apparently leads Firefox to not use the video acceleration features of the system on the actual graphics driver (which is still present and available). This is specific to Firefox; it may be caused by incorrectly dealing with the different available adapters (one virtual, from Miracast one physical from the actual graphics card) in the decode pipeline. 


Expected results:

The video playback should be accelerated, regardless if wireless or physically connected display is used. While it seems to be ultimately an OS issue, not propagating the underlying device info to Miracast, the browser should be more gracious in handling this case to allow acceleration to occur.

Comment 1

[Matt Woodrow (:mattwoodrow)]

Jeff, do we have any hardware in the office that can do Miracast? My TV doesn't support it.

Comment 2

[Jeff Muizelaar [:jrmuizel]]

(In reply to Paul Blinzer from comment #0)
> Steps to reproduce:
> 
> Repro steps:
> Use a system with Wireless display capability (Miracast, etc) 

Paul, what's the easiest way for us to get the hardware to reproduce this locally? Do you have a hardware recommendation?

Comment 3

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

It looks like http://www.airserver.com/PC can act as a display.

Comment 4

[Matt Woodrow (:mattwoodrow)]

(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #3)
> It looks like http://www.airserver.com/PC can act as a display.

But not on OSX :(

Comment 5

[Paul Blinzer]

Hardware needed: 
Miracast receiver with HDMI output to connect to any HDTV or Computer monitor.
No-name ones can be had for about US$ 20,- or less: http://www.ebay.com/itm/like/252166299800?lpid=82&chn=ps&ul_noapp=true, they typically work just fine, though some may have some trouble connecting with some wireless devices.
Brand name (Belkin, ASUS) Miracast wireless dongles are at about US$ 60,-, depending on the shop. 

Windows 10 PC with Wireless Display capability. Any recent Laptop (AMD A10-8xxx based, Intel Haswell or Skylake based) should be able to support it, if Windows 8.1 or Windows 10 is installed. 
How to find out: 
Press Windows Key + P, a selection entry should slide in from the right with "PC Screen only, Duplicate, Extend", if wireless is supported an additional text line should show up "Connect to Wireless Display" 
Connecting to the wireless display works similar as pairing with a Bluetooth device.

After connection occurred and the display is redirected to wireless, the issue should be observable.

Comment 6

[Paul Blinzer]

Jeff, any more information needed?

Comment 7

[Jeff Muizelaar [:jrmuizel]]

I've ordered a dongle and will update the bug with info when I can reproduce it.

Comment 8

[Paul Blinzer]

Jeff, any update on the issue, did you see the repro?

Comment 9

[Jeff Muizelaar [:jrmuizel]]

I can reproduce it. I have an idea of what the reason might be.

Comment 10

[Jeff Muizelaar [:jrmuizel]]

Attachment: Remove white list

This should make things work. I'm not sure what the consequences will all be. I will look at telemetry to find it.

Comment 11

[Milan Sreckovic [:milan] (needinfo for best results)]

Comment on attachment 8761827 [details] [diff] [review]
Remove white list

Review of attachment 8761827 [details] [diff] [review]:
-----------------------------------------------------------------

I'm pretty sure this will break the blocklisting tests.

Comment 12

[Paul Blinzer]

(In reply to Milan Sreckovic [:milan] (June 10 PTO) from comment #11)
> Comment on attachment 8761827 [details] [diff] [review]
> Remove white list
> 
> Review of attachment 8761827 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> I'm pretty sure this will break the blocklisting tests.

In general, a hardcoded vendor whitelist approach like implemented here is somewhat problematic as it does not allow to enable or realize a feature in time as the feature is not testable/visible for a particular vendor.
a feature flag with a default setting to off based on a device version list is a substantially preferred approach.

Comment 13

[Milan Sreckovic [:milan] (needinfo for best results)]

Fair enough, though I'm not convinced that device 0x0 and vendor 0x0 is something that we want to just let through with full acceleration and hope for the best, especially since it seems like user virtualization (e.g., AppSense) also leaves us with device and vendor set to 0x0.

Is there some other indication that we're in the situation we're in, that we have a Miracast wireless display going on?

Comment 15

[Paul Blinzer]

(In reply to Milan Sreckovic [:milan] from comment #13)
> Fair enough, though I'm not convinced that device 0x0 and vendor 0x0 is
> something that we want to just let through with full acceleration and hope
> for the best, especially since it seems like user virtualization (e.g.,
> AppSense) also leaves us with device and vendor set to 0x0.
> 
> Is there some other indication that we're in the situation we're in, that we
> have a Miracast wireless display going on?

Looking through the Windows GfxInit code, it should be possible to identify the underlying GPU hardware and drivers by directly enumerating the hardware via SetupDi...() APIs and looking for a Graphics device class driver instead of first using EnumDisplayDevices() and then the SetupDi...() mechanism and distinguish it from user virtualization scenarios. That would also simplify some of the detection code for Hybrid Graphics and PowerXpress with multiple GPUs.

Comment 16

[Milan Sreckovic [:milan] (needinfo for best results)]

Ernest, lets see if we can do what Paul suggests in comment 15.

Comment 17

[Milan Sreckovic [:milan] (needinfo for best results)]

Also see the special case for Windows 8, where we deal with 0x0 devices:
https://dxr.mozilla.org/mozilla-central/source/widget/windows/GfxInfo.cpp#308

Comment 18

[Ernest Yim [:ernest]]

By using EnumDisplayDevices() we are given a struct DISPLAY_DEVICE that has a flag that allows us to figure out the primary display device. Would you happen to know another way of obtaining whether or not a particular device in the device info set given by SetupDi...() call is the primary display?

Comment 19

[Paul Blinzer]

(In reply to Ernest Yim [:ernest] from comment #18)
> By using EnumDisplayDevices() we are given a struct DISPLAY_DEVICE that has
> a flag that allows us to figure out the primary display device. Would you
> happen to know another way of obtaining whether or not a particular device
> in the device info set given by SetupDi...() call is the primary display?

The SetupDi...() calls enumerate the graphics devices/drivers. Technically the display outputs are child devices of the graphics adapter, but independent. For a remote or some wireless display scenarios, the concept of primary is not as relevant (and with Windows10, the concept of primary display related to taskbar and other features. The primary display can be identified enumerated via the DXGI APIs too...

Comment 20

[Ernest Yim [:ernest]]

In the case of virtual adapters, would you happen to know how d3d decides which display device adapter will we used for a particular context created by a program?

Comment 21

[Paul Blinzer]

I can best refer you to the DXGI and Desktop Window Manager descriptions where its behavior is outlined (at least in general).  

https://msdn.microsoft.com/en-us/library/windows/desktop/bb205075%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396

https://msdn.microsoft.com/en-us/library/windows/desktop/ee417025(v=vs.85).aspx#multiple_monitors

https://msdn.microsoft.com/en-us/library/windows/desktop/aa969540(v=vs.85).aspx

Typically the "primary adapter" is chosen, though it depends on the API chosen and how it is used.

Comment 22

[Paul Blinzer]

Any update?

Comment 23

[Benoit Girard (:BenWa)]

No, I never got time to look at this any more than :ernest did. Sorry :(

Comment 24

[Paul Blinzer]

Anything to help facilitate a resolution of this bug?

Comment 25

[Milan Sreckovic [:milan] (needinfo for best results)]

OK, I have a patch that functionally seems to do the right thing, but it illustrates we can have some code simplification, so I'll see what I can do to clean it up.

Comment 26

[Milan Sreckovic [:milan] (needinfo for best results)]

(In reply to Milan Sreckovic [:milan] from comment #25)
> OK, I have a patch that functionally seems to do the right thing, but it
> illustrates we can have some code simplification, so I'll see what I can do
> to clean it up.

May as well do the code simplification in a separate bug, so that we land functionality change. https://treeherder.mozilla.org/#/jobs?repo=try&revision=bdd8cdcc9efb93ce698dd263f06eaffbe6d68f44

Comment 27

[Milan Sreckovic [:milan] (needinfo for best results)]

Attachment: Bug 1257375: If the first adapter found is 0/0, look for a valid device using the alternative approach (already used for the second GPU detection.)

Review commit: https://reviewboard.mozilla.org/r/129226/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/129226/

Comment 28

[Kartikaya Gupta (email:[e-mail])]

Comment on attachment 8857283 [details]
Bug 1257375: If the first adapter found is 0/0, look for a valid device using the alternative approach (already used for the second GPU detection.)

https://reviewboard.mozilla.org/r/129226/#review132088

I don't really know this code well but the patch looks sane. The code transformations are in line with the comments and the logic of the change seems fine. I just don't know how valid the assumptions behind the change are (i.e. that devices with a single non-zero value are ok but with dual zeros are bad, and that using the second device instead of the first is ok).

Comment 29

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cc1c8ee5af0b
If the first adapter found is 0/0, look for a valid device using the alternative approach (already used for the second GPU detection.)  r=kats

Comment 30

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/cc1c8ee5af0b