1150503 - [Meta] Video/Camera stream output to to the connected external display

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: General, defect)

Description

[Sotaro Ikeda [:sotaro]]

* This is an user story of project "Support multi-screen on FxOS" (bug 1116089). *

*[story 1]
When a user is locally playback a video, user want to watch a video in a Big display. User choose a remote display from a local UI. Then video playback is seamlessly changed to remote display. Local ui might continue to show the video playback. When user could control the video playback via local UI, the remote video playback responds synchronously.

*[story 2]
User is in the middle of WebRTC video chat. User choose a remote display from a local UI. Then peer camera's video stream is shown to the remote display. Local ui might continue show the video stream.

Comment 1

[Sotaro Ikeda [:sotaro]]

There are 2 different ways to realize the story.

- [1] Send locally rendered result to remote display.
  + Wifi Display just forward locally rendered content to remote display.

- [2] Remote display get content and show result.
  + Chromecast basically do this for video content playback.
    For video playback case, if all videos are on the network, local side do not need to forward the video.

Comment 2

[Sotaro Ikeda [:sotaro]]

There is "Presentation API" that is going to handle remote display. Its spec seems to belong to [2] in comment 1. I am not sure if it could handle [1] case.

 Bug 1069230
 http://w3c.github.io/presentation-api/

Comment 3

[Sotaro Ikeda [:sotaro]]

If we want to do [story 1] by [1] of Comment 1, the following mechanism is going to becomes necessary.

- Create a sub document in an application. The sub document exists as part of the application's document.
  But it is not rendered to main display. The sub document is actually a root document of 2nd display.
  It is similar to pop up dialog

- Connect application document's video stream to a sub document's video element or connect same resource
  to a sub document's video element. Then the video stream is also rendered to sub document's video element.

Comment 4

[Sotaro Ikeda [:sotaro]]

I feel that Presentation API seems not fit well to the stories of comment 0.

Comment 5

[Sotaro Ikeda [:sotaro]]

mstange, can you comment to Presentation API?

Comment 6

[Sotaro Ikeda [:sotaro]]

On android, when an application want to render a different content to a 2nd display, it uses Presentation
class. The Presentation class is derived from Dialog class. Then an application can render to 2nd display as same way to render to a local dialog.

A render Surface for 2nd display has a LayerStack(identifier) of 2nd display. SurfaceFlinger identifies Layers for 2nd display by using the LayerStack.

The following is a diagram around android Presentation class.
   https://github.com/sotaroikeda/android-diagrams/raw/master/graphics/Presentation_5.1.pdf

Comment 7

[Marco Chen [:mchen]]

(In reply to Sotaro Ikeda [:sotaro] from comment #2)
> There is "Presentation API" that is going to handle remote display. Its spec
> seems to belong to [2] in comment 1. I am not sure if it could handle [1]
> case.
> 
>  Bug 1069230
>  http://w3c.github.io/presentation-api/

Quote from W3C spec as above - "takes into account displays that are attached using wired (HDMI, DVI or similar) and wireless technologies (MiraCast, Chromecast, DLNA, AirPlay or similar)."

Refer to the discussion on the webscreens mail list [1], some members will mentioned about 1 UA and 2 UAs cases. The [2] in comment 1 can be 2 UA case and [1] would be 1 UA case. 1 UA means that you have a device with browser engine and the attached display is a dumb one.

So [2] is intended to support Presentation API on 1 UA case.
And [3] is intended for 2 UA case.

[1] http://lists.w3.org/Archives/Public/public-webscreens/2014Nov/0013.html
[2] Bug 1116089 - [Meta] Support multi-screen on FxOS
[3] Bug 1115480 - [Presentation WebAPI] built-in device discovery and control protocol

Comment 8

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #6)
> On android, when an application want to render a different content to a 2nd
> display, it uses Presentation
> class. The Presentation class is derived from Dialog class. Then an
> application can render to 2nd display as same way to render to a local
> dialog.
> 
> A render Surface for 2nd display has a LayerStack(identifier) of 2nd
> display. SurfaceFlinger identifies Layers for 2nd display by using the
> LayerStack.
> 
> The following is a diagram around android Presentation class.
>   
> https://github.com/sotaroikeda/android-diagrams/raw/master/graphics/
> Presentation_5.1.pdf

To create this bug, I imaged android's Presentation class. It could output application local data directly to 2nd display. Therefore I put a diagram of android Presentation class in comment 6.

  http://developer.android.com/reference/android/app/Presentation.html

Comment 9

[BMO Automation]

Firefox OS is not being worked on