1252788 - Integrate ChromeCast into Presentation API

Status: RESOLVED WONTFIX

(Core :: DOM: Core & HTML, defect, P5)

Description

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

We should be able to integrate ChromeCast via Google Cast API.
https://developers.google.com/cast/docs/remote

Comment 1

[Andrew Overholt [:overholt]]

("btpp-fixlater" just to indicate that I don't know your timeline :)

Comment 2

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

(In reply to Andrew Overholt [:overholt] from comment #1)
> ("btpp-fixlater" just to indicate that I don't know your timeline :)

Sure no problem! I'm just documenting all the working items and we'll figure out the priority and timeline later.

Comment 3

[Chih-Hsuan Kuo [:tommykuo]]

Hi Jim, I'm working on Chromecast support for Fennec. I can display something remotely on Chromecast already. And now, I want to integrate GeckoView into that. [1] is my WIP patch. And I found Fennec doesn't support multiple GeckoView currently[2].

Could you give me some suggestions to create multiple GeckoView? Any workaround is fine, I just want to create a rough prototype for that. Thank you!

[1]: https://github.com/KuoE0/gecko-dev/commit/6c5c5bd8d7c1492ed85414727022f089d1502848
[2]: https://dxr.mozilla.org/mozilla-central/source/widget/android/nsWindow.cpp#105-107

Comment 4

[(inactive) Jim Chen [:jchen] [:darchons]]

Hi Tommy, using multiple GeckoViews is possible, but you would need a different chrome XUL document than the default, because Fennec's browser.xul does not support multiple GeckoViews. The chrome XUL URI is set at [1], so you could add a string parameter to Open() and pass in a different URI for your second GeckoView.

[1] https://mxr.mozilla.org/mozilla-central/source/widget/android/nsWindow.cpp?rev=16663eb3dcfa#1123

Comment 5

[Chih-Hsuan Kuo [:tommykuo]]

Hi Jim, thanks for your help. Now, I can make Chromecast show 'https://www.mozilla.org/en-US/' by using a bad way. And I have some questions about nsWindow and GeckoView:

1. Is GeckoViewSupport::Open is called when a GeckoView is created? So, if I want to create a new nsWindow for Android, I think I don't need to call nsWindowWatcher.openWindow directly.

2. How to pass the parameters to GeckoViewSupport::Open? I think GeckoViewSupport::Open is called here[1].

3. How does `OnNativeCall` work? Can you give me some hint?

Thank you for your help :)

[1]: https://dxr.mozilla.org/mozilla-central/source/widget/android/nsWindow.cpp#193-198

Comment 6

[(inactive) Jim Chen [:jchen] [:darchons]]

(In reply to Tommy Kuo [:KuoE0] from comment #5)
> 1. Is GeckoViewSupport::Open is called when a GeckoView is created? So, if I
> want to create a new nsWindow for Android, I think I don't need to call
> nsWindowWatcher.openWindow directly.

Right, when a GeckoView is first created, Android calls GeckoView.onAttachedToWindow, and we create a nsWindow in there through the native call to GeckoViewSupport::Open [1].

[1] https://mxr.mozilla.org/mozilla-central/source/mobile/android/base/java/org/mozilla/gecko/GeckoView.java?rev=829d3be6ba64#232

> 2. How to pass the parameters to GeckoViewSupport::Open? I think
> GeckoViewSupport::Open is called here[1].

So to pass a parameter to GeckoViewSupport::Open, you need to add a parameter to GeckoView.Window.open [2], and add an argument to the calls at [1].

After you change the Java code, you need to build Fennec one time. The build will fail, but in the build output you can see a command you have to run to synchronize the C++ JNI code with the Java code changes. After you run the command, you can change the C++ signature at [3], and everything should work.

[2] http://mxr.mozilla.org/mozilla-central/source/mobile/android/base/java/org/mozilla/gecko/GeckoView.java?rev=829d3be6ba64#118

[3] http://mxr.mozilla.org/mozilla-central/source/widget/android/nsWindow.cpp?rev=829d3be6ba64#1108

> 3. How does `OnNativeCall` work? Can you give me some hint?

OnNativeCall is called in C++ when a native Java method is called. The Java call arguments are put into a C++ function object that is passed to OnNativeCall. When you call the function object, i.e. aCall();, the Java call arguments are passed to the target C++ functions. I made a diagram to illustrate the sequence [4]. 

[4] https://people.mozilla.org/~nchen/img/jni-natives.png

Comment 7

[Chih-Hsuan Kuo [:tommykuo]]

Jim, thank you for the answer!!! It is very helpful!

Comment 8

[Chih-Hsuan Kuo [:tommykuo]]

Hi Jim, I have a problem on the resolution. In `onAttachedToWindow()`, I got 1280x760 for Chromecast. But the page shows the resolution is 360x592. I have no idea about where the numbers come from. Is the reason I open an URL from WindowWatcher.OpenWindow() directly? For now, I have not create another browser.xul for Chromecast. Or maybe there is another reason?

My source code is here[1]. Please ignore the commit "add remote_browser.xul", I don't use it yet. Thank you!

[1]: https://github.com/KuoE0/gecko-dev/commits/chromecast-on-fennec

Comment 9

[Chih-Hsuan Kuo [:tommykuo]]

Oh, I know what the problem is. On Nexus 5, the physical resolution is 1080 (w) x 1920 (h), but the CSS resolution is 360 (w) x 640 (h). It caused by the pixel ratio (3 on Nexus 5). I think I need to set pixel ratio to 1 for Chromecast!

Comment 10

[Chih-Hsuan Kuo [:tommykuo]]

After some investigation on the wrong resolution, I found the nsDeviceContext we got on Chromecast is wrong. I think we always get the device context of the primary screen. So, I filed bug 1282003 to make nsScreenManagerAndroid support multiple screens like nsScreenManagerGonk.

Comment 11

[Chih-Hsuan Kuo [:tommykuo]]

I think some people are interested in the frame rate on Chromecast devices. I turn on the pref "layers.acceleration.draw-fps" to show the frame rate on the web page. And open the URL "http://www.testufo.com/#test=framerates" to test the frame rate. And I found the maximum fps is about 35.

Comment 12

[Olli Pettay [:smaug][[e-mail]](vacation until Nov 16)]

Curious, what kind of frame rate does Chrome give?

Comment 13

[Chih-Hsuan Kuo [:tommykuo]]

(In reply to Olli Pettay [:smaug] (high review load, please consider other reviewers) from comment #12)
> Curious, what kind of frame rate does Chrome give?

I have no idea about how to test it on Google Chrome. I don't find any method to render a web page to Chromecast on Google Chrome by Remote Display API.

Currently, Google Chrome (desktop) support to mirror a page or cast a video on the Chromecast. And the Android ver. can only cast a video to the Chromecast.

Comment 14

[Chih-Hsuan Kuo [:tommykuo]]

I found the reason why the frame rate is low, because I used the debug build. I retry with release build, and the frame rate is 50 ~ 60.

Comment 15

[Chih-Hsuan Kuo [:tommykuo]]

Attachment: frame rate test

Comment 16

[Chih-Hsuan Kuo [:tommykuo]]

Frame test video on Youtube: https://youtu.be/kSijqQLLd-M

Comment 17

[Olli Pettay [:smaug][[e-mail]](vacation until Nov 16)]

(In reply to Tommy Kuo [:KuoE0] from comment #14)
> I found the reason why the frame rate is low, because I used the debug
> build. 
Heh :)

> I retry with release build, and the frame rate is 50 ~ 60.
Great, thanks.

Comment 18

[Kagami Rosylight [:saschanaz] (they/them)]

Bug 1697680 removed Presentation API implementation.