945603 - [RTSP] Support Rtsp Protocol in URL Bar on b2g Browser

Status: VERIFIED FIXED

(Firefox OS Graveyard :: RTSP, defect, P1)

Description

[Ethan Tseng [:ethan]]

This is a follow-up bug of bug 884702.
In bug 884702, we made rtsp a non-exposed protocol in b2g OS and also added an external handler for rtsp scheme, which would launch video app. The result is that we support <a href="rtsp://some-url"> for now.
We would also like to support rtsp in URL bar.
If feasible, url bar code could do an exposed-protocol check.

Comment 1

[Ethan Tseng [:ethan]]

Note that bug 884702 made a mistake (we should use an activity instead of a system message to launch the video app) and bug 947132 fixed this mistake.
As a result, to support rtsp in URL bar, the b2g browser just need to new a MozActivity to open the video app.

Comment 2

[Wesley Huang [:wesley_huang] (EPM) (NI me)]

Hi John,
Do you think you can take this? Or please suggest someone?
Somehow we are working on to land RTSP feature in 1.3, and supposedly should be done before Dec12.
This bug enables URL bar case which I think is an important feature.

Comment 3

[Wesley Huang [:wesley_huang] (EPM) (NI me)]

Let's defer it to 1.4 as 1.3 should be FC at this moment.

Comment 5

[Wesley Huang [:wesley_huang] (EPM) (NI me)]

Ben,
Do you think you can take this?

Comment 6

[Ben Francis [:benfrancis]]

Rather than have the browser app fire the web activity, could Gecko launch the Web Activity when the user navigates to an rtsp:// URI, like we already do for mailto: and tel: links?

There wouldn't be much point in adding this feature to the browser app as it's being replaced by the system browser in 1.4.

Comment 7

[Wesley Huang [:wesley_huang] (EPM) (NI me)]

Hi Ken,

Might need to evaluate from architectural perspective whether it's good to let Gecko do it.
Next step I would like to know if it's feasible to spec in 1.4. 
If not, I will communicate with product manager for deferring this feature as early as possible.

Comment 8

[Ken Chang[:kenkai]]

Ethan is preparing the developing story for this bug...:-).

Comment 9

[Ethan Tseng [:ethan]]

(In reply to Ben Francis [:benfrancis] from comment #6)
> Rather than have the browser app fire the web activity, could Gecko launch
> the Web Activity when the user navigates to an rtsp:// URI, like we already
> do for mailto: and tel: links?
> There wouldn't be much point in adding this feature to the browser app as
> it's being replaced by the system browser in 1.4.

Hi Ben and Wesley,

While working on bug 884702 (Make Rtsp non-exposed protocol and support rtsp in HTML <a> tag), we had considered the feasibility of loading RTSP url type from Gecko. Briefly speaking, although feasible, but we think it's not adequate to load RTSP URI in Gecko.

The protocols such as mailto and rtsp are not "exposed protocols" to Firefox. Which means Gecko engine does not handle these protocols internally but hand off them to external helpers instead (mostly launching applications to handle them). On the other hand, for exposed protocols such as http, we will do the URI load by DocShell object, which is responsible for initiating loading of content from a URI, creating the content viewer and hooking it up to the incoming content.

Although we support for mailto in URL bar on B2G, the way we did that is a little tricky and looks like a workaround. We deliberately made mailto as an exposed protocol to let its load enter DocShell. When the "dummy" mailto channel is created by mailto protocol handler, it sends a message to shell.js, which will new a MozActivity to launch Gaia apps. Then the mailto channel throws an exception to break off the load in DocShell. In another word, mailto is still treated as an unknown protocol from the perspective of Gecko/DocShell.

Unfortunately we cannot do the same thing for rtsp because rtsp needs a real channel for media element, such as <video src=rtsp://...>. We cannot throw an exception in rtsp channel.
Logically, URL bar is not part of Gecko. It's a part of the browser app. Therefore, to support rtsp in URL bar, we think the ideal solution is the browser app do the exposed check as we did for link click. 

I tried my best to describe what I know so far. Maybe we could consult Boris Zbarsky. He is the expert of this topic. :)

Comment 10

[Ethan Tseng [:ethan]]

Hi Boris, we had some discussions on this issue last month. Ben asked about the feasibility of launching rtsp for URL bar from Gecko, and I just summarized our considerations above. But I am afraid it's not comprehensive due to my limited knowledge. Could you please provide some advice?

Comment 11

[Boris Zbarsky [:bzbarsky]]

As I said during our discussion, it really depends on how we want to handle <iframe src="rtsp://...">.  If the handling for that should be the same as the handling for url bar, then we can (and should) probably add something to handle them in docshell.  But if you want different handling for iframes and url bar, then url bar should be the one doing whatever special-case handling it wants.  So this comes down to the desired UX.

Comment 12

[[:fabrice] Fabrice Desré]

Just guessing, but I think we want the same behavior as when we load another video resource, ie. use the buitin media player to handle the playback.

Comment 13

[Ethan Tseng [:ethan]]

The iframe case is filed in bug 940840 (Support Rtsp protocol in HTML iframe tag).
I think according to Fabrice's advice, the behaviors of URL bar and iframe are different.

The expected behavior and current status of support are summarized as follows:
+---------------------------|-----------------------------------------|--------+
| Case                      | Expected Behavior                       | Status |
|---------------------------|-----------------------------------------|--------|
| <video>                   | Display by the builtin media player     | Done   |
|---------------------------|-----------------------------------------|--------|
| <a href="rtsp://...">     | Launch video app                        | Done   |
|---------------------------|-----------------------------------------|--------|
| URL bar                   | Launch video app                        | N/A    |
|---------------------------|-----------------------------------------|--------|
| <iframe src="rtsp://..."> | Display by the builtin media player     | N/A    |
+---------------------------|-----------------------------------------|--------+

If we agree to these behaviors, URL bar should be handled in Gaia.
And iframe will need to be handled in Gecko via a mechanism similar to MediaDocument and VideoDocument now.

Boris and Fabrice, please correct me if my comprehension is wrong. :)

Comment 14

[[:fabrice] Fabrice Desré]

Ethan, that's not what I meant. In the browser currently if you use a direct url to a video (or an image) we display it *in the browser*, like if was a simple document with <video src=url>. I think gecko has content viewers for that, but I don't know the details on how to hook up a protocol there (current ones are likely triggered based on mime types)

Comment 15

[Ethan Tseng [:ethan]]

I misunderstood Fabrice's comment. Actually we want the same behaviors for both URL bar and iframe.

The table is updated here.
+---------------------------|-----------------------------------------|--------+
| Case                      | Expected Behavior                       | Status |
|---------------------------|-----------------------------------------|--------|
| <video>                   | Display by the builtin media player     | Done   |
|---------------------------|-----------------------------------------|--------|
| <a href="rtsp://...">     | Launch video app                        | Done   |
|---------------------------|-----------------------------------------|--------|
| URL bar                   | Display by the builtin media player     | N/A    |
|---------------------------|-----------------------------------------|--------|
| <iframe src="rtsp://..."> | Display by the builtin media player     | N/A    |
+---------------------------|-----------------------------------------|--------+

Comment 16

[Ethan Tseng [:ethan]]

Boris, just confirm that, according to the conclusion above, we should handle rtsp loading in Gecko, right?
If yes, I will take this bug. Last time you mentioned that maybe we should create a new idea such as "media protocol" to handle such cases. Do you still think it's the right path to go?

Comment 17

[Boris Zbarsky [:bzbarsky]]

Given comment 15, handling in Gecko makes sense, yes.

How we want to do that... basically you want to create a custom document to handle the load.  We have facilities for doing that by MIME type, so if we can just make rtsp:// channels return a particular MIME type and the document set things up as needed.

Comment 18

[Ethan Tseng [:ethan]]

(In reply to Boris Zbarsky [:bz] from comment #17)
> Given comment 15, handling in Gecko makes sense, yes. 
> How we want to do that... basically you want to create a custom document to
> handle the load.  We have facilities for doing that by MIME type, so if we
> can just make rtsp:// channels return a particular MIME type and the
> document set things up as needed.
Boris, thanks for your information.
I just identified the code path of loading video from http:// channel. I will try to implement the same thing for the MIME type "RTSP".

Comment 19

[Ethan Tseng [:ethan]]

Attachment: WIP bug-945603-wip.patch

This is a WIP patch.
I can successfully create a video document for rtsp with this patch.
However, the problem of bug 902644 (genlock failures during h.264 video playback) happens quite often in my experiment. Need to work further for this.

Comment 20

[Ethan Tseng [:ethan]]

Hi Steve, I added some codes in DecoderTraits member functions to support RTSP channel from URL bar. Just like Fabrice and Boris commented above, the content viewer and document facilities set all the rest things up.
While I am still working to identify the issue in comment 19, could you take a look to the patch and make sure we are on the right path?

Comment 21

[Pin Zhang [:pzhang] (inactive)]

(In reply to Ethan Tseng [:ethan] from comment #15)
> I misunderstood Fabrice's comment. Actually we want the same behaviors for
> both URL bar and iframe.
> 
> The table is updated here.
> +---------------------------|-----------------------------------------|------
> --+
> | Case                      | Expected Behavior                       |
> Status |
> |---------------------------|-----------------------------------------|------
> --|
> | <video>                   | Display by the builtin media player     | Done
> |
> |---------------------------|-----------------------------------------|------
> --|
> | <a href="rtsp://...">     | Launch video app                        | Done
> |
> |---------------------------|-----------------------------------------|------
> --|
> | URL bar                   | Display by the builtin media player     | N/A 
> |
> |---------------------------|-----------------------------------------|------
> --|
> | <iframe src="rtsp://..."> | Display by the builtin media player     | N/A 
> |
> +---------------------------|-----------------------------------------|------
> --+

What's the case for window.open('rtsp://....')?

Comment 22

[Ethan Tseng [:ethan]]

(In reply to Pin Zhang [:pzhang] from comment #21)
> What's the case for window.open('rtsp://....')?
Hi Pin. Since window.open() opens the URL in a new browser window, I think it should behave the same as URL bar, i.e., display in the builtin media player. I will further verify its behavior. Thank you for raising this issue.

Comment 23

[Steve Workman [:sworkman] (INACTIVE)]

Comment on attachment 8362463 [details] [diff] [review]
WIP bug-945603-wip.patch

Review of attachment 8362463 [details] [diff] [review]:
-----------------------------------------------------------------

Ethan, this looks like the right direction to me. However, I think you should have someone from the media team confirm it. Also, CanHandleMediaType seems to be called by MediaSource code - I see comments to suggest that the codec list is already restricted in MediaSource code, but I think you should make sure that there are no weird or unexpected effects here.

Comment 24

[Steve Workman [:sworkman] (INACTIVE)]

Matthew, can you confirm that this patch looks ok for enabling an RTSP load as a video document, and that it doesn't interfere with MediaSource?

Comment 25

[Matthew Gregan [:kinetik]]

Yep, looks fine.  Thanks for checking!

Comment 26

[Ethan Tseng [:ethan]]

Steve and Matthew, thanks for your feedback!
Right now I am working on some issues raised by this patch. It's mainly about resource management. It seems we didn't release rtsp media resource (including its track buffer) while leaving the browser page containing a video document.

Comment 27

[Ken Chang[:kenkai]]

Blocking 1.4 RTSP feature.

Comment 28

[Ken Chang[:kenkai]]

It blocks 1.4 feature.

Comment 29

[Jason Smith [:jsmith]]

No longer going to block - RTSP video needs to be preffed off in 1.4 per a drivers decision to cut scope to only DSDS & QC required features.

Comment 30

[Ethan Tseng [:ethan]]

This bug will be resolved in bug 992568 (Refactor RtspChannel to support HTTP->RTSP redirection and rendering inside the browser), which converging implementations of several bugs.

Comment 31

[Ethan Tseng [:ethan]]

Resolved fixed by bug 992568.

Now we support entering an RTSP link from the URL bar of the B2G browser.

Comment 32

[Ethan Tseng [:ethan]]

*** Note ***
RTSP video is not enabled by default for now.
Turn on it by manually setting preference:
pref("media.rtsp.video.enabled", true);

Bug 999914 will turn on RTSP video.

Comment 33

[Ethan Tseng [:ethan]]

(In reply to Pin Zhang [:pzhang] from comment #21)
> What's the case for window.open('rtsp://....')?

I had verified calling window.open() for an RTSP link in JS.
It works as well.

Comment 34

[William Hsu [:whsu]]

Verified it.
- https://moztrap.mozilla.org/results/cases/?&pagenumber=1&pagesize=50&sortfield=created_on&sortdirection=desc&filter-id=11927&filter-run=4017

* Build information.
 - Gaia      e8a08a3f7a608993f0b302371e016e73faceea70
 - Gecko     https://hg.mozilla.org/mozilla-central/rev/2897fb554990
 - BuildID   20140505160203
 - Version   32.0a1