1463697 - webrtc : screen orientation is not supporting in FF when a call from mobile application to desktop

Status: RESOLVED DUPLICATE of bug 1316448

(Core :: WebRTC: Audio/Video, defect)

Description

[Dipjay Datta]

Attachment: Firefox_issue.webm

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36

Steps to reproduce:

1> use any webrtc application in mobile device and start a conference.
2> join same conference from desktop FF
3> do camera rotation in device between landscape & portrait mode. 


Actual results:

FF desktop is not adjusting the stream feed.


Expected results:

FF desktop should adjust the feed and change the orientation view accordingly.

Comment 1

[Andreas Pehrson [:pehrsons]]

I tried Chrome stable and Firefox stable on appear.in's mobile web on Android (Samsung galaxy S8), and the appear.in native app on Android. For all these, rotating the Android device resulted in a correctly displayed video in Firefox 62.

Dipjay, could you provide more specific steps to reproduce? Thanks!

Byron, do you know what's expected wrt rotation when receiving? Do we support interpretation of the CVO bits as mentioned in [1]?


[1] https://tools.ietf.org/html/rfc7742

Comment 2

[Byron Campen [:bwc]]

We definitely do not try to negotiate the urn:3gpp:video-orientation RTP extension. I do not know whether the webrtc.org code uses it anyway, though. It may also be that screen orientation stuff is happening on the capture end too; jib?

Comment 3

[Andreas Pehrson [:pehrsons]]

We always rotate on the capture end. The question is more how we handle receiving frames from a client that doesn't. I guess if that RTP extension isn't negotiated, the sending peer needs to rotate it's frames before encoding?

Then it again comes down to which the sending client is, and what is negotiated, since I cannot reproduce.

Comment 4

[Byron Campen [:bwc]]

Well, if we don't negotiate the RTP extension, and the other end doesn't handle rotation in capture, then yeah it won't work right. Does Chrome also handle rotation in capture (I would be surprised if it did not)? Maybe Dipjay was using some other webrtc implementation on the Android device?

Comment 5

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Andreas already answered in comment 3 that we rotate on the capture end. From comment 1 it sounds like we can't reproduce on Firefox 62, though note this was reported on 60. Andreas, did you try on 60?

Getting this reproduced seems like the next step, before we take any action or contemplate RTP extensions.

Comment 6

[Dipjay Datta]

(In reply to Andreas Pehrson [:pehrsons] from comment #1)
> I tried Chrome stable and Firefox stable on appear.in's mobile web on
> Android (Samsung galaxy S8), and the appear.in native app on Android. For
> all these, rotating the Android device resulted in a correctly displayed
> video in Firefox 62.
> 
> Dipjay, could you provide more specific steps to reproduce? Thanks!
> 
> Byron, do you know what's expected wrt rotation when receiving? Do we
> support interpretation of the CVO bits as mentioned in [1]?
> 
> 
> [1] https://tools.ietf.org/html/rfc7742

Hi Andreas. sorry for the late reply. Sending client is just a webrtc client application. which is not able to rotate sending frames in its end. I have produced it in FF 60.  So is FF stable is not able to rotate receiving frames ??

Comment 7

[Andreas Pehrson [:pehrsons]]

Same result in Firefox 60 when the remote end is either of [chrome for android, appear.in native app for android, firefox for android].

(In reply to Dipjay Datta from comment #6)
> (In reply to Andreas Pehrson [:pehrsons] from comment #1)
> > I tried Chrome stable and Firefox stable on appear.in's mobile web on
> > Android (Samsung galaxy S8), and the appear.in native app on Android. For
> > all these, rotating the Android device resulted in a correctly displayed
> > video in Firefox 62.
> > 
> > Dipjay, could you provide more specific steps to reproduce? Thanks!
> > 
> > Byron, do you know what's expected wrt rotation when receiving? Do we
> > support interpretation of the CVO bits as mentioned in [1]?
> > 
> > 
> > [1] https://tools.ietf.org/html/rfc7742
> 
> Hi Andreas. sorry for the late reply. Sending client is just a webrtc client
> application. which is not able to rotate sending frames in its end. I have
> produced it in FF 60.  So is FF stable is not able to rotate receiving
> frames ??

So if the receiving end is Chrome, how does it know how to rotate the frames?
Which spec does it rely on and are we broken because 1) we don't negotiate it, or 2) we negotiate it but it doesn't work?

You need to put up some proof that this is indeed a bug in Firefox, as I'm unable to verify.

Alternatively put up a simple example that shows the issue so we can take a look.

Comment 8

[Byron Campen [:bwc]]

(In reply to Andreas Pehrson [:pehrsons] from comment #7)
> Same result in Firefox 60 when the remote end is either of [chrome for
> android, appear.in native app for android, firefox for android].
> 
> (In reply to Dipjay Datta from comment #6)
> > (In reply to Andreas Pehrson [:pehrsons] from comment #1)
> > > I tried Chrome stable and Firefox stable on appear.in's mobile web on
> > > Android (Samsung galaxy S8), and the appear.in native app on Android. For
> > > all these, rotating the Android device resulted in a correctly displayed
> > > video in Firefox 62.
> > > 
> > > Dipjay, could you provide more specific steps to reproduce? Thanks!
> > > 
> > > Byron, do you know what's expected wrt rotation when receiving? Do we
> > > support interpretation of the CVO bits as mentioned in [1]?
> > > 
> > > 
> > > [1] https://tools.ietf.org/html/rfc7742
> > 
> > Hi Andreas. sorry for the late reply. Sending client is just a webrtc client
> > application. which is not able to rotate sending frames in its end. I have
> > produced it in FF 60.  So is FF stable is not able to rotate receiving
> > frames ??
> 
> So if the receiving end is Chrome, how does it know how to rotate the frames?
> Which spec does it rely on and are we broken because 1) we don't negotiate
> it, or 2) we negotiate it but it doesn't work?

   Looking at the offer/answer exchange, Chrome seems to support the urn:3gpp:video-orientation RTP extension, which would be sufficient if the client application on the phone does too. We do not support this extension, which would explain why it does not work in firefox. I'm just going to file a bug for that, and we can mark this one as a duplicate if/when we verify that this is indeed the problem.