Closed Bug 877248 Opened 7 years ago Closed 6 years ago

Preview Surface for Camera is visible in upper left corner

Categories

(Firefox for Android :: General, defect, minor)

ARM
Android
defect
Not set
minor

Tracking

()

VERIFIED FIXED
Firefox 25
Tracking Status
firefox23 --- unaffected
firefox24 + verified
firefox25 + verified
fennec 24+ ---

People

(Reporter: gcp, Assigned: gcp)

References

(Blocks 1 open bug)

Details

(Whiteboard: [getUserMedia][android-gum+])

Attachments

(3 files, 1 obsolete file)

Android requires a preview surface to connect the camera to. Pre-Android 4.0, some tricks must be used which make the surface invisible on most phones.

From my understanding it's impossible to make this surface go away on all phones, but I'd like to to be hidden on as much as possible.

Currently, it's visible as a small white squire on my Android 3.2 Galaxy 10.1. It's not visible on the Nexus 4 and Galaxy S3 (even though we don't use the Android4+ method to ensure that).

I'd like it if QA could report their findings per phone here.
Duplicate of this bug: 872194
QA Contact: aaron.train
Of the devices I have tested most of gUM on thus far here is what I have seen:

Visible surface:

* Asus Transformer Prime TF201 (Android 4.1)
* Asus Nexus 7 (Android 4.2)

No visible surface:

* Alcatel One Touch 8008X (Android 4.1)
* HTC One (Android 4.1)
* HTC Nexus One (Android 2.3)
* Samsung Galaxy SII (Android 4.0 & Android 4.1)
* Samsung Galaxy SIII (Android 4.1)
* Samsung Galaxy Note - (Android 4.1)
* Samsung Galaxy Note II - (Android 4.1)
* Samsung Nexus S (Android 4.1)
* Samsung Galaxy Nexus (Android 4.2)
* LG Nexus 4 (Android 4.2)

Interestingly only the two tablets I have looked at so far exhibit the upper white dot (see https://bugzilla.mozilla.org/attachment.cgi?id=749445). I'll be taking a look at a few more next week.
This is very useful information already. We should do the "guaranteed no visible square" thing for Android 4.x as that will at least fix your two bad devices.
Surface preview

* Samsung Galaxy SIV (Android 4.2)
tracking-fennec: --- → ?
Attached image LG Nexus 4 (Screenshot) (obsolete) —
I retract on the Nexus 4 (4.2.2), I do see the surface preview image only when I animate open the tab tray for whatever reason
Attached image LG Nexus 4 (Screenshot)
Actually this is obvious to see on a new private tab
Attachment #758749 - Attachment is obsolete: true
Whiteboard: [getUserMedia][android-gum-]
Assignee: nobody → gpascutto
tracking-fennec: ? → 24+
After seeing this myself, I'll retract my original non-blocking comment. I didn't realize the little white blob in the corner was the camera until I looked closer. That's probably worth blocking on.
Whiteboard: [getUserMedia][android-gum-] → [getUserMedia][android-gum+]
I got this yesterday too just after updating to the latest Nightly build. I hadn't seen this before--or hadn't noticed it before!

Using an LG Nexus 4, Android 4.2.2

I had to "downgrade" to a build from past week (7th June) because I couldn't get any WebRTC camera stuff to work (getting NS_FAILURE or similar).
>After seeing this myself, I'll retract my original non-blocking comment.

As explained in the initial comment, this bug is not fixable in the general case (for Android < 4.0), so I don't think we can block on it.

>I had to "downgrade" to a build from past week (7th June) because I couldn't get 
>any WebRTC camera stuff to work (getting NS_FAILURE or similar).

Which bug is that filed as?
(In reply to Gian-Carlo Pascutto (:gcp) from comment #9)
> >After seeing this myself, I'll retract my original non-blocking comment.
> 
> As explained in the initial comment, this bug is not fixable in the general
> case (for Android < 4.0), so I don't think we can block on it.

Why can't this be fixed on Android < 4.0? We can't even introduce a workaround to hide the preview surface?
(In reply to Gian-Carlo Pascutto (:gcp) from comment #9)
> 
> >I had to "downgrade" to a build from past week (7th June) because I couldn't get 
> >any WebRTC camera stuff to work (getting NS_FAILURE or similar).
> 
> Which bug is that filed as?

It was an strange coincidence, but what seemed to be WebRTC's fault was actually WebGL. The bug is already fixed resolved :-)
https://bugzilla.mozilla.org/show_bug.cgi?id=881681
(In reply to Jason Smith [:jsmith] from comment #10)
> (In reply to Gian-Carlo Pascutto (:gcp) from comment #9)
> > >After seeing this myself, I'll retract my original non-blocking comment.
> > 
> > As explained in the initial comment, this bug is not fixable in the general
> > case (for Android < 4.0), so I don't think we can block on it.
> 
> Why can't this be fixed on Android < 4.0? We can't even introduce a
> workaround to hide the preview surface?

I haven't looked into the implementation details, but I can guesstimate why, as I used to fight over this exact issue when I did Android programming. You need to display the camera preview on a SurfaceView on Android < 4.0. This SurfaceView area "cuts" through the view and is always on top. The only solution is to hide it by making it small enough. But not too much or some phones fail :-P

Android 4.0 introduced a new class that CAN be hidden.
(In reply to Soledad Penades [:sole] [:spenades] from comment #12)
> (In reply to Jason Smith [:jsmith] from comment #10)
> > (In reply to Gian-Carlo Pascutto (:gcp) from comment #9)
> > > >After seeing this myself, I'll retract my original non-blocking comment.
> > > 
> > > As explained in the initial comment, this bug is not fixable in the general
> > > case (for Android < 4.0), so I don't think we can block on it.
> > 
> > Why can't this be fixed on Android < 4.0? We can't even introduce a
> > workaround to hide the preview surface?
> 
> I haven't looked into the implementation details, but I can guesstimate why,
> as I used to fight over this exact issue when I did Android programming. You
> need to display the camera preview on a SurfaceView on Android < 4.0. This
> SurfaceView area "cuts" through the view and is always on top. The only
> solution is to hide it by making it small enough. But not too much or some
> phones fail :-P

Hmm...okay. If we have this restriction, then maybe UX can work some magic to reduce the impact of this bug if we're required to show this.

Ian - Do you have any ideas from a UX perspective on what we can do here to reduce the impact of this limitation on Android < 4.0?

> 
> Android 4.0 introduced a new class that CAN be hidden.

Okay, so we can fix this on Android 4.0 then.
Flags: needinfo?(ibarlow)
(In reply to Jason Smith [:jsmith] from comment #13)

> Hmm...okay. If we have this restriction, then maybe UX can work some magic
> to reduce the impact of this bug if we're required to show this.
> 
> Ian - Do you have any ideas from a UX perspective on what we can do here to
> reduce the impact of this limitation on Android < 4.0?


Heh, "UX magic". My first suggestion would be to keep looking for a way to hide that tile ;)

Let me make sure I understand this correctly: 
In order for the camera to work, we have to show a small video preview on older devices? 

If that is the case then I have other questions:
* Is this true even now that we have icons in the notification bar indicating the state of a user's camera or microphone?
* What about the actual video preview in web content? How does that relate to this little thing in the top corner?
* Obvious question, but, what do other mobile browsers do on Android? 
* Lastly, if there is simply no way to hide that tile, can we maybe not put it over top of UI chrome, and tuck it in the bottom right corner of the web content area?
Flags: needinfo?(ibarlow)
Comment 12 nails it, see also my blog post from a while ago: http://www.morbo.org/2013/02/why-twitter-pictures-are-sideways.html

>* Is this true even now that we have icons in the notification bar indicating the state of a user's camera or microphone?

Yes, it's completely unrelated. It's a technical limitation of pre-ICS Android.

>* What about the actual video preview in web content? How does that relate to this little thing in the top corner?

No relation at all. The webpages can do anything: show a preview, can show the remote video, both or neither of those things. But the local video *must* be previewed somewhere on the screen, and I don't really see a way to reuse the preview (which would be in a <video> element!) that the webpage *might* have as we have no idea how or where that will be.

Basically we're stuck with having to put that preview somewhere visible on screen. But only on select pre-ICS devices where none of the workarounds work.

>* Obvious question, but, what do other mobile browsers do on Android? 

Not work. [1]

>* Lastly, if there is simply no way to hide that tile, can we maybe not put it over top of UI chrome, and tuck it in the bottom right corner of the web content area?

Yeah, that seems like the most sensible thing to do.

[1] Not support WebRTC on pre-ICS Android, more specifically.
> >* Lastly, if there is simply no way to hide that tile, can we maybe not put it over top of UI chrome, and tuck it in the bottom right corner of the web content area?
> 
> Yeah, that seems like the most sensible thing to do.

Can we make it as small as possible then? Would a 1dp x 1dp square still allow it to function?
>Can we make it as small as possible then? Would a 1dp x 1dp square still allow it to function?

On some devices yes, on some devices, no. QA will have to test what works where.
Note II with Nightly 24.0a1(2013-06-21), I can see the small region appear at top left corner. Hope this help.
* Nexus 7 (Android 4.2.2)
* Galaxy Nexus (Android 4.2.2)

Both of devices with 2013-06-21.

In case of Galaxy Nexus, to reproduce a problem, you should zoom page and make title bar visible.

https://www.dropbox.com/sc/v15idgpk75ezsp0/MadZThpJOR
1) Despite claims that you can make the View of a SurfaceTexture/TextureView invisible and have the camera work, this didn't seem to work on my test devices. However what does seem to work is to set the alpha channel of the texture to transparent. (On HoneyComb this takes effect after a delay, or at least it looks like this, very strange). I'm leaving in the code to use TextureView instead of SurfaceView where supported, because I presume that does actually work faster/better where it exists.

2) You can't have a class that implements an interface that belongs to a higher SDK than the one on the device. The classloader will fail. This is worked around with making the SurfaceTextureListener an inner class that is only instantiated if you're on the right device SDK level.

3) Modifications to the layout so the Surface, if visible, should be near on the lower right instead.

QA should prolly play with the try build before we land this:
https://tbpl.mozilla.org/?tree=Try&rev=42654a5381a2
Attachment #768387 - Flags: review?(blassey.bugs)
Tested the try-build on Acer Iconia A500 (Android 3.2), Motorola RAZR (Android 2.3.3) and Samsung Galaxy R (Android 2.3.4) and the preview surface is not visible when testing on http://mozilla.github.io/webrtc-landing/gum_test.html. On the latest Nightly all these devices display the surface view so this patch fixes the issue.
(In reply to Adrian Tamas from comment #21)
> Tested the try-build on Acer Iconia A500 (Android 3.2), Motorola RAZR
> (Android 2.3.3) and Samsung Galaxy R (Android 2.3.4) and the preview surface
> is not visible when testing on
> http://mozilla.github.io/webrtc-landing/gum_test.html. On the latest Nightly
> all these devices display the surface view so this patch fixes the issue.

Can you try an Android 4.0 or higher device as well?
Attachment #768387 - Flags: review?(blassey.bugs) → review+
https://hg.mozilla.org/mozilla-central/rev/4f24e84e1da8
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 25
Keywords: verifyme
Note - this needs to be uplifted to Aurora.
Whiteboard: [getUserMedia][android-gum+] → [getUserMedia][android-gum+][webrtc-uplift]
Comment on attachment 768387 [details] [diff] [review]
Patch 1. Make Surface invisible where possible

[Approval Request Comment]
Bug caused by (feature/regressing bug #): WebRTC support
User impact if declined: White square visible over browser chrome when getUserMedia is used.
Testing completed (on m-c, etc.): Landed on m-c few days ago
Risk to taking this patch (and alternatives if risky): White square elsewhere, or other general browser chrome/layout breakage. 
String or IDL/UUID changes made by this patch: NA
Attachment #768387 - Flags: approval-mozilla-aurora?
Attachment #768387 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
Status: RESOLVED → VERIFIED
Keywords: verifyme
Whiteboard: [getUserMedia][android-gum+][webrtc-uplift] → [getUserMedia][android-gum+]
You need to log in before you can comment on or make changes to this bug.