Closed Bug 704229 Opened 13 years ago Closed 8 years ago

[meta] Refine touch video controls


(Firefox for Android Graveyard :: General, defect, P3)



(relnote-firefox 48+, blocking-fennec1.0 -, fennec+)

Tracking Status
relnote-firefox --- 48+
blocking-fennec1.0 --- -
fennec + ---


(Reporter: jaws, Unassigned)



(Keywords: meta, Whiteboard: [html5Video])


(13 obsolete files)

Please see bug 470628. We probably need a full-screen button on Fennec Native video controls.
Depends on: 470628
Assignee: nobody → margaret.leibovic
Priority: -- → P3
tracking-fennec: --- → 11+
blocking-fennec1.0: --- → ?
I don't think this needs to block. It wasn't a feature in XUL fennec, and you can still get to full screen from the context menu.
(In reply to Margaret Leibovic [:margaret] from comment #1)
> I don't think this needs to block. It wasn't a feature in XUL fennec, and
> you can still get to full screen from the context menu.

solid reasoning
tracking-fennec: 11+ → 14+
blocking-fennec1.0: ? → -
tracking-fennec: 14+ → 16+
Not sure this needs to track 16. Maybe tracking "+" ?
tracking-fennec: 16+ → ?
I haven't had time to work on this myself, so I'm marking it as a mentor bug. It may not be a good *first* bug, but it should be relatively straightforward. A good starting point is definitely to take a look at bug 470628.
Assignee: margaret.leibovic → nobody
Whiteboard: [mentor=margaret][lang=js][lang=css][lang=xbl]
tracking-fennec: ? → -
I wish to work on this particular bug.
Assignee: nobody → sucheta.ghoshal
(In reply to Sucheta from comment #5)
> I wish to work on this particular bug.


Sucheta, do you have a build environment set up for Fennec? If not, you should see:

Like I mentioned, this bug will be very similar to bug 470628, and actually touch some of the same files. The video controls are in a file shared by desktop and mobile [1], although there is a separate "touchControls" binding for mobile [2], which is what you'll want to change for this patch. I would start by trying to get a full screen button to show up in there, using the same button icon that was used for desktop. Once we see what that looks like we can get some help from UX to see if there's a different button style we want for mobile.

Feel free to leave questions in here, or attach a WIP ("work in progress") patch for feedback. You can also join us in #mobile on for more real-time help.

Sucheta, hey. Just curious if you've had any time to play with this and have any questions. Happy to help!
Blocks: 834931
Wes, can you take this bug? Without it, fullscreen videos on B2G lack a visible way to exit them.

If you can't take it, please reassign to someone who can get to it quickly.
Flags: needinfo?(wjohnston)
No problem. On paternity now, but I'll be back in next week and can take this. We'll at least need some artwork though (and these controls ideally need a fuller redesign on mobile, but we can put that off for a different bug).
Flags: needinfo?(wjohnston) → needinfo?(ibarlow)
No longer blocks: 834931
Blocks: 834931
I'm taking this bug because I've been assigned to work on bug 834931
Assignee: sucheta.ghoshal → dflanagan
Since bug 834931 looks to be fixing this only for b2g, I'm going to hijack this to move the shared styles/images into toolkit. I'm also going to try and rework these to be a bit prettier. I'll post screenshots and hopefully ian can give me some feedback.

We'll put that into toolkit as our default "touch" theme, and products that want to alter it (maybe Android will want to use Android stock icons for play/pause/etc eventually) can extend or alter those.
Summary: Add a full-screen button for HTML5 video (Fennec Native) → Unify touch video controls
No longer blocks: 834931
It turns out we can fix this for FirefoxOS and Fennec separately, so bug 834931 no longer depends on this one. I'm reassigning to Wesley.

Wesley: note that for FirefoxOS, you can't rework stuff to be prettier without sending everything through the visual design team. I know that the FirefoxOS team would like to change these controls dramatically to make them similar to what we have in the FirefoxOS Video app. I don't know if Fennec has its own team of visual designers. If so, maybe you can get the Fennec and B2G designers to agree on a shared design.

Otherwise, it would be best to continue to give FirefoxOS a way to override the default styles and images, or you'll risk the wrath of the FirefoxOS visual design team.
Assignee: dflanagan → wjohnston
I've been told that Ian Barlow is working on media controls for Fennec currently, so I'll sync up with him to decide on a unified direction.  Please hold off on any work until we figure out the visuals.

Attached patch WIP Patch (obsolete) — Splinter Review
This is a WIP. We're actually getting closer and closer to just using the desktop controls with some interaction tweaks here. This switches things so that the touch controls look and work mostly fine using just the desktop theme images. We show the clickToPlay button for video's instead of having a play button in the control bar. It also moves the fullscreen button to the left and makes some tweaks to the scrubber bar.

It needs a better scrubber image (I faked one here using CSS), a better "pause" image for the click to play button, and potentially some larger icons for fullscreen/mute. Also, better rtl support, and there's some issue with the controls not showing at all in fullscreen...

I'll post some images to get UX feedback.
Attached image Video (paused) (obsolete) —
Attached image Audio (paused) (obsolete) —
Comment on attachment 711033 [details]
Audio (paused)

Both the audio and video look good to me (note that the reason the audio image is showing a larger than necessary box is due to bug 704326).

Does the volume control need to be present for touch? As far as I know, most touch devices have hardware volume controls. If the video element has focus, can we use the hardware volume controls to control the video volume? (or just set video volume to 100% and then let hardware controls modify system volume?)
Attached patch WIP v2 (obsolete) — Splinter Review
Getting better. This fixes the default images and sorta fixes the fullscreen bug (our controls are getting reinitialized, but the clickToPlay button isn't hidden correctly leading to crazyness, I fixed it by force-showing the controls any time you click and they aren't currently shown).

Biggest fix is getting some slightly better images. I did some SVG spriting tricks to reuse the click-to-play svg stuff.
Whoops. Meant to post, build with these is at:
Whiteboard: [mentor=margaret][lang=js][lang=css][lang=xbl]
Blocks: 842130
Blocks: 804989
This is blocked on getting feedback from fennec+b2g ux teams. In the long run, we can slip images/size/border/color/etec differences in and our for different platforms. Initially, we just need something that looks good enough to be our "base" theme, and more importantly, to have the correct layout/interactions.
Maybe peter can help unfreeze this?
Flags: needinfo?(pla)
Whiteboard: [html5Video]
Assignee: wjohnston → nobody
Flags: needinfo?(ibarlow)
Will mock up player controls for both small embedded and full screen videos that take into consideration FFOS controls and current FFX Android/Desktop controls.
As per ^,

Based off of ibarlow's mockup, included new play and pause icons that I made but didn't get the chance to redo the 'full screen' icon yet so I'm using the FFOS one I found for the time being.

Wesj is in the process of building out a quick prototype to test it out on actual devices.
Attached image Screenshot (obsolete) —
Still trying to fix this up a lot, but wanted to give UX something. WIP:

Some test files;

Note the play/pause buttons are wrong. I can move in some images that look more like the screenshot. We have a similar image, but its a java resource, not available here. Its also not SVG like the image desktop uses here.

I think if we're going to change these images, we should do it uniformly across Firefox Desktop/OS/Android, so I'm a bit nervous about changing them at all :(

I also think I want to hide the play button based on the size of the video (currently its a portrait/landscape decision).
Attachment #711032 - Attachment is obsolete: true
Flags: needinfo?(pla)
As per wesj's comments, as it stands right now we're going to try and use the FX Desktop icons for the media controls and get a build going to test out the issues with different (namely really small) video sizes. 

Reusing icons also helps build some visual consistency between our products.
We should reconsider this as part of the video improvements.
tracking-fennec: - → ?
tracking-fennec: ? → +
Let's stop trying to unify and just get these refinements done.
Summary: Unify touch video controls → Refine touch video controls
Help us antlam you're our only hope
Flags: needinfo?(alam)
Attached image prev_videocont_mock2.png (obsolete) —
Attaching a WIP for this first...

Main points: FS icon (unFS icon not in the mock but I have one), mute/unmute icon, new icons all around (hand crafted just for us).

As a middle ground (since we know the scrubber is finicky, I came up with a compromise in the middle but I think we can push for something a little sweater (far right). 

On that note, Far right mock opens up space along the bottom for future "cast" icon, additional sliders (volume?), etc. This is also still missing video clip progress and length indicators so I will be trying to add those in. But I also foresee a lot of tweaking especially around smaller video sizes where we can't surface all these controls.

Stay tuned!
Flags: needinfo?(alam)
Attached image prev_mediacont_icons.png (obsolete) —
Also attaching a quick snap shot at what our lib of media controller icons are looking like ATM
Attached image prev_videocont_mock3.png (obsolete) —
Talked to Ian about this a little bit and made some iterations. Attaching a WIP so at least we can have something to move the needle on.

Takeaways from this are mainly the "Proposed" and "Proposed 2". I want to see if we can handle "medium" sized videos like this (i.e. move the controls around). The visuals are still not 100% yet doing some crits on it with Ian about the look & feel but this is the direction. "Proposed 2" is the state that shows when the scrubber is active, putting into the foreground the spot in the video in which the scrubber is at (0:07 in the mock).

"Landscape FS" is simply full screen mode. Only thing out of the ordinary here is maybe the double "play" buttons. I simply want to leverage the entire video as a "play/pause button" so it would be easier to trigger as this is a common interaction.

Lastly, "Proposed (small)" is where it gets interesting and essentially what we talked about Wes. We sometimes run into this situation where the video is too small to put all the controls in there so I want to explore if we can do something smarter and display the appropriate controls in the appropriate videos. 

In this particular WIP, I'm experimenting with pulling out all the controls and floating a media control bar at the bottom (not unlike our treatment with second screen initiatives like Roku). I'm open to some feedback here of course as I realize this would have many implementation implications.
Attached image prev_mediacont_icons3.png (obsolete) —
Attaching an updated overview of our current media icons as it stands
Attachment #8438759 - Attachment is obsolete: true
Attachment #8438758 - Attachment is obsolete: true
(In reply to Anthony Lam (:antlam) from comment #34)
> Created attachment 8439543 [details]
> prev_videocont_mock3.png

Looks awesome!
Attached image prev_videocont_mock4.png (obsolete) —
Wanted to nudge this bug along as it's a relatively quick win it seems.

Attaching design mocks so we can move forward with this... Please ignore the dotted line around the video that was on the screen shot hah. :P  

This mock will contain info just on videos that are large enough to put controls on. I'll follow this up with what we can hopefully do for smaller videos in my next comment.

The controls in the middle mock sit on a #363B40, 0.45 opacity background matte that goes from the bottom of the scrubber track to the bottom of the screen (this is to help ensure controls visibility). 

Proposed 2 (Right mock) -  
When the user picks up the scrubber, the orange surrounding it enlarges, and the time/place is displayed in Roboto, Light on white (42 sp in the mock). The overlay is the same (#363B40) but at 0.7 opacity and is over the entire video. Letting go fades away the coverlay and resumes the video.
Attachment #8439543 - Attachment is obsolete: true
Flags: needinfo?(wjohnston)
Attached image prev_videocont_mock4b.png (obsolete) —
^Part 2!

Here is the mock for full screen and smaller videos.

I would like to try and pull out the video controls on websites that have tiny videos playing. Not unlike our interface for our second screen work, the bar that contains our video controls would be pinned to the bottom (#363B40 1.0 opacity).

For full screen, the background matte for the controls bar (#363B40) would revert to 0.45 opacity. Ideally, the entire area above the bar would be used as a hit area for play/pause as well.

Let me know as questions pop up.
Friendly ping on the status here :)
Attached file (obsolete) —
Attaching some icons (+ more) that I've used over on bug 1055809.. we could re-use these for our video player (when the video is too small, as outlined above :)
Attached file (obsolete) —
adding icons for FS mode (bigger)
NI-ing Mark :)

Just wondering - these video controls seem to surface themselves more often now that we have the second screen projects picking up for 34.. should we spend some time on these controls now?
Flags: needinfo?(mark.finkle)
Just be careful. The video element controls can become a slipper slope. Focus on small sets of changes that give incremental improvements. Also note that even though the Media ControlBar (for casting) and the video element controls look similar, they are different formats and have different UI limitations.

In general, I think it's fine to work to make the controls look the same, but the work is not a high priority (does not need to ship with fx33 or fx34).
Flags: needinfo?(mark.finkle)
Flags: needinfo?(wjohnston)
What is the state of the mock-ups in this bug? Are they still up to date?

What are the changes we want to make here? If it's just about updating the icons, we can do that in bug 1065076.

Maybe we should turn this into a meta bug, and track more concrete pieces of work in separate bugs.
Flags: needinfo?(alam)
Depends on: 1227599
Keywords: meta
Summary: Refine touch video controls → [meta] Refine touch video controls
(In reply to :Margaret Leibovic from comment #47)
> What is the state of the mock-ups in this bug? Are they still up to date?
> What are the changes we want to make here? If it's just about updating the
> icons, we can do that in bug 1065076.
> Maybe we should turn this into a meta bug, and track more concrete pieces of
> work in separate bugs.


Trying to remember right now and I think the changes we want to make here are focused around the player's behaviours as a whole and bug 1065076 is just about icons..
Flags: needinfo?(alam)
Attachment #711028 - Attachment is obsolete: true
Attachment #711033 - Attachment is obsolete: true
Attachment #713108 - Attachment is obsolete: true
Attachment #819195 - Attachment is obsolete: true
Attachment #8439547 - Attachment is obsolete: true
Attachment #8446098 - Attachment is obsolete: true
Attachment #8446107 - Attachment is obsolete: true
Attachment #8475533 - Attachment is obsolete: true
Attachment #8475540 - Attachment is obsolete: true
QA Contact: flaviu.cos
Depends on: 1259442
I don't think there's anything else left to do here. If there are more improvements we want to me, let's file a new bug.
Closed: 8 years ago
Resolution: --- → FIXED
Depends on: 1267935
Hi all, I will take this feature as a QA. Here is the Test plan based on which the testing is made:
QA Contact: cos_flaviu → sorina.florean
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.