Closed Bug 1009684 Opened 9 years ago Closed 5 years ago

Define a UX specification for handling CSS :active states on touch devices

Categories

(Firefox OS Graveyard :: General, defect)

Product:
Firefox OS Graveyard
Component:
General
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: tif, Unassigned)

References

Details

(Whiteboard: interaction-design) 

Basically, when does the highlight appear and disappear for touch targets. The basics should be defined, but there are also many edge cases as well:

- What happens if you touch and your finger moves within the target area? Outside of it?
- After how many ms does the touch target appear
- What happens if you touch and hold on a button with one finger and a second finger is placed somewhere else on-screen?
- What happens if there is an existing touch on-screen and a second finger touches a button with an :active state?

And I'm sure there are many more...
Hey Rob/Harly - who would be the right person to write the spec? This is related to Bug 986884 - at a higher level.
Flags: needinfo?(rmacdonald)
Flags: needinfo?(hhsu)
The spec should be iOS / Android. They behave consistently with each other in the fundamental touch interactions. It's not in our interest to reinvent those established patterns. If firefox OS behaves differently on those basic interactions we can alienate new smartphones users jumping into our wagon and we would make harder for existing android/ios users to switch.
(In reply to Diego Marcos [:dmarcos] from comment #2)
> The spec should be iOS / Android. They behave consistently with each other
> in the fundamental touch interactions. It's not in our interest to reinvent
> those established patterns. If firefox OS behaves differently on those basic
> interactions we can alienate new smartphones users jumping into our wagon
> and we would make harder for existing android/ios users to switch.

+1
Given comment #3, shall we just resolve this invalid and assume iOS/Android is the correct behavior? Tif? I don't want a UX spec or comment on expected behavior to delay a fix, and this reads like we have agreement - or not? :)
If everyone is happy with that plan than I'm sure the poor person who would have been tasked with creating the spec would happily pass up the opportunity. Just as long as we are all on the same page.

Is that cool Rob?
I still think even if we do go with this approach that it would be worthwhile to at least break down all the parts of the touch interactions on iOS/Android so that there is no confusion for the developer implementing the behaviors. It doesn't have to be anything extraordinary, just a list of a couple example interactions and their expected behaviors.
Then it's back to Rob and Harly to figure out if Frameworks team will address this.
I just talked with Harly and, if it's urgent, we're fine with working with existing standards for now. Due to workshops, the next frameworks meeting isn't until the week of the 26th so it'd be at least a couple of weeks before we have a proposal in place.

We'd still need to break it down and document it as Justin suggested. It'd be great if engineering could give us a head start by sending us the details.

Also, it's likely Casey has already considered these issues so I'll NI him on the bug.
Flags: needinfo?(rmacdonald)
Flags: needinfo?(kyee)
Flags: needinfo?(hhsu)
I can start documenting the expected behaviors tomorrow.
Whiteboard: interaction-design
I've marked this as a blocker of Bug 995532 - [camera] buttons don't consistently highlight when tapped

The reason I believe that we are not seeing consistent highlighting of buttons is in times where perhaps the event loop is backed up or the UI is bogged down, we *currently* display highlighting using a CSS :active state. However, the behavior of the :active state will cause the highlighting to only appear when your finger is actually pressed down on the screen. If you tap quickly and release your finger before the UI has a chance to update, you will not see any highlighting at all.

I've observed the following in iOS and I believe this is part of why iOS seems to feel highly responsive with regards to *instant* visual feedback for touch input (using the "Contacts" app in iOS as an example for the following two cases):

1.) If you tap and hold on any item on-screen, you will notice there is a slight delay in the highlight state. The reason this seems to be the case is in the case where you are sliding your finger up/down to scroll the list. Since there is a delay in drawing the highlight state while your finger is still pressed down on the screen, you do not see any accidental highlighting appear in the list as you scroll.

2.) If you tap and release *quickly*, you will *immediately* observe the highlighted state at the exact moment that your finger is released from the screen. It seems that in this case, there is a "momentary" active state that is toggled briefly immediately following a quick tap. The reasoning behind this, I believe, is to give the user a sense of immediate visual feedback even if the touchstart/touchend events happen very quickly before the event loop has a chance to respond. If the highlighted state is only drawn while a finger is still pressed down on the screen, there is a chance that they may release the touch before the UI can be updated. I think this is the reason why we are seeing inconsistency in Bug 995532 with regards to the highlighted states sometimes not being observed at all.

Currently in FxOS, if you tap quickly you may or may not observe the :active state (depending on how busy the device is). But, if you tap and hold, you *always* can observe the :active state, which goes away the moment the touch is released.
CSS :active states seem to be *completely* broken using latest nightly. Simple test case with a couple buttons with :active styles here: http://jsbin.com/vorib
(In reply to Justin D'Arcangelo [:justindarc] from comment #11)
> CSS :active states seem to be *completely* broken using latest nightly.
> Simple test case with a couple buttons with :active styles here:
> http://jsbin.com/vorib

Indeed! Filed bug 1013378.
More inconsistent handling of CSS :active states and their correlation to 'click' events reported here:

Bug 1013556 - FxOS 'click' events don't match :active element
Blocks: 1014322
We had a discussion during the work week and have a prototyped solution that addresses this issue here using a CSS animation:
http://codepen.io/caseyyee/pen/lEzsj

I would love to hear feedback on the solution and if there are any comments/concerns about implementing what we have here.
Flags: needinfo?(kyee)
(In reply to Casey Yee [:cyee] from comment #14)
> We had a discussion during the work week and have a prototyped solution that
> addresses this issue here using a CSS animation:
> http://codepen.io/caseyyee/pen/lEzsj
> 
> I would love to hear feedback on the solution and if there are any
> comments/concerns about implementing what we have here.

Casey: I really like the hack you used to make the :active state linger around a bit longer after the touch is released. However, we need to establish a clear specification as to how the underlying platform should behave with the CSS :active states with regards to touch input. For instance, if you have one finger already pressed somewhere on the screen and a 2nd finger touches a button in a different part of the screen, what should happen? Similarly, what should happen if you touch a button and slide your finger around *within* the bounds of the button? These are the types of questions we need answered in a clear specification.

That being said, I think your proof-of-concept of using CSS animation to further emphasize the :active/highlighted states on buttons is a great idea to help reinforce visual feedback to the users
cyee: Nice one, this looks like a promising fix! I was initially worried about us cooking up some horrible JS solution. I like how this 'extends' the normal :active behaviour to fulfil a specific requirement :)

Noticed that there are still issues around the 'tap, hold, drag' scenario losing :active state.
(In reply to Justin D'Arcangelo [:justindarc] from comment #15)
> (In reply to Casey Yee [:cyee] from comment #14)
> > We had a discussion during the work week and have a prototyped solution that
> > addresses this issue here using a CSS animation:
> > http://codepen.io/caseyyee/pen/lEzsj
> > 
> > I would love to hear feedback on the solution and if there are any
> > comments/concerns about implementing what we have here.
> 
> Casey: I really like the hack you used to make the :active state linger
> around a bit longer after the touch is released. However, we need to
> establish a clear specification as to how the underlying platform should
> behave with the CSS :active states with regards to touch input. For
> instance, if you have one finger already pressed somewhere on the screen and
> a 2nd finger touches a button in a different part of the screen, what should
> happen? Similarly, what should happen if you touch a button and slide your
> finger around *within* the bounds of the button? These are the types of
> questions we need answered in a clear specification.

I'm not sure that we want to be changing the platform behavior for this.    I think its pretty important to be consistent in how the :active styles behave (for us and others) and changing these may lead to unintended behavior elsewhere.  

In terms of issues around how this would work for tap, hold and drag scenarios -- Are there specific issues that need to be addressed?  I think the default behavior here is in my opinion, fine.

The only thing of course is that a CSS solution only will work with apps that apply it.   But I think if we apply these in as building blocks for our own apps/styles, I think that shouldn't be a problem.
(In reply to Casey Yee [:cyee] from comment #17)
> I'm not sure that we want to be changing the platform behavior for this.   
> I think its pretty important to be consistent in how the :active styles
> behave (for us and others) and changing these may lead to unintended
> behavior elsewhere.  
> 
> In terms of issues around how this would work for tap, hold and drag
> scenarios -- Are there specific issues that need to be addressed?  I think
> the default behavior here is in my opinion, fine.
> 
> The only thing of course is that a CSS solution only will work with apps
> that apply it.   But I think if we apply these in as building blocks for our
> own apps/styles, I think that shouldn't be a problem.

The more I think about it, I'm a bit torn on this issue lately. Of course we *could* always get the user experience we want by avoiding :active states altogether and putting together some elaborate JS solution that listens for the touch events and applies an `.active` CSS class. But, to me, that feels like we would be coding *around* the platform to circumvent the default behavior. I think there's still room to compromise some of these issues within the platform without damaging pre-existing content. The main issue I have with encapsulating what we consider to be expected app behavior within our building blocks is that 3rd-parties would also have to use our building blocks to get the same behaviors. While I think its a fantastic idea to have a common library of building blocks for use across all of the built-in Gaia apps (provides a consistent look-and-feel), I don't think the building blocks should be required to mask strange behaviors with :active states.

However, regardless of how we choose to solve this in Firefox OS (platform vs. building blocks), we should *still* come up with a detailed UX specification that outlines exactly what is expected around all of the edge cases.

Also, if the consensus is that we should do nothing about the current behavior of :active states on touch input devices, then perhaps we should have a way for apps to "opt-in" to a new/better behavior for :active states. It could be as simple as defining an additional parameter on the viewport that hints to Gecko that we should treat :active states differently (optimized for "apps"). Or, it could be a -moz- prefixed pseudo state in CSS that behaves differently. Something like:

button:-moz-active {
  background-color: #f00;
}

Just a thought. It would be nice if we can allow 3rd-party apps to leverage whatever solution we do come up with so that they can also enjoy a better user experience without having to re-invent the wheel.
This needs UX design. Adding to the UX backlog for 2.1.
ux-b2g: --- → 2.1
Any progress on this? 2.1 FL is done now so presumably this is getting bumped to 2.2?
Stephany/Rob,

Any update on this? Its probably too late for 2.1 also.

Thanks
Hema
Flags: needinfo?(swilkes)
Flags: needinfo?(rmacdonald)
Yes, it is too late - though I'm not sure why I added UX *after* Casey already did a prototype. I would resolve this as invalid, actually, and take the work Casey did for CSS across Gaia, for 2.2, and just cover this that way.
Flags: needinfo?(swilkes)
As mentioned in the discussion above, the work Casey did is not a comprehensive spec. It's a prototype of how the active element highlighting could work in one specific case. A full spec should indicate how it behaves in *all* the cases (e.g. what happens when you put down a second finger? How far do you have to drag your finger to lose focus?)

And this is apart from the fact that you have to view Casey's prototype in a browser, and so how the browser implements focus/active elements will affect what the prototype looks like. What you see the prototype behaving as may be quite different from what I see it behaving as depending on which browsers we're using.
Flags: needinfo?(swilkes)
The work Casey did is not a comprehensive spec, but the work that Przemek did to specify 2.2 styles and behaviors (some of which Casey implemented) is. 

And this is still for 2.2.
Flags: needinfo?(swilkes)
Can you provide a pointer to the work that Przemek did? This is the first I've heard of it.
(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #25)
> Can you provide a pointer to the work that Przemek did? This is the first
> I've heard of it.

Przemek, can you point Kartikaya to the files?
Flags: needinfo?(rmacdonald) → needinfo?(pabratowski)
OK - I guess Rob and Przemek say there's not enough detail in Przemek's doc to address this, so we can add that for 2.2 (which is between now and 2.2 start on Jan. 5, 2015).
Thanks. Keep in mind that we will need to implement the spec as well, so if the spec is delivered right before the end of the year the implementation won't make in 2.2.
Kartikaya, UX has a particular schedule that we follow for every release. Specs are always due, reviewed etc. well in advance of the first sprints beginning. Please let me know which PMs and EPMs you work with, as this really should be standard procedure across teams for nearly two years at this point. Thanks!
ux-b2g: 2.1 → 2.2
On the platform side we don't really work in sprints. That being said I forgot that 2.2 got pushed back to a 6-month cycle, so we might have enough time to do this. The PM we work with for the gfx platform work is Sandip Kamat, feel free to coordinate with him and Milan for timelines and such. Thanks :)
I don't have a spec on this issue at the moment.
Flags: needinfo?(pabratowski)
Przemek, that's fine. It's for 2.2 anyway.
Blocks: 1062636
Milan/Sandip,

We have one more bug around the active state issue manifesting in ringtones too. Linking that bug as well. 

Adding NI for you too just to make sure you have this in your radar and will add it to your 2.2 backlog for discussion and prioritization. 

Thanks
Hema
Blocks: 1074687
Flags: needinfo?(skamat)
Flags: needinfo?(milan)
Hema, as I understand it, once the specification exists, graphics would be able to work on bug 1013556.  Is this what we're talking about?
Flags: needinfo?(milan) → needinfo?(hkoka)
Flags: needinfo?(skamat)
(In reply to Milan Sreckovic [:milan] from comment #34)
> Hema, as I understand it, once the specification exists, graphics would be
> able to work on bug 1013556.  Is this what we're talking about?

Yes, we need to get the specs before we can figure out the fixes. And this bug is tracking the spec work. I just wanted to make sure it is in the radar for 2.2

Thanks
Hema
Flags: needinfo?(hkoka)
Hi Hema, 

May we know if this is going to happen on 2.2, will it get any chances to backporting to 2.0 by partner themselves ?  and will that be high risk ?

Could you please kindly provide your recommendations ?

Thank yo very much!
Flags: needinfo?(hkoka)
AS this doesn't seem to have any involvement from platform yet we have had to resort to building our own implementation of :active state in a library called 'pressed' [1]. The library uses touch event listeners to add CSS class hooks to elements when they are pressed.

In brief the library gives us TWO things that platform's `:active` implementation doesn't:

1. ENFORCING MINIMUM TIME-PERIOD

We are able to enforce a minimum period an element is marked as :active. This means that with quick taps, buttons still appear active for a short period of time after the touch has ended. This clarifies to the user that their touch was recognised, especially on small buttons where their finger may have been covering the button.

2. ADDITIONAL 'RELEASED' STATE

The library gives us an extra style hook that we call 'released'. An element is given the 'released' class for a few milliseconds after the finger is released. We used this class to apply specific transitions that we only want present after an interaction finished. A good example of this is a slow color fade back to the default state.

AFAIK there is no current CSS selector that we can use to target this 'released' event. We can apply transitions to the default element style properties, but then that interferes with other states that the element may have.

Example:

button {
  transition: opacity 1000ms;
}

button:active {
  opacity: 0.5;
  transition-duration: 0ms;
}

button[disabled] {
  opacity: 0.5;
  transition-duration: 0ms;
}

The issue here is when the button changes from disabled to not-disabled; we end up seeing the slow fade intended the 'release' effect. What we really want is a quick snap between disabled and not-disabled, AFAIK it isn't possible to have both, but I'd love suggestions :)

---

These are the current limitations of `:active` we have experienced when building the new gaia-components. Please ask questions if anything isn't clear :)



[1] http://github.com/gaia-components/pressed
feature-b2g: --- → 2.2+
Rachelle,

I spoke to Stephany last week about this and looks like she has marked it blocking for 2.2, so I am assuming the specs will be ready for 2.2 for Milan/Kartikeya to check what work is involved on the platform side.
Flags: needinfo?(hkoka)
I think I have pretty good idea what is required on the visual/front-end side of things, so would be happy to work with platform to give the feedback required see this through.

I'd rather not ship the pressed.js library with v2.2/v2.3 as it adds extra complexity to all our components.
Looks like this bug doesn't have any update for 2 months. Are we still targeting to have it in 2.2? Who should be the best owner? Thanks.
Flags: needinfo?(swilkes)
Given the significant changes in scope for 2.2, this is not included, per the 2.2 list from the Devices team. We can discuss with Thomas if it should be.
Flags: needinfo?(swilkes)
ux-b2g: 2.2 → ---
(In reply to Stephany Wilkes from comment #41)
> Given the significant changes in scope for 2.2, this is not included, per
> the 2.2 list from the Devices team. We can discuss with Thomas if it should
> be.

Thanks. In this case, let me remove feature-b2g:2.2+ for now. Please discuss with Thomas about this. Thanks. Ping Stephany and Thomas.
feature-b2g: 2.2+ → ---
Flags: needinfo?(tho)
Flags: needinfo?(swilkes)
I wasn't aware of this bug before. Do we still have time to make it in 2.2? There are some bugs dependent on this one. Thanks.
Flags: needinfo?(tho) → needinfo?(mtsai)
From the previous comments, it seems that this bug won't be fixed for 2.2.
UX can help to review or give some definitions if we decide to implement it in 2.2. 
Loop Rob for more inputs since this is more a system platform related issue.
Flags: needinfo?(mtsai) → needinfo?(rmacdonald)
Rob has been out so I'm removing him here. Francis and Tif, please let me know if you can help on this in time for 2.2.
Flags: needinfo?(tshakespeare)
Flags: needinfo?(rmacdonald)
Flags: needinfo?(fdjabri)
Removing the flag for myself again since this is already assigned to Rob.
Flags: needinfo?(swilkes)
Flags: needinfo?(tshakespeare)
Flags: needinfo?(rmacdonald)
Flags: needinfo?(fdjabri)
This bug has been punted around a lot so I'm adding this to the agenda of the ux frameworks meeting tomorrow to see if we can identify an owner. I'll report back post-meeting.
Flags: needinfo?(rmacdonald)
I'm getting some mixed messages about whether this is required for 2.2. Wilson, can you please clarify and flag Przemek if this is still needed?
Flags: needinfo?(wilsonpage)
(In reply to Rob MacDonald [:robmac] from comment #48)
> I'm getting some mixed messages about whether this is required for 2.2.
> Wilson, can you please clarify and flag Przemek if this is still needed?

This still needs attention at the platform level, but isn't a 2.2 blocker.
Flags: needinfo?(wilsonpage)
No-Jun brought up another use case that is similar in some respects to this bug and would also benefit from UX feedback. Please see bug 1129558.
Firefox OS is not being worked on
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.