986884 - [Camera][Madai] Settings options lose the active state when sliding the finger

Status: RESOLVED WONTFIX

(Core :: Panning and Zooming, defect)

Description

[Diego Marcos [:dmarcos]]

See attached video

Comment 1

[Diego Marcos [:dmarcos]]

Attachment: activeStateLost.MOV

Comment 2

[Wilson Page [:wilsonpage]]

Looks like a Gecko issue. We're only using CSS `:active` to style this. I wouldn't know who to talk to about this.

Comment 3

[Diego Marcos [:dmarcos]]

Who can look at this?

Comment 4

[Hema Koka [:hema]]

mike - can you help with this one?

Comment 5

[Mike Habicher [:mikeh] (high bugzilla latency)]

On my nexus 4, this is consistent with other "active links". If you go to the browser and open, e.g. www.slashdot.org, then touch a link to make it active and move your finger, the link stops being active even though my finger is very much still on it.

This may very well be intentional. Someone who understands CSS and the front end should take a look at this.

Comment 6

[Wilson Page [:wilsonpage]]

I have a fair amount of experience working with mobile WebKit, and I don't remember this ever being an issue there. I would expect `:active` state to remain until the element in question is no longer under the user's finger.

Comment 7

[Diego Marcos [:dmarcos]]

We need to summon some gecko wizards here

Comment 8

[Jason Smith [:jsmith]]

Note for the qawanted request - this is test to see if this reproduces on master after the camera-new-features branch merges to master.

Comment 9

[Hema Koka [:hema]]

dbaron: we need your help here. thanks!

Comment 10

[Marcia Knous [:marcia]]

After the merge to master this still occurs on Nexus 4 using:

Gaia   d65d0f7fa9cda0a41b98149e3c540a41eb27b11f
SourceStamp
BuildID 20140328023052
Version 31.0a1

Comment 11

[David Baron :dbaron:]

Maybe smaug can help here faster than I can?

(I certainly don't know the answer off the top of my head, so I'd need to investigate.)

Comment 12

[Josh Schmitt [Joshs]]

(In reply to Jason Smith [:jsmith] from comment #8)
> Note for the qawanted request - this is test to see if this reproduces on
> master after the camera-new-features branch merges to master.

I was able to repro on the latest 1.5 Buri build

Environmental Variables:
Device: Buri 1.5 MOZ
BuildID: 20140331040200
Gaia: 26839cb46f856d610b192f5655a8c38a6bfe0829
Gecko: d8e8f13bd4ae
Version: 31.0a1
Firmware Version: V1.2-device.cfg

Comment 13

[Olli Pettay [:smaug][[e-mail]]]

Haven't still found a machine which can play that video...
but I guess.
:active handling hasn't been spec'ed anywhere for multitouch case, and in general
it is applied only for one point.

Comment 14

[Olli Pettay [:smaug][[e-mail]]]

Ah, not about that, but about targeting different element.
What does the relevant html look? Link to the source code, please

Comment 15

[Diego Marcos [:dmarcos]]

Olli. If you want to reproduce the video just flash gaia master and go to the settings menu. Right upper icon on the screen. Touch any of the options and slide your finger to see how the active state is not applied anymore. The relevant html and css are:

<li class="setting icon-hdr-menu">
 <h4 class="setting_title">HDR</h4>
 <h5 class="setting_value">ON</h5>
</li>

.setting {
  display: flex;
  height: 7rem;
  align-items: center;
  margin: 0 1.5rem;
  border: solid 1px rgba(255,255,255,0.15);
  border-width: 1px 0 0;
  box-sizing: content-box; /* 1 */
}

/**
 * @last-child
 */

.setting:last-child {
  border-width: 1px 0 1px;
}

/**
 * @active
 */

.setting:active {
  background: #008eab;
}

.setting:active .setting_title {
  color: rgba(255,255,255, 1);
}

/** Icon
 ---------------------------------------------------------*/

.setting:before {
  display: block;
  width: 4rem;
  font-size: 3rem;
  padding-left: 1rem;
}

.icon-hdr-menu:before {
  content: "\e612";
}

Comment 16

[Olli Pettay [:smaug][[e-mail]]]

Do you perhaps have link to the code, so that I could see the whole context?

(I don't have b2g dev environment atm)

Comment 17

[Olli Pettay [:smaug][[e-mail]]]

My initial guess is anyway http://mxr.mozilla.org/mozilla-central/source/dom/browser-element/BrowserElementPanning.js?rev=fb4819dd57ea&mark=277-279#251
assuming Settings run as a "browser"

Comment 18

[Olli Pettay [:smaug][[e-mail]]]

Btw, is this a regression? Bug 986752 might be related?

Comment 19

[Diego Marcos [:dmarcos]]

The html is generated dynamically but the code is here:

https://github.com/mozilla-b2g/gaia/blob/master/apps/camera/js/views/setting.js

Comment 20

[Diego Marcos [:dmarcos]]

This problem is different than bug 986752 I don't know if they might be related somehow on the gecko side

Comment 21

[Preeti Raghunath(:Preeti)]

Is there a functionality issue here or just loss of highlight?

Comment 22

[Marcia Knous [:marcia]]

(In reply to Preeti Raghunath(:Preeti) from comment #21)
> Is there a functionality issue here or just loss of highlight?

No loss of functionality, to me it is just the behavior you would expect - if I slide my finger across the button I would expect it to remain highlighted.

Comment 23

[Preeti Raghunath(:Preeti)]

(In reply to Marcia Knous [:marcia - use needinfo] from comment #22)
> (In reply to Preeti Raghunath(:Preeti) from comment #21)
> > Is there a functionality issue here or just loss of highlight?
> 
> No loss of functionality, to me it is just the behavior you would expect -
> if I slide my finger across the button I would expect it to remain
> highlighted.

Thanks Marcia. I wouldn't block on this issue then. I'll backlog it.

Comment 24

[Justin D'Arcangelo [:justindarc]]

Is this a legitimate bug? Bug 986752 has been fixed and that deals with an issue where the :active state gets stuck when using multiple touches. However, the STR in the original video here seem to exhibit expected behavior. When you slide a touch past a certain point (even if the touch never leaves the element), the :active state is turned off. I'm not suggesting that this is desirable behavior, but it seems to be consistent with the rest of Gaia (including websites viewed through the Browser app).

Comment 25

[Wilson Page [:wilsonpage]]

This is definitely not a camera bug. Does anyone know the correct component it should be assigned to?

Comment 26

[Wilson Page [:wilsonpage]]

This is still an issue.

STEPS (working)

1. Enter camera settings menu (top-right).
2. Tap, hold for 200 - 300ms, then slide the length of the menu item.
3. Observe button remains blue while finger remains pressed.

STEPS (broken)
 
1. Enter camera settings menu (top-right).
2. Tap and *instantly* slide the length of the menu item.
3. Observe button turns blue, then flicks back to default appearance, even when finger remains pressed.

Comment 27

[Chris Lord [:cwiiis]]

I wonder if this is selection/dragging behaviour interfering somehow? Or the double-tap delay stuff maybe?

Changing component to something I hope is more suitable.

Comment 28

[Chris Lord [:cwiiis]]

needinfo?kats in case anything here jumps out wrt bug 941995. Could BrowserElementPanning be getting in the way? I'm not really sure of the sequence of events for touches on b2g atm.

Comment 29

[Chris Lord [:cwiiis]]

And actually setting it this time. (see comment #28 and comment #26).

Comment 30

[Kartikaya Gupta (email:[e-mail])]

The code for this now lives with the Panning and Zooming component, so moving this bug over to that component.

I agree the behaviour is a little unexpected here, but as mentioned above, it is consistent with behaviour in other apps. When you put your finger down and move it instantly, that is considered to be a "pan" gesture, and so the active element is cleared, and the blue highlight goes away. When you leave your finger there for 300ms before moving it, that is a "long tap" gesture, and panning after that is ignored (this is true in all apps). Since there is effectively no panning, the menu button remains as the active element.

The main thing that distinguishes this case from other more "normal" cases is that the camera screen isn't actually pannable because the content fits in the viewport. So "pan" gestures do nothing. We special-case this behaviour in gecko so that the active element highlight shows up instantly, rather than waiting for 100ms first. Other than that it should behave the same as everywhere else in Gaia.

Here are some possible courses of action:
- Leave the behaviour as-is.
- Enhance the special-case behaviour so that if you're on a page that's not pannable, and you attempt a pan, it leaves the :active state on the element you touched down. This will make the "broken" steps from comment 26 behave like the "working" steps.
- Try to maintain an invariant like "the element is :active if lifting your finger will deliver a click to it". This would mean changing the "long tap, followed by move" behaviour so that the element is no longer :active after moving, and make the "working" steps from comment 26 behave like the "broken" steps.
- Un-special case the behaviour I mentioned above. This would make it a little harder to hit this problem (although it will still happen), and will negatively affect other use cases. I don't think this is a good idea.

If any of these feel particularly "correct" to you please speak up. I'm not yet sure what consequences some of these changes will have on other use cases; I'll have to think about that.

Comment 31

[Chris Lord [:cwiiis]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #30)
> Here are some possible courses of action:
> - Enhance the special-case behaviour so that if you're on a page that's not
> pannable, and you attempt a pan, it leaves the :active state on the element
> you touched down. This will make the "broken" steps from comment 26 behave
> like the "working" steps.

This feels like the correct thing to do to me. I can't immediately think of any ill side-effects.

Comment 32

[Justin D'Arcangelo [:justindarc]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #30)
> - Try to maintain an invariant like "the element is :active if lifting your
> finger will deliver a click to it". This would mean changing the "long tap,
> followed by move" behaviour so that the element is no longer :active after
> moving, and make the "working" steps from comment 26 behave like the
> "broken" steps.

To me, this sounds a little bit closer to the desired behavior. But I think there's one more condition to add and that would be "if a move causes a pan, the element will no longer deliver a click if the finger is lifted". That additional condition would also cause the element to no longer be active because of the aforementioned condition.

However, the special-case behavior is also notable because if it is impossible to pan, then I would think you could definitely improve upon the current implementation.

Comment 33

[Justin D'Arcangelo [:justindarc]]

(In reply to Chris Lord [:cwiiis] from comment #31)
> (In reply to Kartikaya Gupta (email:[e-mail]) from comment #30)
> > Here are some possible courses of action:
> > - Enhance the special-case behaviour so that if you're on a page that's not
> > pannable, and you attempt a pan, it leaves the :active state on the element
> > you touched down. This will make the "broken" steps from comment 26 behave
> > like the "working" steps.
> 
> This feels like the correct thing to do to me. I can't immediately think of
> any ill side-effects.

The more I'm thinking about this, it doesn't sound right. If you `touchstart` on an element and you move *within* the bounds of the element before lifting causing a `touchend` to occur, a `click` should still be delivered to the element IMO. The only time the `click` should be cancelled (and therefore also the :active state) would be if either you move *outside* of the element before releasing your finger -or- if your move caused a pan to occur.

Comment 34

[Diego Marcos [:dmarcos]]

It's easy. The criteria to make a decision should be consistency with the behavior of the mainstream touch based systems (ios / android). I don't have my android and ios devices with me right now but I will upload examples of similar scenarios in both systems

Comment 35

[Justin D'Arcangelo [:justindarc]]

What I described in Comment 33 is pretty much what iOS does. I opened up the Settings app and observed the following:

1.) Touch down and hold on a list item -- *slight* delay in receiving the active/highlighted state
2.) Slide finger (any speed) carefully to the left or right (as not to scroll) -- continue observing the active/highlighted state
3.) Slide finger up or down (causes scrolling of the list) -- observe the active/highlighted state disappear *immediately*

NOTE: As long as the item appears active/highlighted (meaning you haven't scrolled), releasing your finger triggers a click. Also, if you tap quickly, the active/highlighted state appears briefly coinciding with the triggering of the click.

Comment 36

[Kartikaya Gupta (email:[e-mail])]

(In reply to Diego Marcos [:dmarcos] from comment #34)
> The criteria to make a decision should be consistency with the
> behavior of the mainstream touch based systems (ios / android).

Why?

Comment 37

[Justin D'Arcangelo [:justindarc]]

I agree with Diego that this needs to be as consistent as possible with other mainstream platforms that rely on touch input. The reason is simple: when our users interact with apps in FxOS, they should receive the same level of visual feedback that they are accustomed to seeing in other platforms. If we behave vastly different than iOS/Android with regards to touch events, it will lead to an undesirable user experience. In the past several weeks, I have seen numerous bugs come through that are related to CSS :active states where the actual behavior is not matching the expected behavior.

Setting NI? for UX so they can add their input here as well.

Comment 38

[Justin D'Arcangelo [:justindarc]]

Related bugs:

Bug 1004516 - Scrolling around in contact list leaves things highlighted

Bug 1000985 - [B2G] 'click' events are not triggered if there is an existing touch on-screen

Bug 997235 - Followup to bug 986752 - CSS :active states get stuck with multiple on-screen touches

Bug 995475 - [B2G][Camera]User attempt to adjust Zoom slider negatively affects Options icon performance intermittently

Comment 39

[Kartikaya Gupta (email:[e-mail])]

(In reply to Justin D'Arcangelo [:justindarc] from comment #37)
> when our users interact with apps in FxOS, they should receive the same
> level of visual feedback that they are accustomed to seeing in other
> platforms.

You're assuming that most of our users are *not* first-time smartphone users. If previous statements by higher-ups in the management team are to be believed, then most of our users aren't going to have any expectations in this regard.

I think we should do what we feel makes intuitive sense, regardless of what other platforms are doing. I don't know what that is, but I don't think blindly copying others is the way to go. If it happens that we decide what others do is the best solution them I'm all for it. But I'd rather think it through from a user's experience than by looking at what other platforms do.

Comment 40

[Kartikaya Gupta (email:[e-mail])]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #30)
> - Try to maintain an invariant like "the element is :active if lifting your
> finger will deliver a click to it". This would mean changing the "long tap,
> followed by move" behaviour so that the element is no longer :active after
> moving, and make the "working" steps from comment 26 behave like the
> "broken" steps.

After thinking some more, I don't know if this invariant is a good one. It is inconsistent with the web on desktop - there, if you drag a link, it goes :active when you mousedown and remains active until you mouseup, even though the mouseup doesn't trigger the link navigation. There are probably also other scenarios where we want the element to remain :active, such as drag+drop. I can imagine using long-tap to select a draggable element, and then dragging it around; in that case we probably want it to remain :active.

Comment 41

[Justin D'Arcangelo [:justindarc]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #39)
> You're assuming that most of our users are *not* first-time smartphone
> users. If previous statements by higher-ups in the management team are to be
> believed, then most of our users aren't going to have any expectations in
> this regard.

This is certainly true for lower-end devices such as the Tarako, but will likely not be the case for higher-end devices/markets.

> I think we should do what we feel makes intuitive sense, regardless of what
> other platforms are doing. I don't know what that is, but I don't think
> blindly copying others is the way to go. If it happens that we decide what
> others do is the best solution them I'm all for it. But I'd rather think it
> through from a user's experience than by looking at what other platforms do.

This is also a good point and we should have a discussion about this. However, it is also worth noting that iOS in particular is widely-accepted as having one of the best "feeling" user experiences on a touchscreen device. So, it does make sense to try and break those interactions down and figure out why that is.

For instance, in the case of Bug 1004516, scrolling a contact list leaves items stuck in a highlighted state. To me, this sounds like the reasoning behind why there is a <100ms delay in iOS for showing the highlighted state (see Comment 35). However, if there was *always* a <100ms delay in showing the highlighted state, the UI would feel slightly sluggish and unresponsive. Again, I think this is the reason why a quick 'tap' in iOS triggers a brief active/highlighted state that coincides with the 'click' event. In those cases of quick taps, the user is still getting an *immediate* visual feedback that their input has been recognized by the app.

Comment 42

[Diego Marcos [:dmarcos]]

Users have expectations about how a touch UI behaves. We've launched in Brazil, Colombia, Venezuela, Spain. I've been in all those countries (I'm Spanish by the way) and man believe me we don't live in the middle of the jungle. Many people might not be able to afford an iphone but they watch TV, they have ATMs, public spaces with touch screens, malls with android and iphones to play with and friends that actually can afford those expensive devices. They've seen and experienced how a touch screen behaves. Web standards are good right? I allows people to interoperate and collaborate. In the UI world we don't have a standards body but we have a UI language and set of metaphors that are broadly used. There's no good reason to alienate Firefox OS users from the rest of the world by implementing basic interactions that are common practice in a different way.

Comment 43

[Kartikaya Gupta (email:[e-mail])]

(In reply to Justin D'Arcangelo [:justindarc] from comment #41)
> However, it is also worth noting that iOS in particular is widely-accepted
> as having one of the best "feeling" user experiences on a touchscreen
> device. So, it does make sense to try and break those interactions down and
> figure out why that is.

Fair point.

> For instance, in the case of Bug 1004516, scrolling a contact list leaves
> items stuck in a highlighted state. To me, this sounds like the reasoning
> behind why there is a <100ms delay in iOS for showing the highlighted state
> (see Comment 35). However, if there was *always* a <100ms delay in showing
> the highlighted state, the UI would feel slightly sluggish and unresponsive.
> Again, I think this is the reason why a quick 'tap' in iOS triggers a brief
> active/highlighted state that coincides with the 'click' event. In those
> cases of quick taps, the user is still getting an *immediate* visual
> feedback that their input has been recognized by the app.

Bug 1004516 is really just an implementation bug. The intended behaviour is what you describe, where there is a 100ms delay in showing the highlighted state so that we don't flash it if the user is panning, but to eliminate the delay in cases where we know the user isn't panning and for quick taps. But you're totally right that this feels like the best overall behaviour, which is probably why iOS also does it this way.

(In reply to Diego Marcos [:dmarcos] from comment #42)
> Users have expectations about how a touch UI behaves. We've launched in
> Brazil, Colombia, Venezuela, Spain. I've been in all those countries (I'm
> Spanish by the way) and man believe me we don't live in the middle of the
> jungle.

I didn't mean to offend you, sorry if I did. But even if people *do* have experience with touch UIs, I don't think it necessarily follows that they have expectations about how it a new device that is clearly a different platform will behave, or that they will be "alienated" if it behaves slightly differently. I've used iOS and Android phones extensively and even if my life depended on it I don't think I could accurately describe how their active element highlighting behaves in all scenarios. I expect that is true for average users as well. I don't think any of us could, without actually having the phones in front of us and experimenting.

I think that if something *feels wrong* then sure, people might compare it to other phones and say "oh this is different" but as long as it feels right, nobody cares. That's what I'm after - to have it feel right so that people can just ignore it and go on with whatever they're actually trying to do. The key point here is that what feels right on other platforms may not feel right on our phone, because it has to keep in mind consistency with other parts of our platform. If everything on FxOS were identical to iOS and iOS feels 100% right then sure we can just copy it and not worry about it but FxOS isn't identical to iOS in all respects and we have to make sure we don't copy something that ends up feeling alien.

Comment 44

[Kartikaya Gupta (email:[e-mail])]

(Also for the record I'd be more than happy to defer this decision to the UX designers, for obvious reasons).

Comment 45

[Justin D'Arcangelo [:justindarc]]

Including Rob here for UX input as well.

Comment 46

[Tiffanie Shakespeare [:tif]]

The core of the argument is not about keeping up with the Jones's (hopefully that's not too American of a reference), it's about usability and user expectation. Our highlight state does not seem to consistently work therefore the user isn't always sure what is going to happen. 

We need to be a little more forgiving both with finger movement and quick taps - no one's tap is going to be 100% still and held for x milliseconds every time in order to achieve the highlight state. People are on the bus or walking down the street in addition to sitting in a cafe or lounging on the couch. They can be rushed or distracted, frustrated or calm. That's the awesomeness of mobile design - the unpredictable environment! :)

It sounds like what we really need is a UI spec that clearly outlines, across the board, how the highlight state should work. When it should appear, when it should go away, etc. Let me discuss with Rob and see who should work on that if it doesn't already exist.

Comment 47

[Rob MacDonald [:robmac]]

(In reply to Tiffanie Shakespeare from comment #46)
> 
> It sounds like what we really need is a UI spec that clearly outlines,
> across the board, how the highlight state should work. When it should
> appear, when it should go away, etc. Let me discuss with Rob and see who
> should work on that if it doesn't already exist.

Casey, is this something that is within the scope of the existing building blocks work? If not I'll add it to the BB backlog.

Comment 48

[Justin D'Arcangelo [:justindarc]]

Rob: I should clarify that this problem is part of Gecko. So, anything using CSS ':active' to indicate highlighted UI states (pretty much the de facto way of doing this) will be affected. While, it would technically be possible to re-implement this behavior in Gaia, this would not be desirable and the implementation would likely be cumbersome.

Comment 49

[Casey Yee]

(In reply to Rob MacDonald [:robmac] from comment #47)
> (In reply to Tiffanie Shakespeare from comment #46)
> > 
> > It sounds like what we really need is a UI spec that clearly outlines,
> > across the board, how the highlight state should work. When it should
> > appear, when it should go away, etc. Let me discuss with Rob and see who
> > should work on that if it doesn't already exist.
> 
> Casey, is this something that is within the scope of the existing building
> blocks work? If not I'll add it to the BB backlog.

Hey Rob, I think that putting this into the BB backlog would make the most sense and discussed as part of Frameworks.   I think this would fit into the buttons pattern as part of it's behavior and how it should work.

We have been able to use some rather hacky CSS to get the :active hit states to hang around for a few hundred ms after button presses but It doesn't seem reliable and it might be worth exploring other solutions including some platform work if required.

Comment 50

[Chris Lord [:cwiiis]]

Given that this bug exists and it's already being hacked around, it kind of sounds like we just want to make the fix in platform as mentioned in comment 31. It would be a pretty short fix, can we just get some UX comment on that? A full spec would be good, but I don't think this needs to block on that.

Comment 51

[Justin D'Arcangelo [:justindarc]]

(In reply to Casey Yee [:cyee] from comment #49)
> (In reply to Rob MacDonald [:robmac] from comment #47)
> > (In reply to Tiffanie Shakespeare from comment #46)
> > > 
> > > It sounds like what we really need is a UI spec that clearly outlines,
> > > across the board, how the highlight state should work. When it should
> > > appear, when it should go away, etc. Let me discuss with Rob and see who
> > > should work on that if it doesn't already exist.
> > 
> > Casey, is this something that is within the scope of the existing building
> > blocks work? If not I'll add it to the BB backlog.
> 
> Hey Rob, I think that putting this into the BB backlog would make the most
> sense and discussed as part of Frameworks.   I think this would fit into the
> buttons pattern as part of it's behavior and how it should work.

I'm not sure this is a BB issue. Any set of UI components, whether it be our BB or some other UI library, will ultimately exhibit the same behavior if relying on CSS :active states.

> We have been able to use some rather hacky CSS to get the :active hit states
> to hang around for a few hundred ms after button presses but It doesn't seem
> reliable and it might be worth exploring other solutions including some
> platform work if required.

Yes, I believe this needs to be addressed in the platform. However, the interactions surrounding it should be clearly defined by UX before going into implementation because there are still a lot of unanswered questions. We need to know things such as:

- What happens if you touch and hold on a button with one finger and a second finger is placed somewhere else on-screen?
- What happens if there is an existing touch on-screen and a second finger touches a button with an :active state?

I think it would be worthwhile to gather all of these edge cases together to get some clearly defined answers on what UX thinks the expected behaviors should be.

Comment 52

[Diego Marcos [:dmarcos]]

The spec should be iOS / Android. They behave consistently with each other in the fundamental touch interactions. It's not in our interest to reinvent those established patterns. If firefox OS behaves differently on those basic interactions we can alienate new smartphones users jumping into our wagon and we would make harder for existing android/ios users to switch.

Comment 53

[Tiffanie Shakespeare [:tif]]

See related Bug 1009684.

Comment 54

[Tiffanie Shakespeare [:tif]]

I don't think anything further is needed from me specifically? Please re-NI me if that isn't the case.

Comment 55

[Kartikaya Gupta (email:[e-mail])]

Never got the promised UX spec, and now it's too late.