522035 - Investigate alternate behaviors for Aero Peek

Status: NEW

(Firefox :: General, defect)

Description

[Benjamin Smedberg]

I can't switch to some Firefox windows from the new aero peek taskbar menu. It gives you a list of recent tabs to choose from, in some order I haven't figured out (maybe most-recently-used?). If you only have a few tabs, I guess this is ok, but I typically have 20+ tabs grouped into multiple windows, e.g.

* a window for e10s stuff
* a window for general stuff and mail
* a window for reviews
* a window for my current coding task

The new context menu works really poorly for me now: I just want to switch to my reviews window, which now doesn't even appear on the list because I haven't used it recently. I'd like to request that there be an entry for each of the currently open windows, and then also, if there's room remaining, entries for specific tabs within the windows. That way switching to a window is at least possible.

Comment 1

[Rob Arnold [:robarnold]]

For Aero Peek, there is a soft cap of 20 previews (pref is browser.taskbar.previews.max) that we intend to tune. There is no notion of recent windows. We have limited control over the ordering but the tabs will appear in the correct ordering (that is, left to right) and the actual positions are determined by the time that we made the preview. This means that the tabs are grouped at startup by window, but as you open more windows/tabs, they will be grouped by time.

That said, we can make a "tab" preview for each window. The drawing code is essentially already written in PreviewController.drawPreview and the behavior won't be too difficult to hook up.

Comment 2

[u88484]

I was about to file a bug for this.  I think the previews should always show most of the windows that are opened.  This is going to be hard for me to explain but I'll give it a shot.  I'll use a soft cap of 5 previews for this example.

Here is how I believe this should be:
One window and five tabs -> five previews, one of each tab
Two windows, first window focused with four tabs -> all four of focused windows tabs should have previews...fifth preview is of the active tab in the second window
Three windows, first window focused with four tabs -> three of the focused windows tabs should have previews...forth and fifth preview is of the active tabs in the second and third windows.

So basically.  All windows should have a preview of their active tabs in the task bar previews.  The focused or last focused window's tabs should fill up as many previews minus the number of opened windows until the five previews are filled.

Comment 3

[Rob Arnold [:robarnold]]

I completely forgot about this because I have it turned off. Before we go around changing our UI, there is a setting in the Windows taskbar which does not group windows into a single button so you can select individual windows from the taskbar if you disable that. Note that you will still get the individual previews above the taskbar.

Comment 4

[Benjamin Smedberg]

That sounds like a bad tradeoff: why would I want a separate taskbar item for each Firefox window just to solve this bug?

Comment 5

[Jim Mathies [:jimm]]

I'd suggest the following changes to our default setting:

1) when there is just one window open, display as we do now, except, the foreground tab is displayed as an application preview, rather than a tab preview. 

2) when we have multiple windows with fewer than say 15 tabs total, all tabs are displayed. The foreground tab for a window is displayed as an application preview, background tabs are displayed as tab previews. Application previews should be listed first in the list.

3) when we have multiple windows with more than 15 tabs, we apply rule #2 and fill the end of the list with as many recent tabs as we can. 

we could also provide other scenarios based on a display pref, depending on what people request.

Comment 6

[Rob Arnold [:robarnold]]

(In reply to comment #5)
> I'd suggest the following changes to our default setting:
> 
> 1) when there is just one window open, display as we do now, except, the
> foreground tab is displayed as an application preview, rather than a tab
> preview.

So rather than show the content of the tab in the thumbnail, we show the contents of the window? That seems less useful to me. Otherwise this should be the same for the live preview.

> 2) when we have multiple windows with fewer than say 15 tabs total, all tabs
> are displayed. The foreground tab for a window is displayed as an application
> preview, background tabs are displayed as tab previews. Application previews
> should be listed first in the list.

This breaks up the contents of a window in a way that greatly disturbs the workflow of users who group related items spatially (commonly by window).

> 
> 3) when we have multiple windows with more than 15 tabs, we apply rule #2 and
> fill the end of the list with as many recent tabs as we can.

What do you mean by "end of the list"? We don't have a supported way to accurately know how many thumbnails will fit on the screen.
 
> we could also provide other scenarios based on a display pref, depending on
> what people request.

Yes, this may be the best option. I am not sure there is a single behavior that works well for all users.

Comment 7

[Jim Mathies [:jimm]]

What I think we need to solve here is the window identification problem. See attached screen cap. This has two windows, each with a foreground tab, both minimized on the tray. The arrows show the two tabs that are foreground. I was suggestion that we render these two differently. Whether that's by rendering their chrome (indicating, 'this is a window') or by applying some other treatment, like an icon of some sort, or a hue color, or whatever.

Moving these two to the left side seems fine to me. we break the ordering, but users would become accustom to seeing all the base windows on the left, and remaining tabs on the right. Although in test here, I've noticed windows give preference to windows, so they don't fall off the deck, so one of my main concerns (losing windows) can't happen.

Comment 8

[Jim Mathies [:jimm]]

Attachment: two windows

Comment 9

[Marco Bonardo [:mak]]

(In reply to comment #7)
> Moving these two to the left side seems fine to me. we break the ordering, but
> users would become accustom to seeing all the base windows on the left, and
> remaining tabs on the right.

Personally, since i use this feature from some time, i'd find it extremely confusing to mix tabs of a window with tabs of another window, the same way i find confusing power switches in my computer's room (they are mixed up) and i keep switching on the wrong light. I guess this is what will happen mixing up previews. The current ordering is awesome, it's just hard to see where a window starts and ends, a grouping border would be the best bet imo.

Comment 10

[(not currently active) Ted Mielczarek]

Personally I miss having the ability to simply click on the Firefox icon in the taskbar to bring my Firefox window back to the foreground, with whatever tab was current still visible. Right now clicking on the taskbar icon just toggles the previews, but so does hovering the icon. Is that something we can control, or is this just a crummy design decision in Windows 7?

Comment 11

[James Ross]

Ted, control-click will activate the app. Luckily apps can't change it AFAIK.

Comment 12

[Jim Mathies [:jimm]]

Alex, can we get some ui feedback on this? Generally people seem to like the way tabs are ordered (they should match the same order as the tabs in the browser). When you have two windows open, the first window and it's tabs come first in the list, and the second and subsequent windows follow. The idea here is to create some sort of visual indicator for the first tab in each window. (See attached screen shot for why this is an issue.) I was thinking some sort of window icon drawn in the corner of the tab might suffice.

Comment 13

[Alex Faaborg [:faaborg] (Firefox UX)]

Having a little bit of extra padding separating the tabs between two windows and introducing an etch on the glass should work well.  If we tag the first tab of each window we sort of indicate those particular tabs are special where really we just want to represent that this set is comprised of two (or more) groups.

Comment 14

[Rob Arnold [:robarnold]]

(In reply to comment #13)
> Having a little bit of extra padding separating the tabs between two windows

We don't have control over this.

> and introducing an etch on the glass should work well.

Is this really visible on most previews?

> If we tag the first tab
> of each window we sort of indicate those particular tabs are special where
> really we just want to represent that this set is comprised of two (or more)
> groups.

I'm not sure how we could mark the tabs as a group. We could perhaps draw a solid 1px border around each preview where the color of the border indicates the window grouping.

Comment 15

[Alex Faaborg [:faaborg] (Firefox UX)]

Attachment: Mockup of two windows in aero peak

Here is a proposal for visually grouping two sets of tabs

Comment 16

[Jim Mathies [:jimm]]

(In reply to comment #15)
> Created an attachment (id=439448) [details]
> Mockup of two windows in aero peak
> 
> Here is a proposal for visually grouping two sets of tabs

That's perfect but unfortunately we don't have that kind of control yet over the taskbar preview display. The apis are still pretty basic - 

http://msdn.microsoft.com/en-us/library/dd562040(v=VS.85).aspx

So we need to work with the previews themselves, and come up with some sort of treatment there.

Comment 17

[Alex Faaborg [:faaborg] (Firefox UX)]

Ok, so what area can we draw to?

Comment 18

[Rob Arnold [:robarnold]]

(In reply to comment #17)
> Ok, so what area can we draw to?

We have full control over the thumbnail's aspect ratio and contents (it is expected to be a rectangle though I think) but not the padding or presence of the close button. Essentially we get a 200x120 area to draw however we please and that's where the thumbnail contents come from.

Comment 19

[Alex Faaborg [:faaborg] (Firefox UX)]

Ok, so theoretically could the first thumbnail in the second window have a wider aspect ratio, a normal sized thumbnail, and an etch and transparent area on the left side? :)

Comment 20

[Marco Bonardo [:mak]]

Attachment: Example of different sized thumbs

what about having the first tab in the window bigger, and other tabs smaller? The remaining space above the thumb could be used for some colored indication (otherwise a colored glow that is transparent for the first window, colored for next ones, so color is the grouping, but the glow blends with aero color)

Still, when the graphical representation changes to the textual one (after 13 or 14 thumbs), which kind of control do we have? recognizing windows is hard there too

Comment 23

[Jim Mathies [:jimm]]

Attachment: New Support for Aero Peek, like Windows Media, mockup

(from bug 566058)

Comment 24

[:Felipe Gomes (needinfo for replies!)]

Inspired by this latest mockup I did a PoC for something similar: all windows would start in window preview mode, with a button below the previews that when clicked would switch it to tab preview mode. This way, clicking on the thumbnail would just immediately switch it to the window, and clicking on the button shows all the tabs.

Here is a clip of it in action:
http://flic.kr/p/85sLcZ

Comment 25

[Marco Bonardo [:mak]]

Felipe, I love that idea, so we could have 3 modes: windows, tabs, or hierarchic-tabs. But iirc the buttons can just be media buttons, or we can personalize the glyphs? How does textual only mode work?

Comment 26

[Marco Bonardo [:mak]]

hm, nvm, I've seen the icon is customizeable, awesome.

Comment 27

[:Felipe Gomes (needinfo for replies!)]

Yeah, we can set the icons and tooltips, but not change their size or format.

If there are too many tabs such that it'd require the text list, Windows doesn't make a smooth transition like this one, which is disappointing. It just hides the preview panel and it will be on textual mode the next the taskbar icon is clicked.

Comment 28

[(not currently active) Ted Mielczarek]

Felipe: with your PoC, does clicking on the window preview just take you right to the active tab in that window? Because that's exactly the behavior I would want.

Comment 29

[tymerkaev]

Attachment: New Support for Aero Peek, like WMP12, mockup with tab previews in action

Here is mockup that show you what I mean.

Comment 30

[tymerkaev]

Attachment: New Support for Aero Peek, like WMP12, mockup with tab previews in action (small)

See previous comment

Comment 31

[Marco Bonardo [:mak]]

(In reply to comment #28)
> Felipe: with your PoC, does clicking on the window preview just take you right
> to the active tab in that window?

You can still CTRL+click on the taskbar icon to get the same behavior, this is an OS level shortcut. Btw from what I got Felipe is switching just modes, thus when you're in window preview mode it will go to the active tab.

Comment 32

[Marco Bonardo [:mak]]

s/you can still/you can already/

Comment 33

[:Felipe Gomes (needinfo for replies!)]

Yeah Mak is right, when we're in window preview mode (not expanding it to tab previews), clicking on the preview just takes that window back to front, without any tab switching, thus showing the active tab on that window.

Comment 34

[:Felipe Gomes (needinfo for replies!)]

Attachment: PoC patch v1

I'm posting the current PoC as a patch so anyone who wants to try using FF with this behavior can apply the patch and rebuild browser/components/wintaskbar.

The patch doesn't yet preserve the tab ordering correctly in all situations, but should work on some. At the very least the tabs from the same window should remain grouped together.

After you expand a window to tab previews, the tabs will be regrouped in a single window after 30 seconds. The rationale is that the user would have time to switch to various tabs if he's searching for something, and it's regrouped together later to go back into the quick "switch to window" behavior.

It also moves the threshold of disabling tab previews from 20 global previews to 10 previews/window.

Comment 35

[Jim Mathies [:jimm]]

Attachment: identifying windows

A slight variation on the window segmenting we originally wanted. Basically, paint the chrome for the foreground tab, and just the content for background tabs. This also helps organizes tab previews into window groups.

Thoughts?

Comment 36

[Jim Mathies [:jimm]]

Attachment: add chrome to first tab's thumbnail patch

Simplified down, this paints the chrome on the first tab so that windows are segregated in the previews panel. This doesn't apply any special treatment to active tabs.

Comment 37

[Jim Mathies [:jimm]]

Attachment: example of 495489 changes

Comment 38

[Alex Faaborg [:faaborg] (Firefox UX)]

Comment on attachment 495490 [details]
example of 495489 changes

pretty subtle, but it is native to include chrome in these previews.  I fine with us making this change, but overall I'm still in favor of the approach proposed in bug 605338

Comment 39

[Jim Mathies [:jimm]]

(In reply to comment #38)
> Comment on attachment 495490 [details]
> example of 495489 changes
> 
> pretty subtle, but it is native to include chrome in these previews.  I fine
> with us making this change, but overall I'm still in favor of the approach
> proposed in bug 605338

These and some additional changes I'm working on (see felipe's poc for an example) are independent of whether we turn it off by default or not. I'm trying to improve this for people who enable the feature.

Comment 40

[Jim Mathies [:jimm]]

Attachment: poc variation

This is a variation on felipe's original work. Basically, after reaching a certain number of previews, only one window is displayed completely at any time. The rest are compacted down into a single window preview. You can expand each group using a button under the window preview.

This still feels a little clunky, because:

1) you can't place buttons under tab previews, giving the user the option of expanding and contracting tab sets.
2) we have no control over the order window previews. windows manages this completely. so window order can get mixed up.
3) it adds complexity to user action when interacting with coalesced tab sets.
4) the buttons are hard to interact with. Hovering over previews to get to the buttons can bring up live previews, which can be distracting.

Comment 41

[Robert Strong (they/them - no direct email)]

Shell Integration is for default browser code so moving to general.