Closed Bug 1246706 Opened 8 years ago Closed 7 years ago

Allow listening for mouse events on tabs

Categories

(WebExtensions :: Frontend, defect)

Product:
WebExtensions
Component:
Frontend
Version:
44 Branch
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox57 ?)

Status:
RESOLVED WONTFIX
Tracking Flags:
Tracking Status
firefox57 --- ?

People

(Reporter: blask, Unassigned)

References

(Blocks 3 open bugs)

Details

(Whiteboard: [design-decision-denied]) 

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:44.0) Gecko/20100101 Firefox/44.0
Build ID: 20160125133541

Steps to reproduce:

In my extension https://addons.mozilla.org/en-US/firefox/addon/tab-deque/ I have the ability to bring the currently selected tab to the background by clicking on it(see https://github.com/sblask/firefox-tab-deque/blob/1b216da838e586e2653306410b4fcffb74887372/content/tabdeque.jsm#L178). I am sure I have seen other extensions that also add some special behaviour when clicking a tab.


Actual results:

With e10s and WebExtensions coming it seems only a matter of time until listening for mouse events will stop working the way it does now and there is no replacement in sight. The closest thing is https://developer.chrome.com/extensions/tabs#event-onActivated, but it doesn't work for my use case.


Expected results:

It would be great to have the ability to listen for mouse events on tabs through a WebExtensions API.
Component: Untriaged → WebExtensions
Product: Firefox → Toolkit
My addon "Multiple Tab Handler" also requires this feature.
https://addons.mozilla.org/firefox/addon/multiple-tab-handler/
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: (webext-tabs) Allow listening for mouse events on tabs → Allow listening for mouse events on tabs
Whiteboard: [design-decision-needed] triaged
Component: WebExtensions: Untriaged → WebExtensions: Frontend
Priority: -- → P3
To be discussed during Nov. 15, 2016 web extensions triage meeting.
I think we said in the Nov 15th meeting that this was something we won't fix. According to the meeting notes it said "this is something we will likely do in settings". Kris can you comment on this please so we can justify the won't fix?
Flags: needinfo?(kmaglione+bmo)
My Vertical Tabs Reloaded add-on will also need this.
https://github.com/Croydon/vertical-tabs-reloaded
Blocks: 1311472
My add-on, https://github.com/mthamil/Tab-Wheel-Scroll, requires listening for mouse scroll events on the tab bar.
A few observations:

 - it looks like several different use-cases could be covered with this API
 - I don't see any obvious disadvantages to implementing this (wrt encapsulation or performance)
 - we could rate-limit the MouseMove events if needed (though I doubt even that would be a problem)
 - no new low-level details need to be exposed to make this work (we already have a Tab abstraction)

In light of this, I would like to ask for re-examining the decision on this bug, or at least make it P5 "will take patches", as it seems several existing extension developers (who could work on this) are interested in the API.
Priority: P3 → --
Whiteboard: [design-decision-needed] triaged → [design-decision-needed]
(In reply to matthamilton from comment #7)
> My add-on, https://github.com/mthamil/Tab-Wheel-Scroll, requires listening
> for mouse scroll events on the tab bar.

And this extension is necessary for Firefox to match the tab-bar behaviour common to every other application on a Linux desktop.

On Linux, I can personally attest that spinning the scroll-wheel on a tab bar switches tab focus in:

* anything using GTK+'s GtkNotebook
* anything using Qt's QTabWidget (which should include Opera, as I remember)
* anything using wxWidgets' wxNotebook (because it wraps GTK+ on Linux)
* Chrome/Chromium

While I don't have any Java GUIs handy at the moment, that should also mean that anything Java-based which uses SWT (eg. Eclipse, Vuze) will behave that way, since SWT works as a toolkit abstraction layer similar to wxWidgets and wraps GTK+ on Linux.

Firefox is a glaring exception to platform norms in this way... especially given how much its interaction paradigm encourages switching tabs.
Mouseover is useful for displaying a tab preview on tab hover.
Blocks: 1333837
An attempt to port FireGestures could also use an ability like this, but extended to the full chrome/window, not just tab elements. I noticed this through a recent discussion here: https://www.reddit.com/r/firefox/comments/642oax/i_created_a_mouse_gestures_web_extension_as_a/

Without the ability to detect mouse events except through a content script, it's impossible to detect gestures while a loading tab has stalled, or on restricted pages. This is a sub-par UX for gestures that allow opening/closing tabs, for instance.

Would it not be possible to just allow background scripts to listen for mouse/touch events on all windows, just with drastically reduced information/abilities returned on secure pages and chrome elements? As in something like this:

  // in a background.js
  window.addEventListener("mousedown/touchstart/etc", function(evt) {
    evt.target === undefined
    evt.targetType === "content-element", "chrome-element", chrome-tab", "chrome-devtools", etc, as desired by the user-agent.
    evt.tabId, windowId, and frameId are set so that addon can distinguish them.
    evt.windowId is instead set when a non-tab chrome element is being interacted-with.
    evt.pageX/pageY/offsetX/offsetY/etc could be undefined (unless desired for some reason).
    evt.movementX/Y would at least be available for gesture-tracking purposes.
    evt.preventDefault/stopPropagation could be ignored except where desirable (ie, on tabs).
  });

There would not need to be any guarantees about which targetTypes (or how fine-grained they are) each user-agent would expose. Nor could there be, as users may already hide UI elements in most user-agents. But this would still allow consistent gesture-tracking and basic chrome-interactions to be detected without breaking restrictions. User agents could also hide more details behind permissions, as desired.

I believe this would satisfy a significant number of use-cases like the ones presented in this bug, as well as for addons like FireGestures (I'm just not sure if pointer-locking would also need to be exposed for gestures). Addons could coordinate these listeners with the ones seen by content scripts, so there would be no need to have a mousemove listener in every tab's content script, so there may be a performance benefit for some use-cases as well.

One consideration that may prove tricky is that if extensions would be permitted to "replace" UI elements, they should probably be allowed to specify a targetType of their own (and potentially specify a tabId/etc as needed somehow).
My https://addons.mozilla.org/en-US/firefox/addon/autoclose-tabs/ add-on needs to listen to double click events on tabs.
Hi all, we're going to re-open this and discuss it at the May 23 WebExtension APIs triage meeting. Sblask, Piro, Croydon, matthamilton, Stephan, Tim, and Florian -- would you be able to attend the meeting? 

Wiki: https://wiki.mozilla.org/Add-ons/Contribute/Triage#Details_.26_How_to_Join

Agenda: https://docs.google.com/document/d/1-j08Zo4sbwAuRZndNNtdIRlDM8TblddDA-PAeRgYMWU/edit#
Yes, I believe so.
Flags: needinfo?(amckay)
WebExtensions are about providing interaction with web content and less focusing on the browser chrome.

For example: being able to scroll up and down, using a mouse on the tab strip is already something in Firefox and capturing those events could potentially break it. There were also questions in the meeting about how this would work once WebExtensions are in their own process.

Whilst this bug is specifically about tabs, its about allowing specific events on the user interface to be captured and that's something that we've been avoiding as much as possible because of extensions breaking the browser chrome.
Status: NEW → RESOLVED
Closed: 7 years ago
Flags: needinfo?(amckay)
Resolution: --- → WONTFIX
Whiteboard: [design-decision-needed] → [design-decision-denied]
In that case, would you mind putting bug 1285812's priority up for reconsideration?

Firefox's default behaviour is a serious UX wart on Linux desktops, where all major widget toolkits (as well as Chrome/Chromium) map "scroll wheel on a tab bar" to "change tab focus".

(In fact, it's a serious enough break from my muscle memory that, as a backup plan, I've started speccing out a utility which functions sort of like a keylogger under the hood, but instead hijacks scroll events on the top 37px of any window with WM_CLASS = ("Navigator", "Firefox") and replaces them with synthesized Ctrl[+Shift]+Tab X11 events.)
It's funny because Jared said, just 10 months ago in that bug:
> At this point, it would be functionality better left for an add-on, allowing users to opt-in to the changed behavior that they want.

https://bugzilla.mozilla.org/show_bug.cgi?id=1285812#c2

That is going to be impossible.
(In reply to Andy McKay [:andym] from comment #16)
> WebExtensions are about providing interaction with web content and less
> focusing on the browser chrome.
> 
> For example: being able to scroll up and down, using a mouse on the tab
> strip is already something in Firefox and capturing those events could
> potentially break it. There were also questions in the meeting about how
> this would work once WebExtensions are in their own process.
> 
> Whilst this bug is specifically about tabs, its about allowing specific
> events on the user interface to be captured and that's something that we've
> been avoiding as much as possible because of extensions breaking the browser
> chrome.

How about mouseover (for tab preview on mouse over) or doubleclick events ?

I don't see how those 2 are affected by any of the 2 problems you've mentioned.
Flags: needinfo?(amckay)
(In reply to Andy McKay [:andym] from comment #16)

> Whilst this bug is specifically about tabs, its about allowing specific
> events on the user interface to be captured

Why would the event need to be captured? If my add-on listens to double click events, I don't mind if some existing Firefox code also receives the event...
(In reply to Andy McKay [:andym] from comment #16)
> WebExtensions are about providing interaction with web content and less
> focusing on the browser chrome.

Maybe for you, but there are a lot of people out there who would not agree and just want to get the same functionality they have today provided through extensions. And they won't mind if the browser doesn't work the default way, because that's the whole point of getting an add-on

> For example: being able to scroll up and down, using a mouse on the tab
> strip is already something in Firefox and capturing those events could
> potentially break it. There were also questions in the meeting about how
> this would work once WebExtensions are in their own process.

If I want to, I can write an add-on today that breaks all websites by injecting some content script. And that's OK, because users have the choice and have to give the permission. Nobody would think about taking this possibility away?
I don't see how a different processing model would change anything. Firefox allows context menus on tabs, how is a click there different from a left click on a tab?

> Whilst this bug is specifically about tabs, its about allowing specific
> events on the user interface to be captured and that's something that we've
> been avoiding as much as possible because of extensions breaking the browser
> chrome.

The problem is: Firefox today is broken. It is not providing the functionality people want, that's why they write add-ons. If everything that I want would be built-in, I would not need to spend my evenings trying to fix my experience.
> How about mouseover (for tab preview on mouse over) or doubleclick events ?

We don't have any other events like this in WebExtensions. We've been repeatedly pointing people to building their own tab strip in HTML as a sidebar where they can do whatever they want.

> The problem is: Firefox today is broken. It is not providing the functionality people want, that's why 
> they write add-ons. If everything that I want would be built-in, I would not need to spend my evenings 
> trying to fix my experience.

And I get it. But this is the fundamental difference between what people used to do with add-ons and what WebExtensions are for. The goal of WebExtensions is different from what it used to be for add-ons. Every design decision in Firefox's history and every controversial thread in a mailing list that resolved in "just write an add-on" is part of the problem people have with WebExtensions.

You can do whatever you want on Nightly and "fix" Nightly as you want, because you can load bootstrapped add-ons there.
Flags: needinfo?(amckay)
(In reply to Andy McKay [:andym] from comment #22)
> You can do whatever you want on Nightly and "fix" Nightly as you want,
> because you can load bootstrapped add-ons there.

How stable is nightly, generally?

Aside from the expected breakages from writing extensions against APIs now considered internal, would I be setting myself up for significantly more pain if I migrated my mother over to it?
(In reply to Andy McKay [:andym] from comment #22)

>We've been repeatedly pointing people to building their own tab strip in HTML as a sidebar where they can do whatever they want.

And what if there are two add-ons wanting to do different things? Do I need two tab bars, three,... ? Also, I wouldn't want to change how Firefox looks like. It's hard enough already to get a consistently looking UI as it is.

> And I get it. But this is the fundamental difference between what people
> used to do with add-ons and what WebExtensions are for. The goal of
> WebExtensions is different from what it used to be for add-ons. Every design
> decision in Firefox's history and every controversial thread in a mailing
> list that resolved in "just write an add-on" is part of the problem people
> have with WebExtensions.

When I read https://developer.mozilla.org/en-US/Add-ons/WebExtensions/What_are_WebExtensions it sounds very different though. Especially the "add-ons can add new features to the browser" part. And I get that you want to provide an API so add-ons don't break with regular updates because they access internals. But just saying we won't do it because webextensions are not meant to do it, where the existing documentation or communication doesn't say so is just bad in my opinion. Or can you point me to some other place where it's spelled out what webextensions are for?

> You can do whatever you want on Nightly and "fix" Nightly as you want,
> because you can load bootstrapped add-ons there.

The nightly is not stable enough though.
bug 1367187 isn’t a duplicate. it’s about the tab bar, not individual tabs

and yeah, i’m going to jump on the bandwagon and say:

i have a very simple addon that just reacts to a click event on some UI element (in my case the tab bar, for this bug report the tabs)

this should be possible with WebExtensions, else they will never be able to replace other addons.
The reasoning is the same, this outside of the WebExtensions scope.
Blocks: 1226546
status-firefox57: --- → ?
(In reply to Andy Mercer from comment #33)
> (In reply to Andy McKay [:andym] from comment #27)
> > The reasoning is the same, this outside of the WebExtensions scope.
> 
> You are incorrect. The scope of WebExtensions is to replace the current
> Addon system. In fact, Mozilla has said multiple times that the goal is to
> make sure that all of the top 10 addons are able to be ported. Tab Mix Plus
> depends on this.

I'm sorry you misunderstand the goal of WebExtensions. There's some more documents on this [1] and if you want to chat about this more, I suggest you try the mailing list [2].

[1] https://wiki.mozilla.org/WebExtensions/Vision, https://wiki.mozilla.org/WebExtensions/FAQ#Will_I_be_able_to_do_everything_I_can_in_a_legacy_technology.3F 
[2] https://mail.mozilla.org/listinfo/dev-addons
Hi Martijn, please keep it civil. Bugzilla is an issue tracker, not an advocacy forum, and civility and professionalism matter. 

Please take a few minutes to review the Bugzilla Etiquette guide and Mozilla's Community Participation Guidelines before making your next comment. 

Bugzilla Etiquette: https://bugzilla.mozilla.org/page.cgi?id=etiquette.html 

Community Participation Guidelines: https://www.mozilla.org/about/governance/policies/participation/
(In reply to gh from comment #37)
> Another collector contradiction from the link:

Those parts of the wiki are from 2015 and written by others. I updated them and put text at the top saying: "Some of these are relevant, but come from an FAQ around the decision to implement WebExtensions in August of 2015. They may not still be relevant." rather than just delete them in the effort of transparency. Perhaps we should delete them to prevent confusion.
Hi folks, comments on this bug will now be restricted. As mentioned in comment 39, Bugzilla is an issue tracker and not an advocacy forum. At this point, this bug has stopped providing useful information and has become an argument. 

If you would like to continue the conversation, please see comment 35 for mailing list information.
Restrict Comments: true
Bug is closed, so removing year-old NI for kmag.
Flags: needinfo?(kmaglione+bmo)
Product: Toolkit → WebExtensions
You need to log in before you can comment on or make changes to this bug.