1286387 - [meta] Add webextensions API(s) for reader mode

Status: NEW

(WebExtensions :: General, task, P5)

Description

[Andrew Swan [:aswan]]

Reader View is a useful feature: https://support.mozilla.org/en-US/kb/firefox-reader-view-clutter-free-web-pages

There are probably some interesting things we could do with it from webextensions.  Programmatically putting individual tabs in or out of reader mode is one option, getting at the de-cluttered content would be another.

Comment 1

[Shane Caraveo (:mixedpuppy)]

Pocket adds its button to reader mode, and to continue supporting that I had updated the reader mode to support adding arbitrary buttons from addons in bug 1215694 attachment 8702687 [details] [diff] [review].  The one api used directly from ReaderMode.jsm is getOriginalUrl, but that may not be necessary to support since the Reader:Clicked-* message passes an article object which has the url.

Other than that, I'm not sure what use cases there are for extending reader mode.

Comment 2

[Shane Caraveo (:mixedpuppy)]

OTOH, I'm not certain adding a button to reader mode has any benefit over having a page or browser action button.  The effort I mentioned was to keep parity of features when migrating pocket to an addon.

Comment 3

[Kris Maglione [:kmag]]

Assuming that we want to be able to Pocket reader mode URLs, we'll need to do something to support this, if only allowing Pocket to access the original URL. I don't know whether or not adding a separate button to reader mode is actually necessary, though.

Comment 4

[Shane Caraveo (:mixedpuppy)]

I think that any addon should be able to get at the original url for a reader mode tab, but probably easier said than done.

Comment 5

[Niklas Gollenstede]

> Programmatically putting individual tabs in or out of reader mode is one option, getting at the de-cluttered content would be another.

In a SDK add-on I am porting I was able to do exactly that. And adding CSS to the reader view can also be useful. I would really like to do those things in the WebExtension too.

So I have downloaded, patched and build the Firefox sources.

My patches are pretty simple:
1. Allow '<all_urls>' to match 'about:reader' URLs. (1) This is enough to allow `tabs.executeScript()` to run, and should work for `"content_scripts"` and CSS too.
2. Allow extensions to navigate to 'about:reader...' URLs.

> I think that any addon should be able to get at the original url for a reader mode tab

The URL returned by for example `tabs.query({ })` already is 'about:reader?url='+ encodeURIComponent(originalUrl).

Are these patches acceptable? Which tests would I need to run/add/pass to get this reviewed, and how do I do that (both the testing and proposing for review)?

(1) I don't think more fine gained URL matching is conceptually possible with match patterns, and also not necessary. I can't come up with a use case where an extension only wants access to a some 'about:reader' URLs. If an extension wants to argument the reader mode, it will want to do so for all URLs, and must specify the '<all_urls>' permission. If an extension wants to work with a specific site, I don't think the reader mode is of any interest.
One more useful thing would be a form of match pattern that matches only but all 'about:reader' URLs, to attach content scripts via the manifest.json.

Comment 6

[Shane Caraveo (:mixedpuppy)]

(In reply to Niklas Gollenstede from comment #5)
> My patches are pretty simple:
> 1. Allow '<all_urls>' to match 'about:reader' URLs. (1) This is enough to
> allow `tabs.executeScript()` to run, and should work for `"content_scripts"`
> and CSS too.

I don't think we'd allow content scripts to get attached to any about urls.

> 2. Allow extensions to navigate to 'about:reader...' URLs.

I think we could allow this somehow, but I'd like clarity on what you mean by "navigate to".

Comment 7

[fkohrt]

(In reply to Shane Caraveo (:mixedpuppy) from comment #6)
> (In reply to Niklas Gollenstede from comment #5)
> > 2. Allow extensions to navigate to 'about:reader...' URLs.
> 
> I think we could allow this somehow, but I'd like clarity on what you mean
> by "navigate to".

I do have a use case since I try to achieve just that: My SDK add-on changes the active tab's URL to the reader view version by setting tabs.activeTab.url. The active tab then loads the reader view version of the previously loaded site. With WebExtensions instead it obviously doesn't work to just update some tab's url to about:reader with tabs.update(). That's what I would need.
Not sure though if Niklas Gollenstede thinks similar about "navigating to".

Comment 8

[Niklas Gollenstede]

> I don't think we'd allow content scripts to get attached to any about urls.

I can't see any security or usability concerns with doing it.
As far as I know, `about:reader` holds normal HTML pages without any special APIs or permissions (unless it's origin is a problem with things like XHRs?), and the potential of breaking things isn't any higher than with any other page (and it is not a crucial part of the UI).
If this is a "dogmatic" thing like "https?:, ftp:, ... is scriptable, about: not" then there already are exceptions where it makes sense.
If access to AMO is not allowed because it has elevated rights and scripts could harm usability, why not make an exception the other way where this is not the case and it would benefit Add-ons and their users?

> > I'd like clarity on what you mean by "navigate to".

> update some tab's url to about:reader with tabs.update().

That and `tabs.create()`. Booth check if an URL is "save to navigate to". I added an exception to that check.

Comment 9

[cat_in_136]

I did try to create an addon for reader mode.

https://bitbucket.org/cat_in_136/vertical_reader_mode/

This is an Addon-SDK-based addon.
But I gave up to convert it to an webextension.
So I could not post to AMO. Thank you.

Comment 10

[Oct4v14n]

I tried create a extension that adds automatic language translations to (user-)selected words.
As I use the reader mode quite often, supporting my extension in this mode was the main priority for me.

However adding
"permissions": ["about:reader?url=*://*/*"]
to my code results in errors ("about:reader?url=*" also delivers the same error).
While I understand that extensions should not alter about:* pages I think the reader and possible the pdf-reader should be separated from this rule.

PS: If I'm missing a way to access the selected text and dom of the reader please feel free to point me to the right methods :)

Comment 11

[Sergio]

I'm not sure if this is the right place to ask, so probably it's not, but i need a clear answer here, are we going to be able to interact with about: like urls?

I've a highlighter addon, and it works fine on reader mod i'm using the sdk and a bit of xul but an sdk only addon can access reader mode just fine, so can web-extensions work on about:reader urls?

PS. probably about:reader is the only important about like url that i need to access, but who knows, maybe some users want it to use it to find a certain setting on preferences but not being able to interact with other about pages don't seem such a burden as about:reader.

Comment 12

[dw-dev]

I am the developer of a WebExtensions add-on called Save Page WE.

Save Page WE is a highly accurate, very fast add-on for saving a web page in a single HTML file, with all resources included in the file as data URI's.

Save Page WE is designed as a replacement for UnMHT and MAF.  It also runs on both Firefox and Chrome.

As the numbers of users increases, I am getting questions asking why it is not possible to save Reader pages with Save Page WE.  Of course, the answer is that the content script will not run in an 'about:reader' page.

It seems to me that saving an 'about:reader' page is a very reasonable thing to want to do.

I am sure it would help many users if Save Page WE could save Reader pages, and I think this will be increasingly so in the future.

Comment 13

[Jonathan Kingston [:jkt] he/him]

Assuming Bug 1371786 is ok with :pauljt's team then this likely would be interesting for Shield studies on smarter Reader modes etc. Adding :groovecoder to the cc list as he likely will be interested.

Comment 14

[Jens Bannmann]

Another use case not explicitly mentioned above is allowing users to define URL patterns for which to use reader view as the default, like implemented by this legacy extension: https://addons.mozilla.org/en-US/firefox/addon/automatic-reader-view/

Comment 15

[Bob Silverberg [:bsilverberg]]

(In reply to Jens Bannmann from comment #14)
> Another use case not explicitly mentioned above is allowing users to define
> URL patterns for which to use reader view as the default, like implemented
> by this legacy extension:
> https://addons.mozilla.org/en-US/firefox/addon/automatic-reader-view/

I believe you can do that yourself in an extension using tabs.toggleReaderMode [1].

[1] https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/tabs/toggleReaderMode

Comment 16

[fkohrt]

Through a complaint [1] by one of my add-on's users I noticed the following:

Ín FF <= 57, while the Firefox GUI only showed the reader mode button if a page qualified for it, it was possible to open it nevertheless by prepending "about:reader?url=" manually or through the API.
With FF >= 58, the API (especially `tabs.toggleReaderMode()`) checks for that itself, making it impossible to circumvent Firefox' own algorithms that determine if a page may be qualified for the reader mode (i.e. "is an article"), leaving only the option to prepend it by hand.

> if (!tab.isInReaderMode && !tab.isArticle) {
>   throw new ExtensionError("The specified tab cannot be placed into reader mode.");
> }
[Excerpt from toggleReaderMode(tabId)]

My add-on solely existed to "force" the reader mode to open because Firefox doesn't always gets it right when determining what an article is.

It would be great if things could change in a way that when clicking the regular reader mode button Firefox checks for the `isArticle` property but when using the API functions it doesn't.

Links
[1] https://addons.mozilla.org/de/firefox/addon/activate-reader-view/reviews/977141/

Comment 17

[Bob Silverberg [:bsilverberg]]

Thanks for the feedback, fkohrt. I'm not sure what, if anything, we can do about that, but in order for it to be investigated can you please open a new bug describing the problem? This bug is a tracking bug and isn't the right place to discuss it.

Comment 18

[fkohrt]

(In reply to Bob Silverberg [:bsilverberg] from comment #17)
> Thanks for the feedback, fkohrt. I'm not sure what, if anything, we can do
> about that, but in order for it to be investigated can you please open a new
> bug describing the problem? This bug is a tracking bug and isn't the right
> place to discuss it.

Thanks for creating the API in the first place and for helping out now!
I opened the new bug: https://bugzil.la/1419520

Comment 19

[fkohrt]

(In reply to fkohrt from comment #16)
> ...
> Ín FF <= 57, while the Firefox GUI only showed the reader mode button if a
> page qualified for it, it was possible to open it nevertheless by prepending
> "about:reader?url=" manually or through the API.
> ...

And, of course, it should say "In FF < 57".

Comment 20

[u621339]

Another use case:

I use Vimium [0] to browse and move in pages. I frequently use Reader Mode, but it forces me to fall back to Space, arrow keys or mouse.

[0] https://addons.mozilla.org/en-US/firefox/addon/vimium-ff/

Comment 21

[Sergio]

As it seems some bugs are being closed cause "there has been very little interest" on them i want to confirm that i'm still interested on this bug.

Comment 22

[u654615]

Allowing extensions in Reader View would be very nice to have. I am a keyboard-only user using Vimium. I use Reader View heavily, and it's really aggravating to go back to the mouse/direction keys.

Comment 23

[plaice.adam+persona]

I am a keyboard-only user using Vimium. I use Reader View heavily, and it's really aggravating to go back to the mouse/direction keys.

To further expand on this (since everybody will (have) be(en) annoyed by a notification anyway...):

In addition to the annoyance of having to switch keyboard shortcuts, the lack of Vimium in reader mode makes some functionality straight-out impossible (or at least far more convoluted). For instance, Vimium has "marks" (which are very similar to Vim's marks or Emacs's position registers) — you can set a "mark" at a certain scroll position (or multiple marks at multiple positions), scroll away somewhere else and then jump back to one of your marks. This is extremely useful when reading a long article and, say wanting to remind yourself of what was written earlier, wanting to quickly cross-reference multiple passages, or referring back to several diagrams or equations that are mentioned later. (For some of these use-cases a well-designed HTML document with appropriate internal/section links would suffice, but even the best-designed document can't really cover all possible combinations, and besides the design of a document is obviously beyond the reader's control.)

BTW, as an aside: is there an "approved" place to collect potential use-cases (ideally in a globally editable wiki-like location) without having to distract everyone following this bug, and similar bugs? The user-voice linked to from here appears to be no longer available.

Comment 24

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee is inactive on Bugzilla, so the assignee is being reset.

Comment 25

[Alexander Ploner]

I'd really like to write a WebExtension that adds text to speech functionality for websites. Support for accessing reader mode would be a requirement for me because of several reasons.

• Getting access to the simplified version/the actual text content works great in reader-mode and I do not have the time to write my own logic that extracts the actual text.
• If a website owner checks for compatibility, he'll do that (maybe) for reader mode but definitely not for my add-on.
• I can add a button to the browser actions but having a button in the reader mode as well seems like a good place for this functionality.
• There might be features like "read from here" that require access as well.

I'd like to have this functionality on desktop and on mobile. At least on mobile there is no read functionality in Firefox and on desktop the quality of readings is (as far as I remember) based on the operating system, which is typically pretty outdated and not comparable to modern text to speech services.

That's my use case. I think others might come up with other use cases as well, ultimately supporting Firefox' potential for its users.

Comment 26

[Paul [pwd]]

Unfortunately I can't remember the bug. But essentially there's two implementations of Reader Mode on Android. One is via GeckoView and the other is via an internal extension. The latter is currently in use and the former has the ability to tie onto the system TTS engine. However as things stand, there's currently no performance benefits to switch and so the decision was made to maintain the status quo. If we could get TTS working natively, that would be a massive win for android users. Particularly those with accessibility needs.