1315616 - Give extensions access to cached favicon URLs

Status: NEW

(WebExtensions :: Compatibility, enhancement, P3)

Description

[Stephan Sokolow]

User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
Build ID: 20161017194958

Steps to reproduce:

Tried to write an extension that uses tab titles, urls, and favicons to produce an analogue to the backup and export options in the bookmark manager.


Actual results:

To resolve tabs.Tab.favIconUrl, I had to grant the addon "<all_urls>" permission and I've spent a whole day (and counting) trying to write suitably non-fragile fetch() code when all I want is already retrieved and staring me in the face to the left of each Tab.title.


Expected results:

Given that favicons serve a similar role to tab titles and have similar privacy and permissions implications, there should be a simple API to retrieve the already-in-memory data using only the "tabs" permission and without worrying about cache misses.

In line with the naming scheme already in place, I propose tabs.Tab.favIconBlob.

Comment 1

[Caitlin Neiman [:caitmuenster]]

To be discussed at January 24 WebExtensions Triage mtg. 

Agenda: https://docs.google.com/document/d/1add-6FL8mzksvzbyB83HUmEkVmKERd-nt740AYr-4PE/edit#

Comment 2

[Kris Maglione [:kmag]]

We talked about this, and decided that we want to support this use case, but that we should do it by giving extensions access to URLs backed by our favicon cache rather than by handing them image blobs.

Comment 3

[Stephan Sokolow]

(In reply to Kris Maglione [:kmag] from comment #2)
> We talked about this, and decided that we want to support this use case, but

That's good to hear.

> that we should do it by giving extensions access to URLs backed by our
> favicon cache rather than by handing them image blobs.

Is the rationale documented anywhere I could read? I'm curious about what makes cache URLs easier to support than blobs in this situation.

Comment 5

[Kris Maglione [:kmag]]

(In reply to Stephan Sokolow from comment #3)
> Is the rationale documented anywhere I could read? I'm curious about what
> makes cache URLs easier to support than blobs in this situation.

In the case of blobs, we would need to fetch, decode, and store them in memory for every tab, and there would be additional overhead involved in displaying them as well. Providing access to favicon cache URLs has better performance characteristics and a simpler implementation, and still allows extensions to fetch blobs if and when they actually need them.

Comment 6

[Stephan Sokolow]

...oh yeah! e10s is a thing now!

I was still in a mindset that it'd be cheaper to just reference the existing decoded blob used to render the tab.

(Because I don't know of a way to display an arewee10syet-style report for just the ~30 extensions I have installed and because I don't want to risk my privacy extensions silently failing, I decided to just disable e10s in my Aurora and wait until I'm forced to migrate onto it... despite my loadout and tab use being pathologically unsuited to single-process operation.)

Comment 7

[Tomislav Jovanovic :zombie]

(In reply to Kris Maglione [:kmag] from comment #2)
> We talked about this, and decided that we want to support this use case, but
> that we should do it by giving extensions access to URLs backed by our
> favicon cache rather than by handing them image blobs.

Was the idea to have a tab.faviconUrl property, or some API that would return a (cached) favicon URI for any domain?  And if it's the latter, does it make sense to make it part of the MozURLUtils WebIDL from bug 1315558 comment 4?

Comment 8

[Kris Maglione [:kmag]]

My thought was that faviconUrl would point to a URL backed by the favicon cache, and then we'd have to give extensions access to those URLs.

Comment 9

[Jonathan Kingston [:jkt] he/him]

Some ideas/issues regarding this:

- Apps that serve different favicons for logged in users won't work for container tabs (EG WhatsApp does this)
- Querying for a future page load rather than an existing tab would permit use cases like:
  https://github.com/mozilla/testpilot-containers/issues/500
- In the near/distant future we might permit different icons like PWA icon, mask-icon
  - perhaps other files are relevant here too like manifest, feature policy etc

Would it make sense to permit an api similar to:

browser.cache.icons({url: "example.com", cookieStoreId: 'blah-12'});

Comment 10

[Blake Winton (:bwinton) (:☕️)]

Relatedly, Whimsy would like the user to be able to choose to show the thumbnails of sites instead of gifs on the overridden newtab page, but we have no way of getting them.  A similar api (or an extension to this api) would be very helpful.

Comment 11

[Tetsuharu OHZEKI [:tetsuharu] (UTC+9)]

Surprisingly, Chromium have a way to access to the favicon cached in a browser through "chrome://favicon/" (e.g. chrome://favicon/https://youtube.com/).
And this is used as implicit public way to access to the favicon from chrome extension

- https://stackoverflow.com/questions/10301636/how-can-i-get-the-bookmark-icon-in-chrome
- https://github.com/yemount/IconicHistory
   - https://github.com/yemount/IconicHistory/blob/c75a1c0/iconic_history.js#L111
   - https://github.com/yemount/IconicHistory/blob/c75a1c0/iconic_history.js#L138

Comment 12

[YF (Yang)]

(In reply to Tetsuharu OHZEKI [:tetsuharu] [UTC+9] from comment #11)
> Surprisingly, Chromium have a way to access to the favicon cached in a
> browser through "chrome://favicon/" (e.g.
> chrome://favicon/https://youtube.com/).
> And this is used as implicit public way to access to the favicon from chrome
> extension
> 
> -
> https://stackoverflow.com/questions/10301636/how-can-i-get-the-bookmark-icon-
> in-chrome
> - https://github.com/yemount/IconicHistory
>    -
> https://github.com/yemount/IconicHistory/blob/c75a1c0/iconic_history.js#L111
>    -
> https://github.com/yemount/IconicHistory/blob/c75a1c0/iconic_history.js#L138

Firefox has page-icon: protocol, for chrome context so far. It may need to be extended to support getting (and policy for privacy), status, and so on.

Comment 13

[Stephan Sokolow]

And you'll want to make sure it's clearly documented as the recommended way to do thing in all of the places people are likely to look.

Even with the StackOverflow answers pointing to it, the approach Chrome took comes across as an internal API that got accidentally exposed to the world and is only allowed to continue to work because of the sheer number of extensions which would break if they fixed their mistake.

Comment 14

[Kestrel]

Lack of favicon access is resulting in extensions like "Bookmarks Manager and Viewer" fetching them from Google which is a privacy concern.

https://addons.mozilla.org/en-US/firefox/addon/bookmarks-manager-and-viewer/

Comment 15

[Mike Hommey [:glandium]]

(In reply to Kestrel from comment #14)
> Lack of favicon access is resulting in extensions like "Bookmarks Manager
> and Viewer" fetching them from Google which is a privacy concern.
> 
> https://addons.mozilla.org/en-US/firefox/addon/bookmarks-manager-and-viewer/

Same with Tree Style Tabs. And that made me remove favicons from the tab-stat addon.

Comment 9 is very true too.

How about making faviconurl return a data: url of the cached icon when the property is read?

Comment 16

[Mike Hommey [:glandium]]

(that would have the side effect of fixing the issue on all addons using faviconurl without fixing every single one of them)

Comment 17

[Geoffrey De Belie (Smile4ever)]

Related: Bookmarks BookmarkTreeNode API should expose favicon URL - https://bugzilla.mozilla.org/show_bug.cgi?id=1411120

Comment 18

[Brian Mock]

Thanks for linking the two bugs, Geoffrey. This is definitely the same problem I was running into in https://bugzilla.mozilla.org/show_bug.cgi?id=1411120

Comment 19

[aafn]

This missing feature is also what forced me to develop code to fetch() the favicon in my extension Bookmark Search Plus 2 https://addons.mozilla.org/fr/firefox/addon/bookmark-search-plus-2/
to avoid the privacy problem mentioned above.

I had to develop a basic parser to retrieve the favicon URL out of each bookmarke page, then fetch the favicon and store it in local storege as an URI using 

FReader.readAsDataURL(blob);

This is still perfectible, but significant work and there are still some cases failing .. while all is already inside the browser database :-( (but inaccessible)

So this is:
- duplicate work
- duplicate storage


The browser.bookmarks API should make the favicon available as part of the BookmarkTreeNode object, and displayable as an image.

Note: ready to help if I can, let me know.

Comment 21

[Brett Zamir]

Hi,

I'd like to see favicons exposed as part of history search items:

https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/history/HistoryItem

https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/history/search

Since it appears per comments on bug 1411120, that this has become a more generic issue in also tackling BookmarkTreeNode, and since the latter is, if previous FF add-on behavior is any indication, closely related to history search, I wonder if my request should be tacked on here as well, or whether I should file a separate issue?

Comment 25

[juraj.masiar]

Any chance making this work in Firefox 64?

Because Firefox 64 will block popular way to get favicon using Google service:
https://plus.google.com/_/favicon?domain_url=https://bugzilla.mozilla.org

Comment 26

[rugk]

1. This way is really a privacy-nightmare… sending potentially all your domains to Google Plus.
2. Why does Firefox 64 block access to those? Any source? Or why do you think, it does?

Comment 27

[juraj.masiar]

I'm sorry I should mention more info. This is the error you get now in 64:

The resource at “https://plus.google.com/_/favicon?domain_url=http://shmu.sk” was blocked because content blocking is enabled and the resource was classified as a slow tracking resource.

Using the "<all_urls>" permission. Adding new permission for the host will fix it as per docs: "bypass tracking protection if the host is a full domain without wildcards".
I couldn't find any release info for version 64.

Comment 28

[ariasuni]

Any news on this? I still have to use a workaround to load the extension page favicon (chrome://mozapps/skin/extensions/extensionGeneric-16.svg).
https://github.com/eoger/tabcenter-redux/blob/699694ff0ebdc0cc06a5c3a1318c6e4519df663e/src/sidebar/tab.js#L175

Comment 30

[ariasuni]

It would be nice to have this to avoid ugly hacks in WebExtensions.

Comment 32

[Reza]

Only 3 years later and still no progress? Can you guys please fix this API as @aafn mentions
"The browser.bookmarks API should make the favicon available as part of the BookmarkTreeNode object, and displayable as an image."

Comment 33

[Sean Feng [:sefeng]]

Does this bug cover both read access and write access?

Comment 34

[aafn]

Hello @Sean Feng, only read access is demanded here.

Comment 35

[u680145]

Was it resolved? I need to access favicons for my extension.

Comment 36

[serradeil]

It would be nice if this could be resolved to allow extension authors use the ressources available.
I have two extensions installed that need to do their own thing to retrieve page icons.

Sync doesn't cover this type of data so on every machine and every FF instance installed these addons have to retrieve favicons allover again and again and again ... that's painful to watch.

Please! Get this sorted.

Comment 39

[clessili]

Are there any plans to allow the page-icon protocol (bug 1354248) on webExtension pages? I wrote an extension to display my bookmarks on new tabs, using the bookmarks API, but currently to show the favicons I request them from a Google server, by doing: <img src="https://s2.googleusercontent.com/s2/favicons?domain_url='+[the bookmark ur]+'"/>...
It's fairly horrible in terms of performance and privacy, because the favicons are right there in storage, but each bookmark being displayed results in a network request and the url gets sent to Google.

About 40 people use my extension, so I warn them of that in the description because it's not even something that can be added to the permissions section. It looks like this bug is taking a while because of privacy/security concerns, but without an official way to fetch favicons, devs will resort to terrible hacks that leak bookmarks data to sketchy services, or custom fetching code for a job that should be handled by the browser.

Comment 40

[mprops]

What are the technical obstacles that prevent implementing this?