Synced tabs, bookmarks, and history entries don't have a favicon until the user re-visits that page again

NEW
Unassigned

Status

Cloud Services
Firefox Sync: Backend
P3
enhancement
10 years ago
2 months ago

People

(Reporter: Jaime Mitchell (use bugmail@jaimem.org.uk for email), Unassigned)

Tracking

(Depends on: 2 bugs, Blocks: 1 bug)

unspecified
Points:
---
Dependency tree / graph
Bug Flags:
firefox-backlog +

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [sync-engine-addition] p=0)

Weave syncs bookmarks however the FavIcons are missing, should add the choice to also upload the FavIcons to the cloud.

Comment 1

10 years ago
It's a lot of data (in aggregate) to sync, but there are privacy concerns with the current way we do it.

Targeting for 0.3, we should find some solution for this.
Priority: -- → P2
Target Milestone: -- → 0.3

Comment 2

10 years ago
These bugs need to be triaged, removing 0.3 milestone setting.
Target Milestone: 0.3 → Future

Updated

9 years ago
Component: Weave → General
Product: Mozilla Labs → Weave
QA Contact: weave → general

Updated

9 years ago
Component: General → Sync
QA Contact: general → sync

Comment 3

9 years ago
One problem which actually affected me was that I have a lot of favicon-only bookmarks in my bookmarks toolbar, and when I sync them up to a new profile, they're all using the generic icon which are not meaningful at all...

Updated

9 years ago
Blocks: 530399
This bug was reported by me in Weave Google group... ;) I hope this get solved in the future... is nice to help you guys... Keep doing well... I will remain trying to debug and suggest features as a betatester am I :P

Comment 5

8 years ago
Syncing all favicon data would take up a lot of space, so perhaps one idea is to sync only the top 100 favicons based on their frecency. So multiple bookmarks on the same site will get their icons and frequently used bookmarks should have their frecency relatively high.

SELECT (SELECT frecency FROM moz_places h WHERE h.favicon_id = f.id ORDER BY frecency DESC LIMIT 1) frecency, f.id, f.url FROM moz_favicons f ORDER BY frecency DESC LIMIT 100;

Comment 6

8 years ago
Frequency isn't a bad indicator but I think syncing FavIcons from the bookmarks toolbar should come come before syncing the others since 1) those are the most seen and 2) some people remove the title leaving the FavIcon as the only way to recognize a site.

The best option would be to sync them all but if it's a server bandwidth or server space issue then it's understandable.

I also think it would be good to deprioritize in some way FavIcon syncing especially so that it doesn't happen before loading passwords/form data/etc. or conflict with loading tabs.
I agree Fritz!

Comment 8

8 years ago
Maybe, just info "what favicons should be downloaded asap" + idle process for downloading favicons from source will be enough?

Comment 9

8 years ago
My proposition is to not store the favicons themselves but rather just their hash (md5 or sha1). This would save hard disk drive storage space on the weave server. As many favicons are likely to be the same (on the same computer as well as on the different computers) the total amount of storage would significantly drop.
(In reply to comment #9)
> My proposition is to not store the favicons themselves but rather just their
> hash (md5 or sha1). This would save hard disk drive storage space on the weave
> server. As many favicons are likely to be the same (on the same computer as
> well as on the different computers) the total amount of storage would
> significantly drop.

You can't reconstruct the favicon from the hash, so I'm not sure how this proposal solves the problem.
(In reply to comment #9)
> My proposition is to not store the favicons themselves but rather just their
> hash (md5 or sha1). This would save hard disk drive storage space on the weave
> server. As many favicons are likely to be the same (on the same computer as
> well as on the different computers) the total amount of storage would
> significantly drop.

The reason we didn't do that before is that Mozilla would have to provide the service to go from hash -> favicon, effectively diluting the privacy gain from encrypting the bookmark in the first place.

We talked about separating it out and explicitly not saving any logs on that service, but we were hoping to find an alternative solution.

Updated

8 years ago
Duplicate of this bug: 568116

Comment 13

8 years ago
Instead of storing favicons, couldn't the plugin just download the favicons from the bookmarked sites?

Comment 14

8 years ago
(In reply to comment #13)
> Instead of storing favicons, couldn't the plugin just download the favicons
> from the bookmarked sites?

I think Dan Mills stated the case for Mozilla:

> The reason we didn't do that before is that Mozilla would have to provide the
> service to go from hash -> favicon, effectively diluting the privacy gain from
> encrypting the bookmark in the first place.

I do think it would be an issue if ever time you synced it would visit the site and download the FavIcon.  Imagine syncing at work and having it download NSFW websites FavIcons. Etc.  And since it's easily enough done manually...

Comment 15

8 years ago
> I do think it would be an issue if ever time you synced it would visit the site
> and download the FavIcon.  Imagine syncing at work and having it download NSFW
> websites FavIcons. Etc.  And since it's easily enough done manually...

It could be implemented as an option turned off by default, which gives security warning. Someone will make an extension for Fx Sync with such functionality anyway — it's safer if Sync developers will do it themselves.

Comment 16

8 years ago
(In reply to comment #14)
> Imagine syncing at work and having it download NSFW
> websites FavIcons. Etc.

That precise scenario would be fixed if bug 436259 were fixed.

Comment 17

8 years ago
(In reply to comment #16)
> (In reply to comment #14)
> > Imagine syncing at work and having it download NSFW
> > websites FavIcons. Etc.
> 
> That precise scenario would be fixed if bug 436259 were fixed.

No, it's different to say I don't want to sync my NSFW material between computers and I don't want Weave to automatically access third party sites.  The issue for this isn't whether someone will see NSFW material on your computer, it's whether they will see it in your network activity.

I think your bug is much more important.

Comment 18

8 years ago
(In reply to comment #14)
> I do think it would be an issue if ever time you synced it would visit the site
> and download the FavIcon.  Imagine syncing at work and having it download NSFW
> websites FavIcons. Etc.  

Ah, didn't think of that, thanks for explaining.

> And since it's easily enough done manually...

Really, How? Pressing a button to update all favicons would work for me. Although I'd still prefer an option do have it done automatically while at home.
Why not:
* Mark which bookmarks HAVE a favicon upon uploading them to the server.
* When downloading (when you sync), if it is marked as having a favicon, and does not have a favicon locally, retrieve it.

Relatively simple, band-width friendly, and only updates the favicons that need updating.
The main problem is that it effectively leaks a large portion of your past browsing history because you're hitting most of the domains in your history.

Not super awesome, overall, I think there's a better way to do this, in any case.

Comment 21

8 years ago
For an idea this might work. It should be able to keep it your data private, and reduce the storage on the backend.

Instead of saving the favicon per user, save it in a global account accessible via all users. People can submit icons anomalously to the global account.

On the backend: The favicons are saved and a hashmark is generated for it.  

On the user side: The favicon is saved as a hashmark. When the user goes to retrieve their favorite icons, it gets the encrypted hashmark from the user profile and then looks for the matching hashmark it in the global account and then requests it via an encrypted connection. 

The storage amount goes down per users as favicons are added.

Comment 22

8 years ago
(In reply to comment #21)
> The storage amount goes down per users as favicons are added.

Unfortunately, I'm pretty sure the issue was bandwidth and not hard disk space.

Comment 23

8 years ago
If bandwidth is a major concern then a rethink of how everything is transmitted to be synced may be required. Though I haven’t dug into the basic level of how everything is transmitted I am guessing that it’s on a per item bases. If this is the case doing something along the lines of compressing the entire data per sync item (bookmarks, history, cookies, etc…) should yield results greater then 50%.

Basically; backup all bookmarks to a zip file > transmit the zip file > unpack or even keep it packed. From the server’s side send the packed data back and have the client figure out how to sync the data. This keeps the backend extremely simple. 

However you are also hitting up on another major concern, if sync is to be used to replicate, transmit, stay in touch, share with others i.e. Tab Candy/Panorama then syncing a few icons infrequently will have less of an impact. Worst case scenario for a icon would be something like 32x32x24bit icon which is a little over 3072Bytes. The average is probably closer to 1024Bytes or less per icon assuming compression via GIF/PNG.
These are good ideas which would be possible if Weave didn't encrypt/decrypt everything at the source.

I suggest you continue discussing ideas on the mailing list, as you'll get a bigger audience and will keep this bug more focused.

Updated

7 years ago
Duplicate of this bug: 644468
Priority: P2 → P3
Summary: Sync Bookmarks FavIcons → Sync bookmark favicons
Whiteboard: [sync-engine-addition]
Target Milestone: Future → ---
Duplicate of this bug: 668607

Comment 27

7 years ago
Still a problem in FF5.0, I have to visit each link on the freshly synced bookmark toolbar to restore them, and I like to have links with no name just an icon. Very disconcerting, would like bad to a newbie or non-techie.  We want more users, not less.

Anyway, it seems to me that it isn't such an issue with all bookmarks, just the ones that are visible on the bookmark toolbar!

Best,
SSF
(In reply to comment #27)
> Still a problem in FF5.0

Thanks for the input, Steve. You'll note that this bug is still open, with whiteboards and other metadata for tracking; it'll get done sometime when schedules and priorities line up.
Depends on: 674238
Depends on: 675299
Depends on: 675996

Updated

7 years ago
Depends on: 608231
Depends on: 706347

Comment 29

6 years ago
I still have this bug with FF 16.0.1
(In reply to arielsilverstone from comment #29)
> I still have this bug with FF 16.0.1

In general, that will be true for any bug that's still marked as NEW or ASSIGNED, or for which the Target Milestone release is not less than the release you're running.

I hope we'll be addressing this some time next year.
Duplicate of this bug: 823883

Updated

5 years ago
Duplicate of this bug: 606807
FWIW, the new Sync 1.5 does not automagically sync Favicons :-/

(In reply to Richard Newman [:rnewman] from comment #30)
> I hope we'll be addressing this some time next year.

Can it be expected a team manager spending/offering 2 engineers (places, sync?) and them giving this some love? Maybe in some sync 1.5 post-entry-phase something?
Blocks: 950073
(In reply to XtC4UaLL [:xtc4uall] from comment #33)
> FWIW, the new Sync 1.5 does not automagically sync Favicons :-/

Yeah, I know, I built half of it.

Sync 1.5 is Sync 1.1 with FxA auth. Under the hood it's exactly the same beast.


> Can it be expected a team manager spending/offering 2 engineers (places,
> sync?) and them giving this some love? Maybe in some sync 1.5
> post-entry-phase something?

Some day, maybe.

There is no real Sync team. It'll happen eventually.

Updated

4 years ago
No longer blocks: 950073
Flags: firefox-backlog+
Whiteboard: [sync-engine-addition] → [sync-engine-addition] p=0

Updated

3 years ago
Duplicate of this bug: 1137633

Comment 36

3 years ago
I'd really like to see someone work on this. Why can't FF sync base64-encoded favicons of the most 100 recently used bookmarks to the cloud on initial sync, and then periodically refresh the data? That amount of bandwidth can't be too much for Mozilla, and there would be no privacy implications.
Meanwhile, here is a workaround with an add-on that reloads favicons :
https://addons.mozilla.org/fr/firefox/addon/faviconreloader/

Comment 38

3 years ago
(In reply to Antoine Turmel [:GeekShadow] from comment #37)

The mentioned add-on is truly useful, but there's one particular case when it isn't: if the bookmarked link doesn't contain a meta link to a favicon AND the host doesn't serve /favicon.ico

Consider this scenario:
User visits www.bank.com (which has a favicon) and clicks the link to LogIn, which redirects him to login.bank.com/login.do (which has no favicon) and user promply bookmarks that page. He notices his bookmark lacks an icon. He edits the bookmark's location to bank.com and clicks the bookmark. The appropriate favicon appears on the bookmark. He edits the bookmark's location back to login.bank.com/login.do. He is happy, until he realizes he has to repeat this process on all his other Firefox installations.

This is why I suggest that as soon as a favicon is associated with a bookmark it be converted to base64 and synced with other bookmark data.

Comment 39

3 years ago
I am also in favour of storing data in only base64.

Comment 40

3 years ago
The add-on suggested by Antoine is a bit of a workaround. It could be improved a lot with sorting and the TODO the author of it has already listed.
(In reply to James Donohue from comment #36)
> I'd really like to see someone work on this. Why can't FF sync
> base64-encoded favicons of the most 100 recently used bookmarks to the cloud
> on initial sync, and then periodically refresh the data? That amount of
> bandwidth can't be too much for Mozilla, and there would be no privacy
> implications.

Let me briefly answer why.

1. There are lots of open questions.

(a) What would we encode? Sites can refer to multiple icons of different types and formats. Do we put them all in each record? Just one?

(b) Some of these icons can be pretty big; larger than the maximum size of a Sync record.

(c) What about bookmarks that share an icon? Do we duplicate it into each bookmark record?

(d) Can we potentially double or triple the amount of data that a mobile device has to download? We're much less concerned about hosting bandwidth than about client bandwidth.

(e) How do the clients decide which bookmarks to update with a favicon? How do they decide when to reupload records to remove the icon? Sync is a distributed client-controlled system.

(f) When an icon changes, does the client have to reupload each bookmark? Or is it OK for different devices to have different icons?


2. Any change to the bookmark record format is undesirable -- changing the schema in this way is difficult. Adding new synced fields requires touching each client codebase, and making sure that each codebase is able to handle missing or malformed entries without hitting odd corner cases.


3. Compared to other known deficiencies in bookmark sync, this is pretty low priority.


Syncing favicons is superficially straightforward, but it's actually quite complicated and involves a sizable amount of work.


The most likely solution to resolve the real complaint here is that clients will batch-fetch icons for bookmarks they need to show, using some caution to avoid leaking browser history.

The next most likely solution is to sync favicons separately (Bug 428378).

The ultimate solution is to reimplement bookmark sync from scratch, but I don't really see that happening soon.


I think it's unlikely that we'd fix this bug by putting icon data into the bookmark record itself, with the possible exception of very small single favicons… but that juice likely isn't worth the squeeze.

Comment 42

3 years ago
(In reply to Richard Newman [:rnewman] from comment #41)
Thanks for taking the time to craft such a well-thought reply. There are issues here, granted. But a paper airplane doesn't need to go to the moon.

1. Open any version of Firefox for desktop since v1.0
2. Visit a site with a favicon. Press ctrl-D
3. Press ctrl+shift+B and find your bookmark
4. Observe the 16x16 icon to the left of your bookmark

Whatever you see in step 4, sync it! If it's animated then just take the first fame; if that's a hassle to impliment then skip it because animated favicons are annoying. Don't put the data into the bookmark record if that requires too much codebase overhaul. Create 100 new sync preferences, set their string value to the base64 value of the icon, and name the preferences in such a way that the browser can correleate them to the 100 most recently used bookmarks.

Maybe that's not the best way but the point is it's possible to plug in to the SyncStorage API to get this done. Make it a manual process if necessary; put a big ugly "sync favicons now" button in about:preferences#sync -- who cares. Seriously, Mozilla has re-worked Sync countless times over the last 8 years and nobody ever thought to add favicons? They are as integral to a pleasant browsing experience as bookmarks themselves and nobody wants to set up a new installation and stare at a screen full of 1px dashed boxes (referring to the bookmarks library).

Comment 43

3 years ago
Any way to store favicons locally and check against that when syncing?
(In reply to James Donohue from comment #42)

> Create 100 new sync preferences, set
> their string value to the base64 value of the icon, and name the preferences
> in such a way that the browser can correleate them to the 100 most recently
> used bookmarks.

If you knew how the prefs sync engine worked, you wouldn't propose that approach :)


> Seriously, Mozilla has re-worked Sync countless times over the last 8
> years and nobody ever thought to add favicons?

We actually haven't done that, which is the root of the problem.


> pleasant browsing experience as bookmarks themselves and nobody wants to set
> up a new installation and stare at a screen full of 1px dashed boxes
> (referring to the bookmarks library).

But nobody wants to set up a new device and have 16x16 favicons, either; the home screen on Android or iOS would look terrible.

Modern sites have icons that are 76x76 or bigger. And on most devices it's your history, not your bookmarks, that dictate what you see when you first open the browser.

The approach we're taking on iOS is to scrape your top sites on demand and fetch the right icon. It works well enough.

I think it's clear from the history of this bug that 'good enough' isn't going to be good enough for this bug.

* Some desktop users want the icons from their toolbar, not just the most frecent. Desktop users with the bookmarks sidebar will want those icons.
* Some users are on Retina displays, so 16x16 looks terrible.
* Android and iOS users will want top sites icons, and they'll want them to be msTileImage-style large icons, not 16x16 favicons. But we can't downscale those, so now we're syncing two icons per site!
* We'd need to handle users who sync only history or only bookmarks.
* Some users will want a blacklist for adult sites, because they don't want icons for porn sites preemptively synced to their work computer.
* We'd need to build in some kind of expiration: my places.sqlite has over 6000 favicons, totaling 5MB of raw data (much more when expanded into Sync records). We can't just rely on Sync's TTLs, because favicons don't change often enough, and if we throw away the wrong icons we make the whole thing pointless.
* We'd need to address the mismatch inherent in syncing different kinds of things: the durability requirements of storing critical data like passwords don't make sense from a cost perspective for storing favicons, particularly not for hundreds or thousands of records per user.


My proposal: acknowledge that the main problem being solved here is leakage of browsing history onto the network. Otherwise the client could fetch favicons directly from the source site; it knows the URLs!

One way to address that is to build a caching proxy. Maybe use something stored in your FxA to obfuscate the icons you're requesting, or maybe not; just used a fixed salt to hash the requested URL, so the wire traffic isn't enough to see what you've been browsing. The cache itself can be shared between users.

We could extend the bookmarks and history formats to acknowledge that there are multiple icon URLs for a site, each with different properties, and get better behavior than we currently have. Or introduce an intermediate record.
(In reply to cristo morlan from comment #43)
> Any way to store favicons locally and check against that when syncing?

Firefox does that.

Comment 46

3 years ago
Eventually, icons of all used sizes need to be stored, so that all user’s devices have suitable icons.

However, at the time when a bookmark is created, Firefox does not know which sizes will be used. The user might add another device ten minutes later.

Firefox could preëmptively store all icon sizes the site offers. This makes the logic simpler, at the expense of storage space.

Alternatively, Firefox could initially store the icon used by the original device. When that bookmark is synced to a device which needs another icon size, to avoid leaking history, fall back to one of the icons available in the storage, scaling it to the size appropriate for the device. If and when the bookmark is accessed on this device, fetch the appropriate icon and store it along with the bookmark.
Duplicate of this bug: 1238867
Summary: Sync bookmark favicons → Synced tabs, bookmarks, and history entries don't have a favicon until the user re-visits that page again
Blocks: 1239084
There's a long comment history here, so a brief stand-out comment for future consumption:

We've discussed having a 'salts' collection as a way to privately map URL => record ID. If we sync favicons in some way, we would probably do it like that. There's no bug on file, so if we end up addressing this, we should file one.
(In reply to Richard Newman [:rnewman] from comment #48)
> We've discussed having a 'salts' collection as a way to privately map URL =>
> record ID. If we sync favicons in some way, we would probably do it like
> that. There's no bug on file, so if we end up addressing this, we should
> file one.

Do we really want to sync favicons? I'm thinking that (a) it's quite a bit more data to sync with each tab and (b) the synced data may not be appropriate for every device anyway. Would it be good enough to asynchronously fetch a favicon locally once we notice it missing?
(In reply to Mark Hammond [:markh] from comment #49)

> Do we really want to sync favicons? I'm thinking that (a) it's quite a bit
> more data to sync with each tab and (b) the synced data may not be
> appropriate for every device anyway. Would it be good enough to
> asynchronously fetch a favicon locally once we notice it missing?

Those are two of the concerns we've been discussing in this bug.

In fairness, the impact on the user experience is not zero, the binary size of all favicons in my gigantic personal Places DB is only about 5MB*, and the privacy implications of fetching icons live from the web over the user's current network connection could be significant.

We do so for Top Sites on iOS, and it's a pain -- we actually fetch the page in order to parse out potential icon URLs.



* Measured by vacuuming, DELETE FROM moz_favicons;, vacuuming again, and viewing the size on disk of places.sqlite. The DB went from 62MB to 57MB. I had 6,376 icons in the table.
(In reply to Richard Newman [:rnewman] from comment #48)
> There's a long comment history here, so a brief stand-out comment for future
> consumption:
> 
> We've discussed having a 'salts' collection as a way to privately map URL =>
> record ID. If we sync favicons in some way, we would probably do it like
> that. There's no bug on file, so if we end up addressing this, we should
> file one.

Bug 824188 is sort of a bug for that, and is part of this bug tree.
Given that this isn't a regression, this won't block the Synced Tabs UI refresh
No longer blocks: 1239084

Updated

2 years ago
See Also: → bug 1246076

Comment 53

2 years ago
Reply to Chris Karlof [:ckarlof]   

All the graphics are gone.  Pictures, icons, etc.  If I log onto facebook, where there should be pictures, it shows blank text boxes.  For my gmail, icons for mail, drive, etc. are gone.  Text is on top of other text, I think because the graphic isn't there.  Sometimes it shows the 'broken picture' icon, sometimes it's blank.  Depending on the website.  When I refresh Mozilla and leave sync off, I don't have any problem.  Does this help?
Comment 53 was intended for Bug 1253325.
Duplicate of this bug: 1338693

Comment 56

a year ago
Hello,

It's a really useful feature and I really don't understand why it's still not possible.
Can you add this to Firefox please ?

Thanks !

Comment 57

a year ago
Why do we need to sync favicons? That's not a good option.
Why not just pull missing favicons from Internet for all the bookmarks and store them locally?
This would be pretty small amount of data and without any privacy and security concerns.

Artem
(In reply to Artem Polivanchuk from comment #57)
> This would be pretty small amount of data and without any privacy and
> security concerns.

There are privacy concerns - see, eg, comment 14. It is debatable if those concerns are greater than the desire to have this feature, but it is definitely a consideration.

Comment 59

a year ago
Retrieving bookmarks over a long period of time from different IP address is far more safe than re-retrieving all at once from a single IP address.

Bookmarks do change, see e.g. YouTube. It is a big visual aid when (updated) favicons are also contributed from other browser instances on other machines.
> There are privacy concerns - see, eg, comment 14. It is debatable if those concerns are greater than the desire
> to have this feature, but it is definitely a consideration.
This could be a preference, off by default.  On that specific case, the user would leave it on at home but off at work.
it's not just privacy, there are also security concerns, since to fetch an icon we need the principal of the page that requested it, and we don't have it at any time.
It's all solvable problems with resources, we could likely load the page in a chromeless remote window (like we do for thumbs) and refetch icons for that page. It's an expensive problem to solve though.
Blocks: 1269548
Duplicate of this bug: 1368683
Duplicate of this bug: 1371696

Updated

10 months ago
See Also: → bug 1384685

Updated

3 months ago
Duplicate of this bug: 1276111

Updated

3 months ago
Duplicate of this bug: 1046012
You need to log in before you can comment on or make changes to this bug.