1179083 - (dynamic-tiles-v1) [Meta] Teach Page Thumbs (tiles) how to generate a screenshot from Web Manifest

Status: RESOLVED WONTFIX

(Content Services Graveyard :: Tiles, defect)

Description

[Marcos Caceres [:marcosc]]

This bug tracks work related to adding the capability to generate tiles based on a Web manifest. This includes error recovery when that fails, such that we fall back to using icons or the site's screenshot if all else fails.

Comment 1

[Tim Taubert [:ttaubert] (inactive)]

Why is this a confidential bug?

Comment 2

[Marcos Caceres [:marcosc]]

We don't know if we are doing this yet, and don't want to send out the wrong signal to other vendors. There are a bunch on implication for the standardization process of web manifest.

Comment 3

[Marcos Caceres [:marcosc]]

Attachment: Part 1 - Code that generates preview thumbnail

This code is rough and only part of a bunch of other related changes - but it's the most interesting part:) I've left all the dump/debug statements in there on purpose (following the e10s message ping-pong can be a challenge otherwise). 

The code basically does the following:

1. checks if this is a preview request. If it is, it captures the preview page (by doing a round trip to load the preview page instead).
2. If it's not a preview request, it checks if the page's content has a manifest.
 - if it has a manifest, check if it has a valid `preview` member. If it does, request that preview be captured instead (go to step 1).
 - Otherwise, try to capture the icon.
3. If the above fails, fall back to capturing the screenshot.

The above mostly gets the job done. But ideally if the preview fails (e.g., 404), it should probably fall back to trying to use the icon.  

Anyway, looking forward to hearing what you guys think.

Comment 4

[Marcos Caceres [:marcosc]]

Chatted with Tim Taubert about the design a few days ago. Generally, he thinks the way that we do things on the content/child-process is fine, but raised a important issue: PageThumbs.jsm is a generic page snap-shotting utility. Although the only shipping consumer of PageThumbs is the new tab page, there are other consumers that could come along and trash any generated snapshots. Also, some consumers may not want the snapshot to be generated from the Web manifest.  

There are a few solutions to the above: 
* make using the manifest opt-in - and enhance PageTumbStorage to handle both screenshot and manifest-generated previews.  
* Create a new TilesPageThumbs.jsm specifically for the new tab page, which leverages PageThumbs for generating the thumbnail, but uses a different Storage area.

Comment 5

[Marcos Caceres [:marcosc]]

@oyiptong. Some initial sites using manifest below. I need to find time to do a large scale scrape of the top 100K sites form Alexa, but at least it gives us something to start with:

 * cnet.com
 * guardian.com
 * irccloud.com
 * https://events.google.com/io2015/
 * pinterest.com 
 * http://www.skyscanner.es/
 * http://www.tf1.fr/

Comment 6

[Olivier Yiptong [:oyiptong]]

I took a look and spoke to ttaubert.

We want to make PageThumbs more generic, where there are more than one consumer.
Already, there is TabGroups and the newtab page as consumers.

IMO, the best way to go forward is to acknowledge this in PageThumbs, instead of creating a separate version of PageThumbs (Ostensibly TilesPageThumbs).

To minimize the impact on existing code, we could make it so that PageThumbs returns the screenshots by default, and making it return manifests if they exist when specifying a parameter, as suggested by Marcos.

This would entail adding functionality to PageThumbStorage.

Comment 7

[jterry]

(In reply to Marcos Caceres [:marcosc] from comment #5)
> @oyiptong. Some initial sites using manifest below. I need to find time to
> do a large scale scrape of the top 100K sites form Alexa, but at least it
> gives us something to start with:
> 
>  * cnet.com
>  * guardian.com
>  * irccloud.com
>  * https://events.google.com/io2015/
>  * pinterest.com 
>  * http://www.skyscanner.es/
>  * http://www.tf1.fr/

Hey Marcos, are these sites using Manifest as we intend it to be or are they using it primarily for Mobile? Also, can we post some screenshots of what sample Web Manifest Screen Thumb would look like from one of these publishers (just for a reference)?

Comment 8

[Olivier Yiptong [:oyiptong]]

Attachment: Manifest Newtab Sequence Aug 4.png

Green is landed
Yellow is prototype-exists
Red needs to be implemented

State of the world today.
The diagram describes the newtab render sequence.

There is another entrypoint: through PageThumbs.captureAndStore

Comment 9

[Olivier Yiptong [:oyiptong]]

Attachment: Manifest Newtab Sequence.png

Comment 10

[Marcos Caceres [:marcosc]]

Comment on attachment 8628076 [details] [diff] [review]
Part 1 - Code that generates preview thumbnail

don't need feedback