1064715 - [markup view] Support <picture/source> elements like <img>

Status: NEW

(DevTools :: Inspector, enhancement, P2)

Description

[Patrick Brosset <:pbro>]

The new <picture> element is nearly there (see bug 870022 and bug 1017875).

The markup-view should probably provide the same level of support it does for the <img> element:

- hovering over srcset URLs in <source> elements should show the image preview tooltip
- the ctx menu item "Copy image data-uri" should be available on <source> elements too
- probably the same should apply to <picture> elements

I'm guessing this can already be done today without waiting for the <picture> elements to be supported by default in Firefox.

You can see live examples of the <picture> element here:
https://longhandpixels.net/blog/2014/02/complete-guide-picture-element

Comment 1

[Patrick Brosset <:pbro>]

The inspector uses canvas2Dcontext.drawImage to get the data-uri for an image (the uri is then sent to the client for preview in the tooltip).
Passing a <picture> tag to drawImage doesn't work today but will probably in the future, but will most probably only work with the image which URL has been loaded by the <picture> tag, depending on the media query that matched.
So, if we want tooltips for all <source> tags, we're going to have to load the srcset urls via images first.

Comment 2

[Patrick Brosset <:pbro>]

Also an interesting use case for this is being able to know which of the <source> urls has been loaded by the <picture> element.

Comment 3

[Patrick Brosset <:pbro>]

Attachment: bug1064715-preview-source-elements.patch

I don't have much time to work on this right now, and since the picture hasn't even landed, this isn't urgent, but anyway, here's a simple patch that previews <source> images when hovering over the srcset attribute.

Comment 4

[Simon Pieters [:zcorpan]]

+    if (tagName === "img" && srcAttr) {
+      return "img";

img can have srcset attribute also.

+          url = value.split(",")[0].trim().split(" ")[0];

This will produce the wrong result e.g. when the URL contains commas (e.g. data: URL). I think you need a srcset parser to do it properly.

Comment 5

[Patrick Brosset <:pbro>]

(In reply to Simon Pieters from comment #4)
> +    if (tagName === "img" && srcAttr) {
> +      return "img";
> 
> img can have srcset attribute also.
Right, I thought I saw that in the spec indeed, but I'm not too familiar with how this work yet.
> +          url = value.split(",")[0].trim().split(" ")[0];
> 
> This will produce the wrong result e.g. when the URL contains commas (e.g.
> data: URL). I think you need a srcset parser to do it properly.
Yeah, that would definitely be better. This patch was more to prove the concept than anything else anyway.
If you're on irc, I'd love to chat about the syntax/format of these tags and attributes.

Comment 6

[Simon Pieters [:zcorpan]]

Sure, #respimg on irc.w3.org:6665. I'm zcorpan.

Comment 7

[Patrick Brosset <:pbro>]

So, after chatting with Simon, here's what I learned:

- srcset attributes can have multiple "image candidate strings", each containing a URL [1]
- it's up to the UA to decide which of the URLs gets loaded/displayed depending on the screen density, network conditions, etc.
- it's however up the author to decide which <source> element gets chosen, depending on the media query that matches.
- parsing an image candidate string isn't entirely straightforward, here are test cases for it [2]
- which URL gets displayed in the end can be known by accessing the .currentSrc property

So, in the inspector, for each <source> we should be able to display a preview of all URLs in srcset attributes.
This means parsing all URLs and updating the image preview as the user hovers over them.

Another nice feature would be to somehow mark which <source> is currently being used, and from all the URLs in its srcset attribute, which one has been displayed.

Finally, we should also have an image preview on the <picture> element.

[1] http://www.whatwg.org/specs/web-apps/current-work/multipage/embedded-content.html#image-candidate-string
[2] http://w3c-test.org/submissions/1234/html/semantics/embedded-content/the-img-element/srcset/parse-a-srcset-attribute.html

Comment 9

[Patrick Brosset <:pbro>]

(In reply to Patrick Brosset [:pbro] from comment #7)
> Another nice feature would be to somehow mark which <source> is currently
> being used, and from all the URLs in its srcset attribute, which one has
> been displayed.
This will be done in bug 1260230.