1136196 - Extract a color palette from the site for use in the color picker

Status: NEW

(DevTools :: Inspector, enhancement, P3)

Description

[Patrick Brosset <:pbro>]

We're turning the inspector into more and more of an authoring tool these days, and one of the things that tools designers use have is some kind of color palette management.

The use case a color palette would solve is: make it easy for users to set a new text/background/border/gradient color on an element by selecting a color that already exists in the palette.

Now, when it comes to actually creating the palette, I can see 2 schemes:
- automatic creation by extracting the most used colors from the computed styles or document stylesheets,
- or reading from a list of defined colors, either from a list the user can store in the document itself, or from a list previously created via a new devtools UI and stored in the profile.

My gut feeling is that the automatic creation may not result in a very nice list of main colors, but may actually be very useful nonetheless.
Maybe both modes should be possible.

Comment 1

[Patrick Brosset <:pbro>]

I have a feeling there was an old bug about this, but can't find it. Maybe bgrins remembers.

Comment 2

[Brian Grinstead [:bgrins]]

(In reply to Patrick Brosset [:pbrosset] [:patrick] from comment #1)
> I have a feeling there was an old bug about this, but can't find it. Maybe
> bgrins remembers.

I remember discussing it, but I don't think a bug was ever filed for this.

We should probably prepopulate with colors found on the page, I'm not sure if it'd be worth adding a whole UI for picking the palette (especially since it would change per-page / per-site.

In the spectrum project there is also a concept of a 'selection palette' which remembers the last N colors that you've selected, and it shows up below the main palette which in that case is a predefined list: https://bgrins.github.io/spectrum/#options-selectionPalette.  Not sure if that would fit in here, but it's another option.

Comment 3

[Jeff Griffiths (:canuckistani) (:⚡︎)]

(In reply to Patrick Brosset [:pbrosset] [:patrick] from comment #0)
> We're turning the inspector into more and more of an authoring tool these
> days, and one of the things that tools designers use have is some kind of
> color palette management.

Yup, agreed.

We had an idea that the eyedropper could be used to build up a palette - we couldn't think of a way to do that in the eye dropper itself without complicating the UI.

> The use case a color palette would solve is: make it easy for users to set a
> new text/background/border/gradient color on an element by selecting a color
> that already exists in the palette.

Where do you see most of the UI living - in the color picker?

How would we store palettes - per site? As a big searchable list somewhere? I might want to build up a palette from one site and then use it in another site, I'd want to somehow set palette $y as the selected one for site $z.

> Now, when it comes to actually creating the palette, I can see 2 schemes:
> - automatic creation by extracting the most used colors from the computed
> styles or document stylesheets,
> - or reading from a list of defined colors, either from a list the user can
> store in the document itself, or from a list previously created via a new
> devtools UI and stored in the profile.
> 
> My gut feeling is that the automatic creation may not result in a very nice
> list of main colors, but may actually be very useful nonetheless.
> Maybe both modes should be possible.

I think the manually picked palette is the core use case - automatically generating does sound fun tho and only needs to be useful to some people some of the time, or create a 'good start'?

Comment 4

[Patrick Brosset <:pbro>]

If storage is required for this, a new async storage facility landed in devtools: https://groups.google.com/forum/#!topic/mozilla.dev.developer-tools/fJLi3w7lBbA

If we want to extract colors automatically (this could be a nice feature when creating a palette, to automatically fill the empty palette with dominant colors on the site, and then let the user add/remove from it if needed), there are several ways:
- Iterate over the stylesheets in the document and list all colors found in gradients, background, font-colors, borders, shadows, etc... This may potentially return colors the user doesn't want to see in the palette, that no elements on the current page use.
- Iterate over DOM elements, only retaining the ones that are visible, and search for colors in the computed styles. This would only return the currently visible colors.
- Transform the page to an image and iterate over the pixels, using an algorithm that only retains the dominant colors. I can see 2 potential problems: this will sample the colors from images too, as well as intermediate gradient colors or colors from areas covered by semi-transparent shadows.

Comment 5

[Brian Grinstead [:bgrins]]

(In reply to Patrick Brosset [:pbrosset] [:patrick] from comment #4)
> If storage is required for this, a new async storage facility landed in
> devtools:
> https://groups.google.com/forum/#!topic/mozilla.dev.developer-tools/
> fJLi3w7lBbA
> 
> If we want to extract colors automatically (this could be a nice feature
> when creating a palette, to automatically fill the empty palette with
> dominant colors on the site, and then let the user add/remove from it if
> needed), there are several ways:
> - Iterate over the stylesheets in the document and list all colors found in
> gradients, background, font-colors, borders, shadows, etc... This may
> potentially return colors the user doesn't want to see in the palette, that
> no elements on the current page use.

Probably this would be best, at least as a starting point.  Both because it looks like the most straightforward to implement and because there may be colors that are used in some other element in your site that you want to reuse (even if it's not currently visible at this moment).

As Jeff points out, we will need manual editing one way or another so I think we should focus the initial work on that.  Basically:

* Being able to take a currently selected color and add it into the palette
* Remove colors from the palette
* (Maybe) reorder colors or group them inside of a palette

As far as palette management, we could start simple simple and work towards more advanced:

1) You can only have a single user palette
2) You can keep multiple user palettes that you manually create and switch between (one for each project you are working on)
3) These palettes can be supplemented with an automatic one based on the stylesheets on the page.  So you would have two visible palettes at once - one for the current page and the currently active user palette.

Comment 6

[Patrick Brosset <:pbro>]

The Chrome devtools people recently published this:
https://trello.com/c/u6rGRWB6/14-material-color-support

With a nice color palette screenshot:
https://trello-attachments.s3.amazonaws.com/54e71c3dc66e09bf7d77f307/2040x692/00c3b38468395e7b6b3a11d6b92c8834/05_-_Color_Palettes.png

What the screenshot shows seems pretty in line with what we've been discussing:
- auto-extract of colors from the CSS on the page (the "CSS Auto" entry in the screenshot)
- manual palette creation (the "My Site" entry was probably created by click on "Add...").

Comment 7

[Tim Nguyen :ntim]

IE also shows a palette.

Comment 8

[Patrick Brosset <:pbro>]

In terms of UI, we need 2 components: one to create and globally manage palettes, and one to see palettes and pick colors from them.
The latter is mostly needed in the color-picker right now. So that is a relatively simple UI problem to solve. In the color picker tooltip, we could have tabs, one that shows the current color spectrum, and another one that shows the current list of palettes and allows to select colors in them.

As to manage palettes, this is slightly more complex. Right now the toolbox mostly have 2 main UI types of elements: panels, and sidebars. I don't see such a tool living in either of them.
In fact, I think having a toolbar button would make more sense for this. Once clicked, it would display a overlay panel (like the download button in the firefox toolbar) that contains the UI necessary for creating/naming/deleting/re-ordering palettes.

Comment 9

[Patrick Brosset <:pbro>]

Attachment: Bug_1136196_-_Page_Color_Extractor_Actors.diff

Extracting colors automatically from the page is something that needs to be done with access to the page, so on the server + it can be done in several ways as we discussed previously, so I thought having a new actor for this would be a good idea. Here's a WIP patch with a PageColorActor that has a getColors method that can extract colors in several ways (right now it only does this by looking at colors in stylesheets).
It returns a list of ColorOccurrenceActors, each of them represent a unique color on the page and maintains a list of all places where that color occurs. 
The idea behind this was that we could use automatically generated palettes to easily swap colors on a page. Say you want to test how the page looks when you change the main color to something else, but that main color isn't defined in just one place in the css.

Comment 10

[Julian Descottes [:jdescottes]]

Inspector bug triage, filter on CLIMBING SHOES .

Comment 11

[Patrick Brosset <:pbro>]

Inspector bug triage (filter on CLIMBING SHOES).