Last Comment Bug 829803 - Add support for masking to Canvas
: Add support for masking to Canvas
Status: UNCONFIRMED
[shumway]
: feature
Product: Core
Classification: Components
Component: Canvas: 2D (show other bugs)
: Trunk
: All All
: -- normal with 1 vote (vote)
: ---
Assigned To: Rik Cabanier
:
Mentors:
Depends on: 767926 849307
Blocks: 927828 shumway-m5
  Show dependency treegraph
 
Reported: 2013-01-11 15:13 PST by Rik Cabanier
Modified: 2015-06-16 01:58 PDT (History)
8 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Example implementation for Core Graphics (9.54 KB, patch)
2013-01-11 15:15 PST, Rik Cabanier
no flags Details | Diff | Review
screenshot of blending issue (252.77 KB, image/png)
2013-01-15 15:46 PST, Brendan Dahl [:bdahl]
no flags Details
my screen shot (907.96 KB, image/tiff)
2013-01-15 17:03 PST, Rik Cabanier
no flags Details
acrobat screenshot (154.56 KB, image/tiff)
2013-01-15 20:10 PST, Rik Cabanier
no flags Details
file with a blended transparency group (332.61 KB, application/pdf)
2013-01-16 10:31 PST, Rik Cabanier
no flags Details
Example implementation for Core Graphics (18.40 KB, patch)
2013-09-30 14:54 PDT, Rik Cabanier
no flags Details | Diff | Review

Description Rik Cabanier 2013-01-11 15:13:01 PST
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.101 Safari/537.11

Steps to reproduce:

Currently, there is no support for alpha or luminosity masking in canvas.
This makes it almost impossible to efficiently implement soft masks for pdf.js.

I found several people that provided workaround and shims for this feature.
Comment 1 Rik Cabanier 2013-01-11 15:15:39 PST
Created attachment 701331 [details] [diff] [review]
Example implementation for Core Graphics
Comment 2 Rik Cabanier 2013-01-11 15:20:49 PST
I attached an example implementation that will only work with the CG backend.

the proposed API is as follows:
   void mask(CanvasImageSource image, 
                     unrestricted double sx, unrestricted double sy, unrestricted double sw, unrestricted double sh, 
                     unrestricted double dx, unrestricted double dy, unrestricted double dw, unrestricted double dh);

the first parameter points to an image source
sx, sy, sw, and sh point to a rectangle in the source image that you want to use as a mask
dx, dy, dw and dh point to the destination area that you want the mask to apply to.

The API currently converts colors+alpha to luminosity so the pdf.js people can play with this.
white = use the foreground colors
black = use the background colors
Comment 3 Rik Cabanier 2013-01-11 15:24:08 PST
The mask is also part of the graphics state, so it will respond to save/restore.

jsfiddle example: http://jsfiddle.net/NDYV8/11/
Comment 4 Rik Cabanier 2013-01-12 08:41:05 PST
Comment on attachment 701331 [details] [diff] [review]
Example implementation for Core Graphics

Maybe 'maskImage' is a better API
Comment 5 Boris Zbarsky [:bz] 2013-01-12 08:43:43 PST
Comment on attachment 701331 [details] [diff] [review]
Example implementation for Core Graphics

Going to punt this to someone who knows graphics things.  ;)
Comment 6 Brendan Dahl [:bdahl] 2013-01-14 13:34:40 PST
Rik,
I'll hopefully have time to play with this later today or tomorrow to see if it would meet pdf.js needs.
Comment 7 Rik Cabanier 2013-01-14 13:36:59 PST
Great!
I think it will need 2 more parameters to cover all the bases (cliptomask and invert) but you should be able to experiment as-is.
Comment 8 Brendan Dahl [:bdahl] 2013-01-15 15:46:37 PST
Created attachment 702567 [details]
screenshot of blending issue

I think something may be screwy with blending the source and backdrop with blend modes.  I hacked up pdf.js to use mask at https://github.com/brendandahl/pdf.js/commit/765a421df04499afc14b5bcbeab0856e160a18b3 ). Using the example pdf from http://bugs.ghostscript.com/attachment.cgi?id=4177 I get the attached result.  If I comment out the blending modes code in pdf.js I can see the blue squares drawn, but with blending modes on it seems it is not blending the colors where the backdrop is empty (only where the orange circle is).

I can try to come up with a simpler example that doesn't use pdf.js, but if you have any ideas let me know.
Comment 9 Rik Cabanier 2013-01-15 17:03:59 PST
Created attachment 702608 [details]
my screen shot
Comment 10 Rik Cabanier 2013-01-15 17:04:35 PST
it looks fine to me.
I wonder what the stuff at the bottom is though...
Comment 11 Brendan Dahl [:bdahl] 2013-01-15 17:30:53 PST
(In reply to Rik Cabanier from comment #9)
> Created attachment 702608 [details]
> my screen shot

Looks like you don't have the blending modes turned on.  Compare the pdf to acrobat and you'll see that there shouldn't be any blue over the orange circle.

(In reply to Rik Cabanier from comment #10)
> it looks fine to me.
> I wonder what the stuff at the bottom is though...

That's the soft mask, I just added it for debugging purposes to see it.
Comment 12 Rik Cabanier 2013-01-15 20:10:15 PST
Created attachment 702657 [details]
acrobat screenshot

Not sure what you're seeing.
There's blue over the circle in acrobat too... See screenshot
Comment 13 Brendan Dahl [:bdahl] 2013-01-15 20:40:54 PST
Sorry about that I linked to the wrong pdf in that bug it should be http://bugs.ghostscript.com/attachment.cgi?id=4358
Comment 14 Rik Cabanier 2013-01-15 21:33:14 PST
(In reply to Brendan Dahl from comment #13)
> Sorry about that I linked to the wrong pdf in that bug it should be
> http://bugs.ghostscript.com/attachment.cgi?id=4358

Are you executing the form in its own canvas context?
From what I can tell, you execute it in the same context so the first gstate blows the blend mode away.
Execute the form in its own state and its own temporary canvas and then do a drawimage with the result.
Comment 15 Brendan Dahl [:bdahl] 2013-01-16 10:07:18 PST
I am drawing the form/soft mask to a separate canvas (https://github.com/brendandahl/pdf.js/commit/765a421df04499afc14b5bcbeab0856e160a18b3#L0R1477).

The issue I found out was from the pdf.js viewer filling the page with white before we rendered it (https://github.com/mozilla/pdf.js/blob/master/web/viewer.js#L2053). Since the blend mode was screen, white remained white.  Once I removed that everything looks as expected.

I'll play around with adding transparency groups next.  I still need to copy over the state of the current canvas to the form canvas since according to the spec it is suppose to inherit all the graphics state except blend mode, soft mask, and alpha constant.
Comment 16 Brendan Dahl [:bdahl] 2013-01-16 10:23:36 PST
What are you thoughts on creating a function to clear or remove the mask?  It seems kind of clunky right now to have to create new empty mask or having to use save/restore.
Comment 17 Rik Cabanier 2013-01-16 10:31:33 PST
Created attachment 702897 [details]
file with a blended transparency group

the transparency group itself also need to be put in its own temporary canvas
Comment 18 Rik Cabanier 2013-01-16 10:32:28 PST
> The issue I found out was from the pdf.js viewer filling the page with white
> before we rendered it
> (https://github.com/mozilla/pdf.js/blob/master/web/viewer.js#L2053). Since
> the blend mode was screen, white remained white.  Once I removed that
> everything looks as expected.
> 
yes, matting is done as the last step
Comment 19 Rik Cabanier 2013-01-24 11:08:09 PST
(In reply to Brendan Dahl from comment #16)
> What are you thoughts on creating a function to clear or remove the mask? 
> It seems kind of clunky right now to have to create new empty mask or having
> to use save/restore.

I think that sounds reasonable.
do you have suggestions for the syntax? resetMask()
Comment 20 Rik Cabanier 2013-01-24 13:33:52 PST
(In reply to Brendan Dahl from comment #15)
> I am drawing the form/soft mask to a separate canvas
> (https://github.com/brendandahl/pdf.js/commit/
> 765a421df04499afc14b5bcbeab0856e160a18b3#L0R1477).
> 
> The issue I found out was from the pdf.js viewer filling the page with white
> before we rendered it
> (https://github.com/mozilla/pdf.js/blob/master/web/viewer.js#L2053). Since
> the blend mode was screen, white remained white.  Once I removed that
> everything looks as expected.
> 
> I'll play around with adding transparency groups next.  I still need to copy
> over the state of the current canvas to the form canvas since according to
> the spec it is suppose to inherit all the graphics state except blend mode,
> soft mask, and alpha constant.
yes, I think that's correct.
Almost all the time, the first thing a group does is reset that all to an initial state though so if it's complex, you might want to add that later.
Comment 21 Rik Cabanier 2013-08-19 21:37:18 PDT
I've been looking at implementing this for just canvas image sources (since the IDL compiler still doesn't support unions).
The Core Graphics implementation is complete and I'm looking at cairo next. 
What's the best way to submit this? Can I just provide a patch with my current work and then provide patches for the different backends and more image sources?
Comment 22 Brendan Dahl [:bdahl] 2013-09-30 14:43:27 PDT
It would probably be best to post some patches so you can get feedback as you go. Also, support for unions landed a few weeks ago.
Comment 23 Rik Cabanier 2013-09-30 14:54:08 PDT
Created attachment 812291 [details] [diff] [review]
Example implementation for Core Graphics

updated implementation. Still need to add support for unions
Comment 24 Chris Peterson [:cpeterson] 2014-09-18 11:35:06 PDT
Michael says Shumway can emulate this. Is this a dupe or dependent of other gfx work on CSS blend modes and filters for canvas?

Note You need to log in before you can comment on or make changes to this bug.