Bug 102285 (canvas)

create xul canvas tag for custom painting control

RESOLVED DUPLICATE of bug 288714

Status

()

Core
Canvas: 2D
P3
enhancement
RESOLVED DUPLICATE of bug 288714
16 years ago
3 years ago

People

(Reporter: Joe Hewitt (gone), Assigned: vlad)

Tracking

Trunk
Future
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

16 years ago
A few weeks ago I spent a weekend getting this up and running on windows.  Now I
need to get it working on mac and linux and check it in.  I'd appreciate as much
input on the design as possible.  I'll post my patch asap.
(Reporter)

Updated

16 years ago
Status: NEW → ASSIGNED
Priority: -- → P3
Target Milestone: --- → mozilla0.9.7
Random platform buddies and rendering fanboys.

/be
(Reporter)

Comment 2

16 years ago
Created attachment 51435 [details] [diff] [review]
patch with initial windows implementation
(Reporter)

Comment 3

16 years ago
For those who would like to test this out, I've zipped up a windows build for
your amusement.  Download the build from here:
http://joehewitt.com/mozilla/canvas.zip

I've written two demos: one is a little paint program written entirely 
in javascript, and the other is a color-blending animation written with 
javascript and C++, which re-paints every pixel of an image 
continuously at about 60 frames per second.

The demos are included in the test build.  To see them, browse to:
chrome://hewitt/content/photoshop.xul
chrome://hewitt/content/seizure.xul

You can grab the source code for the demos here:
http://joehewitt.com/mozilla/canvas-demos.zip

Comment 4

16 years ago
I just tried out your build and the testcases. This is AWESOME! (Especially the
seizure testcase)

Comment 5

16 years ago
Just adding myself to CC, gonna really look later.
You posted that this is a RGB image? Could this be a RGBA image, too?
That would be really nifty. (And it's gonna be slow, too;-))

Axel
hewitt: why <xul:canvas/> instead of display:-moz-xul-canvas ?

Comment 7

16 years ago
is there a reason you have the NPL header copyright 1998 at the top of all the
new files?
Keywords: 4xp, mozilla0.9.5, nsbeta1, nsbranch+, nsCatFood, regression, testcase, top100
QA Contact: jrgm → tpreston
Summary: create xul canvas tag for custom painting control → Flickering images on mouseover
Whiteboard: [PDT], [ETA 10/1]

Comment 8

16 years ago
uh.  mozilla is doing odd things.  ignore my previous comment

Comment 9

16 years ago
bah. stupid session history.
QA Contact: tpreston → jrgm
Keywords: 4xp, mozilla0.9.5, nsbeta1, nsbranch+, nsCatFood, regression, testcase, top100
Summary: Flickering images on mouseover → create xul canvas tag for custom painting control
Whiteboard: [PDT], [ETA 10/1]

Comment 10

16 years ago
test cases work great.  cant wait to see this on mac and linux.
(Reporter)

Updated

16 years ago
Target Milestone: mozilla0.9.7 → mozilla1.1

Comment 11

15 years ago
Joe, any updates on this?  Would it be possible to make this an .xpi ?

Comment 12

15 years ago
Can someone post the demos that were linked from this bug? That link no longer
works. Thanks.

Comment 13

14 years ago
Are there any plans for continuing work on this? Target Milestone was set to
1.1Alpha, and we're at 1.4RC1 now...

I think a xul canvas element would greatly enhance Mozilla-the-platform's value
as a base for applications. For example, the people at http://games.mozdev.org
would certainly like it, and we at optimoz mouse gestures would be glad if we no
longer had to paint mouse trails by inserting DIVs via DOM...

Comment 14

14 years ago
I ported the code to work in the latest 1.5 builds, but nothing displays on the
screen. Maybe I am doing something wrong (but I can't imagine what).

Joe, do you still have the examples?

Comment 15

14 years ago
I am just going to take this bug. I think my problem is something with the
drawing code, they changed a few things in the last 2 years :-)

Since this would (probably) never be allowed to become a real XUL element, I
will try to just make this a scriptable component and an XBL that can be
installed through an XPI. When I get this working I'll just throw it on mozdev.

I want to use this thing for a game too :-P
Assignee: hewitt → ericmurphy
Status: ASSIGNED → NEW
(Reporter)

Comment 16

14 years ago
Created attachment 129486 [details]
canvas demos

Here are the old canvas demos, I found them on a backup CD.  Hope they are of
some assistance.

Comment 17

14 years ago
Thanks Joe, they have/will help. I've got some drawing happening on the screen
now, but the colors don't seem to be working.

Anyways, back to work...

Comment 18

14 years ago
Eric, speaking of colors: it would be nice if the canvas allowed basic
transparency, so one can stack it upon other elements that shine through
"unpainted" areas of the canvas (no immediate need for sophisticated full alpha
channel support - I guess that will be too slow anyway).

If you need help, I'll do what I can (mainly qa/testcases and JS/XUL stuff I
guess, did not hack mozilla cpp yet).

Comment 19

14 years ago
Just to post an update... I had my hard drive die on me at the wrong time,
losing the code that I ported up to work in 1.5 code :-( However, much of that
code that I changed has little to do with making canvas it's own component. No
progress was really lost, but this has all set me back time-wise.

I am setting a project up at canvas.mozdev.org soon. I'll probably just close
out this bug soon and all code and updates will be posted there.

Comment 20

14 years ago
Eric, does that mean it won't ever be included in the trunk? I think it would be
best if the canvas would be in the official builds...

Comment 21

14 years ago
That will probably never happen. It's not a XUL element, and is a non-standard
element. Maybe if it gets a good response (when it's done), I'll suggest it to
whoever will listen to be added to the builds.

Comment 22

14 years ago
retargeting
Target Milestone: mozilla1.1alpha → Future

Updated

14 years ago
Severity: normal → enhancement
Neal Deakin <enndeakin@sympatico.ca> has created a resurrected version of this
patch at http://www.xulplanet.com/ndeakin/tests/canvas/canvas-patch-main.txt

I will be taking on some crash and stability cleanup of that patch, and then
attempting the proper reviews to get it landed. Some particular changes have
been suggested by Roc:

Remove the lock/unlockPixels methods, and change the setPixel methods to
setPixelRow to set an entire row at a time. He wants to use an octet-triple
array, while I'm still thinking a nscolor (0xRRGGBBAA) array would be easier to
construct, especially if you're drawing solid-color areas. In addition, I might
be able to get a JS-specialty method that draws pixels from a two-dimensional
array (it's not representable in IDL, but using ClassInfo and a scriptablehelper
it should be possible).

Remove the set/getColorString param, which is just redundant.

Remove the cached option entirely. Roc is hoping to get rid of the
offscreen-drawing-surface impl altogether, and unless someone presents an killer
use-case for a cached rendering context we'll just use the render-as-needed
approach.

I'm adding text-measurement APIs to the scriptable rendering context, and making
sure that all coordinates are nscoord (signed long) IDL typed. Do I need to add
pixel-to-twip conversion factors 
Assignee: emurphy79 → bsmedberg
Benjamin, can you say more about what that JS specialty method would look like?
Maybe we *should* extend IDL a bit (it's about time! ;-).

/be
Having a cached surface is useful for drawing image-like things, or doing
animation where the changes from one frame to the next are minimal. Having to
call to JS to redraw the contents every repaint can be slow. The cached version
doesn't need to call to JS.

Instead of a buffer, the actual image classes could be used instead, but those
are awkward to draw to from scripts. Perhaps images could be made to implement
nsIScriptableRenderingContext one day.
> Benjamin, can you say more about what that JS specialty method would look like?
> Maybe we *should* extend IDL a bit (it's about time! ;-).

If we're going for the "idl-extension" route, support for n-dimensional arrays
of fixed size in all three dimensions, such:

  void setPixels(in nscoord x, in nscoord y,
                 in unsigned long width, in unsigned long height,
                 [array(2), size_is(height, width)] in gfx_color pixels);

if we're talking about the scriptable-helper method, the IDL definition would be
empty: void setPixels(), and the scriptable rendering context would implement
nsIXPCScriptable::Call and manually handle the JS call context which would have
a two-dimension array object passed in.

Note that this discussion is somewhat theoretical; I'm not going to do it in a
first-revision patch in any case ;)  My xpconnect/JS FU is not that good.
I guess I can live with the buffer. The user specifies at the start whether to
use retained mode or not? How do you compute the invalid rect/region in retained
mode?

I personally don't feel this is the time or the way to extend IDL...

Regarding choice of units, first read http://ocallahan.org/mozilla/units.html to
understand some issues, if you haven't already I think the way to go here is to
specify that all units are CSS pixels. [In a future revision of this API we can
add tranforms, then users will be able to apply a scaling-down transform and
work in a higher resolution space to get better visual results for high-res
devices.] So you may want to have a function to return the number of CSS pixels
per inch.
Since scaling may be involved, we probably want a single function that can paint
an entire rectangle of pixels, not just one row like I suggested before. Maybe a
good way to go would be to just provide an API that can take an RGB array and
turn it into an image, and then the user can call drawImage etc. An additional
API to use RGBA could be provided too.

Another thing to think about is transparency. I suggest not providing
alpha-aware APIs at this time since we don't have good support yet (except for
in images). However, you want the canvas to be able to be transparent so you can
see what's behind it. This is trivial in immediate mode (unbuffered). In
retained mode you really want the buffer to be RGBA (along with a function to
set pixels to transparent) but we don't support that well yet. An interim
solution would be to specify a background color when you enter retained mode,
set the buffer to that color, and for now don't support transparent pixels in
retained mode.
Come to think of it, you don't need an IDL extension to do n-dimensional arrays;
the IDL could be prototyped thus:

void drawPixels(in nscoord x, in nscoord y,
                in unsigned long columnCount, in unsignedLong pixelCount,
                [array, size_is(numPixels)] in nscolor);

So the number of rows would be pixelCount/columnCount.

roc: why is (octet, octet, octet) better than a single 32-bit value RRGGBBAA (we
typedef gfx_color == nscolor to "unsigned long" in IDL to make it scriptable)? 
IMO the single value would be a lot more efficient from an xpconnect perspective
(and allow aligned 32-bit reads from the c++ side), and would allow the API to
do transparency whenever it becomes available without changing the interface.

Comment 30

14 years ago
Robert O'Callahan wrote:
> Another thing to think about is transparency. I suggest not providing
> alpha-aware APIs at this time since we don't have good support yet (except for
> in images).

Maybe we even need support for a "capabilities" API that XUL applications can
query whether alpha channel is supported or not. Remember that Mozilla's
PostScript print module doesn't support an alpha channel nor offscreen surfaces
nor copying of surface data so we may need such an API (since I doubt that the
PostScript module will ever be redesigned to support this).
Agree with bsmedberg on 32-bit color 4-tuples over octet array madness.  Roc,
what are we missing?

Always glad to avoid IDL extensions -- I was just asking.  But since we're being
touted as a platform, lack of structs does hurt us compared to MS COM.  Who
cares, given .NET?  XPCOM cares, it has a long life ahead of it.

Roland, I've seen too many capability-menu APIs (PHIGS, anyone) go south because
apps have to write combinatorially explosive code querying all the capabilities
and implementing fallbacks for missing caps.  This sucks, and leads to one set
of "required default" caps emerging -- so why waste time and space on the menu?

/be

Comment 32

14 years ago
brendan@mozilla.org  wrote:
> Roland, I've seen too many capability-menu APIs (PHIGS, anyone) go south 
> because apps have to write combinatorially explosive code querying all the 
> capabilities and implementing fallbacks for missing caps.  This sucks, and 
> leads to one set of "required default" caps emerging -- so why waste time and
> space on the menu?

... because we have a certain platform with a certain renderer which doesn't
support that feature... ;-/
I am only asking for a way to query the capabilities. Applications should be
free to ignore it but if an application has the ability to use an alternative
path in that case we would need a query mechanism that the application can
distinguish between both cases.

Comment 33

14 years ago
On the capability menu API point -- while I agree w/ Brendan that it's 
traditionally been a pain, I have also seen recent stuff which seems to 
indicate that there is a great deal of overlap in 'base functionality'.  
Things like transparency and rotated text are now almost the only things that 
you'll find in some backends and not others that I would consider part of the 
scope of a XUL painting control (color blending models etc. I consider out of 
scope).

(See e.g. http://www.scipy.org/site_content/chaco/intro for a description of a 
fairly rich graphics package based on this model).

--david ascher

Comment 34

14 years ago
David Ascher wrote:
> On the capability menu API point -- while I agree w/ Brendan that it's 
> traditionally been a pain,

How do you suggest should the issue be solved then ?
I doubt that anyone will step-up and add alpha channel support to the PostScript
print module. As long we have such problems we should provide a way to detect
the issue and work around it.
There are ways to work around lack of transparency support in Postscript. They
involve ugly client side rendering, but they work. For now Poscript will simply
ignore alpha values and render incorrectly (or rather, as best it can);
eventually we'll hopefully have a GFX backend based on Cairo or PDF or something
similar which can compensate for Postscript's deficiencies.

I don't mind doing platform capabilities-based stuff in Gecko. I do mind
exposing platform capabilities to Web clients and XUL users. Degradation to
"best effort" rendering is something that should be taken care of by Gecko and
not forced onto users and authors.

> But since we're being touted as a platform, lack of structs does hurt us
> compared to MS COM.

Structs are much more interesting than multidimensional arrays. But do we have a
compelling use case for enhancing XPCOM today? I don't see it. I don't think
rich IDLs are the way to go ultimately although I disagree with Microsoft about
the best way to avoid that. But this is a discussion for another time and place.

> Agree with bsmedberg on 32-bit color 4-tuples over octet array madness.  Roc,
> what are we missing?

I was thinking that packing RGBA values into a single 32bit word with shifts and
masks could be done more efficiently on the C++ side than on the Javascript
side. But it occurs to me that many clients will be able to form a single 32bit
value and then use it repeatedly. So I'll yield on this one :-).

Comment 36

14 years ago
Neil, do you still work on the canvas stuff? If so, what's the status?

Comment 37

13 years ago
Benjamin, any news on the implementation? It would be very sad if this
disappeared again just after being "resurrected" with a working patch...

Comment 38

13 years ago
(In reply to comment #37)
> It would be very sad if this disappeared again just after being "resurrected" 
> with a working patch...

Since the last (public) patch dates back to Dec 3, 2003, this is probably the case.

Prog.
Alias: canvas

Comment 39

13 years ago
Just a datapoint: I tried the one in Safari 2.0 (172) included in recent Tiger
prerelease builds (8A323), and it does seem to work fine with the basic things I
tried (several hundred animated rectangles of various colors).    
There's an entirely new canvas implementation coming, based on Cairo, with much
more functionality. And it's Apple compatible.
Assignee: benjamin → vladimir

Comment 41

13 years ago
(In reply to comment #40)
> There's an entirely new canvas implementation coming, based on Cairo, with much
> more functionality. And it's Apple compatible.

Is 'new canvas implementation' bug 288714 ?

Comment 42

12 years ago
Yes and as that can be used in XUL as well using the XHTML namespace I think
this bug can be resolved, not?
Not yet; the other canvas contains just a simple HTML drawing surface element,
as defined by WHATWG/Apple.  Neil had some great ideas about how a canvas would
fit well into XUL, and those should still be implemented for <xul:canvas>.  It
can share the back-end rendering contexts with the HTML <canvas>, while adding
child element positioning and other similar features that are of more interest
in XUL.
Component: XP Toolkit/Widgets → Layout: Canvas
QA Contact: jrgmorrison → ian

Comment 44

12 years ago
I don't understand why this is not a duplicate of bug 288714.  Could someone explain?  (I would think that it would be confusing if xul:canvas is wildly different from html:canvas.)
Mainly history.

*** This bug has been marked as a duplicate of 288714 ***
Status: NEW → RESOLVED
Last Resolved: 12 years ago
Resolution: --- → DUPLICATE

Comment 46

12 years ago
how is this bug resolved? there isn't a xul:canvas tag.
Use html:canvas
You need to log in before you can comment on or make changes to this bug.