Closed Bug 102285 (canvas) Opened 23 years ago Closed 19 years ago

create xul canvas tag for custom painting control

Categories

(Core :: Graphics: Canvas2D, enhancement, P3)

Product:
Core
Component:
Graphics: Canvas2D
Type:
enhancement

Tracking

()

Status:
RESOLVED DUPLICATE of bug 288714
Milestone:
Future

People

(Reporter: hewitt, Assigned: vlad)

Details

Attachments

(2 files)

patch with initial windows implementation
36.19 KB, patch
Details | Diff | Splinter Review
canvas demos
12.00 KB, application/x-zip-compressed
Details 
A few weeks ago I spent a weekend getting this up and running on windows.  Now I
need to get it working on mac and linux and check it in.  I'd appreciate as much
input on the design as possible.  I'll post my patch asap.
Status: NEW → ASSIGNED
Priority: -- → P3
Target Milestone: --- → mozilla0.9.7
Random platform buddies and rendering fanboys.

/be
For those who would like to test this out, I've zipped up a windows build for
your amusement.  Download the build from here:
http://joehewitt.com/mozilla/canvas.zip

I've written two demos: one is a little paint program written entirely 
in javascript, and the other is a color-blending animation written with 
javascript and C++, which re-paints every pixel of an image 
continuously at about 60 frames per second.

The demos are included in the test build.  To see them, browse to:
chrome://hewitt/content/photoshop.xul
chrome://hewitt/content/seizure.xul

You can grab the source code for the demos here:
http://joehewitt.com/mozilla/canvas-demos.zip


I just tried out your build and the testcases. This is AWESOME! (Especially the
seizure testcase)
Just adding myself to CC, gonna really look later.
You posted that this is a RGB image? Could this be a RGBA image, too?
That would be really nifty. (And it's gonna be slow, too;-))

Axel
hewitt: why <xul:canvas/> instead of display:-moz-xul-canvas ?
is there a reason you have the NPL header copyright 1998 at the top of all the
new files?
URL: http://www.payplus.at/footer.asp
Keywords: 4xp, mozilla0.9.5, nsbeta1, nsbranch+, nsCatFood, regression, testcase, top100
QA Contact: jrgm → tpreston
Summary: create xul canvas tag for custom painting control → Flickering images on mouseover
Whiteboard: [PDT], [ETA 10/1]
uh.  mozilla is doing odd things.  ignore my previous comment
bah. stupid session history.
URL: http://www.payplus.at/footer.asp
Keywords: 4xp, mozilla0.9.5, nsbeta1, nsbranch+, nsCatFood, regression, testcase, top100
QA Contact: tpreston → jrgm
Summary: Flickering images on mouseover → create xul canvas tag for custom painting control
Whiteboard: [PDT], [ETA 10/1]
test cases work great.  cant wait to see this on mac and linux.
Target Milestone: mozilla0.9.7 → mozilla1.1
Joe, any updates on this?  Would it be possible to make this an .xpi ?
Can someone post the demos that were linked from this bug? That link no longer
works. Thanks.
Are there any plans for continuing work on this? Target Milestone was set to
1.1Alpha, and we're at 1.4RC1 now...

I think a xul canvas element would greatly enhance Mozilla-the-platform's value
as a base for applications. For example, the people at http://games.mozdev.org
would certainly like it, and we at optimoz mouse gestures would be glad if we no
longer had to paint mouse trails by inserting DIVs via DOM...
I ported the code to work in the latest 1.5 builds, but nothing displays on the
screen. Maybe I am doing something wrong (but I can't imagine what).

Joe, do you still have the examples?
I am just going to take this bug. I think my problem is something with the
drawing code, they changed a few things in the last 2 years :-)

Since this would (probably) never be allowed to become a real XUL element, I
will try to just make this a scriptable component and an XBL that can be
installed through an XPI. When I get this working I'll just throw it on mozdev.

I want to use this thing for a game too :-P
Assignee: hewitt → ericmurphy
Status: ASSIGNED → NEW
Attached file canvas demos — 
Here are the old canvas demos, I found them on a backup CD.  Hope they are of
some assistance.
Thanks Joe, they have/will help. I've got some drawing happening on the screen
now, but the colors don't seem to be working.

Anyways, back to work...
Eric, speaking of colors: it would be nice if the canvas allowed basic
transparency, so one can stack it upon other elements that shine through
"unpainted" areas of the canvas (no immediate need for sophisticated full alpha
channel support - I guess that will be too slow anyway).

If you need help, I'll do what I can (mainly qa/testcases and JS/XUL stuff I
guess, did not hack mozilla cpp yet).
Just to post an update... I had my hard drive die on me at the wrong time,
losing the code that I ported up to work in 1.5 code :-( However, much of that
code that I changed has little to do with making canvas it's own component. No
progress was really lost, but this has all set me back time-wise.

I am setting a project up at canvas.mozdev.org soon. I'll probably just close
out this bug soon and all code and updates will be posted there.
Eric, does that mean it won't ever be included in the trunk? I think it would be
best if the canvas would be in the official builds...
That will probably never happen. It's not a XUL element, and is a non-standard
element. Maybe if it gets a good response (when it's done), I'll suggest it to
whoever will listen to be added to the builds.
retargeting
Target Milestone: mozilla1.1alpha → Future
Severity: normal → enhancement
Neal Deakin <enndeakin@sympatico.ca> has created a resurrected version of this
patch at http://www.xulplanet.com/ndeakin/tests/canvas/canvas-patch-main.txt

I will be taking on some crash and stability cleanup of that patch, and then
attempting the proper reviews to get it landed. Some particular changes have
been suggested by Roc:

Remove the lock/unlockPixels methods, and change the setPixel methods to
setPixelRow to set an entire row at a time. He wants to use an octet-triple
array, while I'm still thinking a nscolor (0xRRGGBBAA) array would be easier to
construct, especially if you're drawing solid-color areas. In addition, I might
be able to get a JS-specialty method that draws pixels from a two-dimensional
array (it's not representable in IDL, but using ClassInfo and a scriptablehelper
it should be possible).

Remove the set/getColorString param, which is just redundant.

Remove the cached option entirely. Roc is hoping to get rid of the
offscreen-drawing-surface impl altogether, and unless someone presents an killer
use-case for a cached rendering context we'll just use the render-as-needed
approach.

I'm adding text-measurement APIs to the scriptable rendering context, and making
sure that all coordinates are nscoord (signed long) IDL typed. Do I need to add
pixel-to-twip conversion factors
Assignee: emurphy79 → bsmedberg
Benjamin, can you say more about what that JS specialty method would look like?
Maybe we *should* extend IDL a bit (it's about time! ;-).

/be
Having a cached surface is useful for drawing image-like things, or doing
animation where the changes from one frame to the next are minimal. Having to
call to JS to redraw the contents every repaint can be slow. The cached version
doesn't need to call to JS.

Instead of a buffer, the actual image classes could be used instead, but those
are awkward to draw to from scripts. Perhaps images could be made to implement
nsIScriptableRenderingContext one day.

> Benjamin, can you say more about what that JS specialty method would look like?
> Maybe we *should* extend IDL a bit (it's about time! ;-).

If we're going for the "idl-extension" route, support for n-dimensional arrays
of fixed size in all three dimensions, such:

  void setPixels(in nscoord x, in nscoord y,
                 in unsigned long width, in unsigned long height,
                 [array(2), size_is(height, width)] in gfx_color pixels);

if we're talking about the scriptable-helper method, the IDL definition would be
empty: void setPixels(), and the scriptable rendering context would implement
nsIXPCScriptable::Call and manually handle the JS call context which would have
a two-dimension array object passed in.

Note that this discussion is somewhat theoretical; I'm not going to do it in a
first-revision patch in any case ;)  My xpconnect/JS FU is not that good.
I guess I can live with the buffer. The user specifies at the start whether to
use retained mode or not? How do you compute the invalid rect/region in retained
mode?

I personally don't feel this is the time or the way to extend IDL...

Regarding choice of units, first read http://ocallahan.org/mozilla/units.html to
understand some issues, if you haven't already I think the way to go here is to
specify that all units are CSS pixels. [In a future revision of this API we can
add tranforms, then users will be able to apply a scaling-down transform and
work in a higher resolution space to get better visual results for high-res
devices.] So you may want to have a function to return the number of CSS pixels
per inch.
Since scaling may be involved, we probably want a single function that can paint
an entire rectangle of pixels, not just one row like I suggested before. Maybe a
good way to go would be to just provide an API that can take an RGB array and
turn it into an image, and then the user can call drawImage etc. An additional
API to use RGBA could be provided too.

Another thing to think about is transparency. I suggest not providing
alpha-aware APIs at this time since we don't have good support yet (except for
in images). However, you want the canvas to be able to be transparent so you can
see what's behind it. This is trivial in immediate mode (unbuffered). In
retained mode you really want the buffer to be RGBA (along with a function to
set pixels to transparent) but we don't support that well yet. An interim
solution would be to specify a background color when you enter retained mode,
set the buffer to that color, and for now don't support transparent pixels in
retained mode.
Come to think of it, you don't need an IDL extension to do n-dimensional arrays;
the IDL could be prototyped thus:

void drawPixels(in nscoord x, in nscoord y,
                in unsigned long columnCount, in unsignedLong pixelCount,
                [array, size_is(numPixels)] in nscolor);

So the number of rows would be pixelCount/columnCount.

roc: why is (octet, octet, octet) better than a single 32-bit value RRGGBBAA (we
typedef gfx_color == nscolor to "unsigned long" in IDL to make it scriptable)? 
IMO the single value would be a lot more efficient from an xpconnect perspective
(and allow aligned 32-bit reads from the c++ side), and would allow the API to
do transparency whenever it becomes available without changing the interface.
Robert O'Callahan wrote:
> Another thing to think about is transparency. I suggest not providing
> alpha-aware APIs at this time since we don't have good support yet (except for
> in images).

Maybe we even need support for a "capabilities" API that XUL applications can
query whether alpha channel is supported or not. Remember that Mozilla's
PostScript print module doesn't support an alpha channel nor offscreen surfaces
nor copying of surface data so we may need such an API (since I doubt that the
PostScript module will ever be redesigned to support this).
Agree with bsmedberg on 32-bit color 4-tuples over octet array madness.  Roc,
what are we missing?

Always glad to avoid IDL extensions -- I was just asking.  But since we're being
touted as a platform, lack of structs does hurt us compared to MS COM.  Who
cares, given .NET?  XPCOM cares, it has a long life ahead of it.

Roland, I've seen too many capability-menu APIs (PHIGS, anyone) go south because
apps have to write combinatorially explosive code querying all the capabilities
and implementing fallbacks for missing caps.  This sucks, and leads to one set
of "required default" caps emerging -- so why waste time and space on the menu?

/be
brendan@mozilla.org  wrote:
> Roland, I've seen too many capability-menu APIs (PHIGS, anyone) go south 
> because apps have to write combinatorially explosive code querying all the 
> capabilities and implementing fallbacks for missing caps.  This sucks, and 
> leads to one set of "required default" caps emerging -- so why waste time and
> space on the menu?

... because we have a certain platform with a certain renderer which doesn't
support that feature... ;-/
I am only asking for a way to query the capabilities. Applications should be
free to ignore it but if an application has the ability to use an alternative
path in that case we would need a query mechanism that the application can
distinguish between both cases.
On the capability menu API point -- while I agree w/ Brendan that it's 
traditionally been a pain, I have also seen recent stuff which seems to 
indicate that there is a great deal of overlap in 'base functionality'.  
Things like transparency and rotated text are now almost the only things that 
you'll find in some backends and not others that I would consider part of the 
scope of a XUL painting control (color blending models etc. I consider out of 
scope).

(See e.g. http://www.scipy.org/site_content/chaco/intro for a description of a 
fairly rich graphics package based on this model).

--david ascher
David Ascher wrote:
> On the capability menu API point -- while I agree w/ Brendan that it's 
> traditionally been a pain,

How do you suggest should the issue be solved then ?
I doubt that anyone will step-up and add alpha channel support to the PostScript
print module. As long we have such problems we should provide a way to detect
the issue and work around it.
There are ways to work around lack of transparency support in Postscript. They
involve ugly client side rendering, but they work. For now Poscript will simply
ignore alpha values and render incorrectly (or rather, as best it can);
eventually we'll hopefully have a GFX backend based on Cairo or PDF or something
similar which can compensate for Postscript's deficiencies.

I don't mind doing platform capabilities-based stuff in Gecko. I do mind
exposing platform capabilities to Web clients and XUL users. Degradation to
"best effort" rendering is something that should be taken care of by Gecko and
not forced onto users and authors.

> But since we're being touted as a platform, lack of structs does hurt us
> compared to MS COM.

Structs are much more interesting than multidimensional arrays. But do we have a
compelling use case for enhancing XPCOM today? I don't see it. I don't think
rich IDLs are the way to go ultimately although I disagree with Microsoft about
the best way to avoid that. But this is a discussion for another time and place.

> Agree with bsmedberg on 32-bit color 4-tuples over octet array madness.  Roc,
> what are we missing?

I was thinking that packing RGBA values into a single 32bit word with shifts and
masks could be done more efficiently on the C++ side than on the Javascript
side. But it occurs to me that many clients will be able to form a single 32bit
value and then use it repeatedly. So I'll yield on this one :-).
Neil, do you still work on the canvas stuff? If so, what's the status?
Benjamin, any news on the implementation? It would be very sad if this
disappeared again just after being "resurrected" with a working patch...
(In reply to comment #37)
> It would be very sad if this disappeared again just after being "resurrected" 
> with a working patch...

Since the last (public) patch dates back to Dec 3, 2003, this is probably the case.

Prog.
Just a datapoint: I tried the one in Safari 2.0 (172) included in recent Tiger
prerelease builds (8A323), and it does seem to work fine with the basic things I
tried (several hundred animated rectangles of various colors).    
There's an entirely new canvas implementation coming, based on Cairo, with much
more functionality. And it's Apple compatible.
Assignee: benjamin → vladimir
(In reply to comment #40)
> There's an entirely new canvas implementation coming, based on Cairo, with much
> more functionality. And it's Apple compatible.

Is 'new canvas implementation' bug 288714 ?

Yes and as that can be used in XUL as well using the XHTML namespace I think
this bug can be resolved, not?
Not yet; the other canvas contains just a simple HTML drawing surface element,
as defined by WHATWG/Apple.  Neil had some great ideas about how a canvas would
fit well into XUL, and those should still be implemented for <xul:canvas>.  It
can share the back-end rendering contexts with the HTML <canvas>, while adding
child element positioning and other similar features that are of more interest
in XUL.
Component: XP Toolkit/Widgets → Layout: Canvas
QA Contact: jrgmorrison → ian
I don't understand why this is not a duplicate of bug 288714.  Could someone explain?  (I would think that it would be confusing if xul:canvas is wildly different from html:canvas.)
Mainly history.

*** This bug has been marked as a duplicate of 288714 ***
Status: NEW → RESOLVED
Closed: 19 years ago
Resolution: --- → DUPLICATE
how is this bug resolved? there isn't a xul:canvas tag.
Use html:canvas
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: