Closed Bug 206347 Opened 21 years ago Closed 19 years ago

Need ability to paint a document to an arbitrary device

Categories

(Core :: Layout, defect)

Product:
Core
Component:
Layout
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WONTFIX

People

(Reporter: john, Assigned: john)

References

Details

Attachments

(2 files)

Patch
65.61 KB, patch
Details | Diff | Splinter Review
mfcembed usage patch
14.12 KB, patch
Details | Diff | Splinter Review 
For the new printing architecture and to generate thumbnails, as well as to
perform good pixel-presentation regression tests, we need an easy way to take a
document and paint it to a device.  The main way of doing it, dumping to the
screen, is cumbersome and involves a lot of initialization code that is spread
throughout the application.  The alternate way, the current printing
architecture, uses a different paginated reflow altogether but does contain much
of the necessary initialization code.  Something new is needed.  There should be
an easy API to:

1. Create a viewport for a document with a width and height
2. Reflow to the viewport using a particular device to get font info
3. Scroll the document or do other common operations on the viewport
4. Paint the viewport onto a device

The new printing architecture (bug 178292) that I am working on uses this as the
basis for actually printing pages, and thumbnails (bug 109196, bug 125998, bug
143697) could use this to generate the image they need to scale down. 
Pixel-accurate regression tests (no bug filed for that yet) could use this to
generate an image to dump to a file.  (We probably need an easy way to do this,
too ... creating a native image and device context for that image might be best
done at or near the level of the image library, which IIRC holds a native bitmap
anyway.)

One hopes that this API could in the future be used internally by Gecko to
create the viewport / presentation for the document and have only one (simple)
initialization route for the presentation.  I have a patch here, I'll upload
shortly.
I don't know what you're planning to do about units, but we worked out where we
want to get to and I'll be interested to see whether this work gets us closer or
further away.

One thing in particular is that ultimately we don't want font size prefs to be
influenced by the device. Font size prefs should be in app units (some fixed
fraction of a CSS pixel, say 1/60) which are device independent. The device will
give you a mapping from physical units (inches, cm, etc) to app units, and also
map app units to device units just as it does today.
This work shouldn't get us closer to or farther from "fix the use of units in
Gecko."  The interfaces specifically avoid the issue by specifying things in
device units, since for images and screen that is what the user of the interface
is thinking and working with anyway; for a printer it's arguable twips is better
but that is because we convert internally to twips in
GetDeviceSurfaceDimensions()--it is arguable we should be able to work with a
device in its own units, which we don't currently usually do.

This does mean that a conversion is done from device to app units--from the
external interface to the device to the internal units gecko uses.  When "fix
the use of units in Gecko" is fixed, this may or may not have to change, but
it's a small deal.
Attached patch Patch — — Splinter Review 
This patch includes the interface, the main implementation, and makes it work
on Windows.  It does not yet work on other platforms.  It does not include the
actual thumbnail generation (which can be added much more easily using this
interface).  My printing code is already building off of this and performing
basic printing.

What it does:

(1) Creates nsIEmbedPresentation and nsIPresentation interfaces that generally
represent a presentation, and nsIEmbedPresentationGenerator /
nsIPresentationGenerator to generate them.  I created these specifically to
simplify the job of working with a presentation from an embedder standpoint.  I
expect people to be able to create standalone programs that, for example, take
a URL and generate a bitmap from it using Gecko.  Or perhaps render a Gecko
window onto a bitmap for an OpenGL texture.  Or render into a part of their
window.  I think for now we could combine the interfaces into one since there
are no embedders using them, but I was trying to limit nsIEmbed* to frozen
interfaces and needed the non-frozen ones for printing (to work with an
nsIDeviceContext basically, which is not frozen and probably should not be).

(2) Fixes the DocumentViewer/PresShell/ViewManager ownership model problem
noted in bug 202029.  roc, I am aware that this is not necessary if we remove
multiple presentations, but I'd really like us not to open that can of worms
just yet--one step at a time.  I specifically designed these interfaces so that
they do not have to create multiple presentations against the same identical
document to do their job.  And when printing converts over, all
multiple-presentation-using code will go through this interface, so we will
only have one place to convert (and it *will* be easier to convert).

(3) adds GetCanvasFrame() to nsIPresShell (which is useful for other places
than just here already, and can be used by the new printing arch too)

(4) makes it possible to construct an nsIDeviceContext from a native DC alone
(before it was only possible to construct from a window) using the InitFromDC()
method.  This makes this stuff cross-platform, pretty much, excepting that
InitFromDC() is only implemented on Windows at the moment--it's something to be
implemented on each platform, which will take some time on each one.  I had to
do a little bit of hacking on the device and rendering contexts, and especially
I had to fix the rendering context's Init() so that, if the nsIDeviceContext
does not support native widgets (and therefore cannot get a DC from a native
widget), it will use the DC directly from the nsIDeviceContext.  This, of
course, is for windowless device contexts like the one we are constructing.
This patch adds a File > View Thumbnail menu item to mfcembed which blits the
thumbnail to a bitmap, creates a new window with a smaller size and shrinks the
bitmap onto it (it could just as easily shrink the bitmap onto another bitmap).
Another thing this could be used for, which I have heard several people muse
about (kmcclusk and sfraser), is speeding up our Back/Forward to almost nil by
storing a bitmap of the front page in the history entry and using that while we
reflow and before we paint.  Fastest browser in the West!
Note that we already have a bug somewhere on being able to, eg, print without X
running.  Not sure whether this change affects that bug at all, but if it does,
you may want to let the people working on it know...
The guy working on it is gisburn, and it will *probably* work if he can just . 
I am, however, designing the new printing architecture with that sort of problem
in mind as well.  The basic idea is, we have four things which an embedder might
work with and combine to present and work with a document: document, shell,
presentation and device.  The document is the document.  The shell is the script
global object and the window in which a document can be embedded.  The
presentation is the document-plus-Gecko-viewport, which represents the reflowed
document that can be painted onto a device.  The device is the screen, printer
of image.

Printing then becomes a matter of:
(1) get a document and shell (for now this is done by having an already-loaded
Gecko window, but if we can get to the point where it is easy to do this without
having or creating a real window this would be great).
(2) get the printer device
(3) create a presentation against the printer device
(4) scroll clip paint, scroll clip paint, ...

Thumbnails are a matter of:
(1) get document and shell
(2) create a presentation against the screen device
(3) create a bitmap the proper size for the reflowed presentation (height may be
different than you expected)
(4) paint to bitmap

The thumbnail example is the mfcembed example posted above.
Heh, I meant to finish that first sentence.  The guy working on it is gisburn,
and it will *probably* work if he can just get a document and shell up and
running properly, which is proving to be problematic.
> speeding up our Back/Forward to almost nil by storing a bitmap of the front page
> in the history entry

I believe IE5/Mac does this.

> roc, I am aware that this is not necessary if we remove multiple presentations,
> but I'd really like us not to open that can of worms just yet--one step at a
> time.

Sure, that's fine.

> makes it possible to construct an nsIDeviceContext from a native DC alone

Aren't nsIDeviceContexts immutable? Can't we just have a GetDeviceContext method
on nsIRenderingContext? I think you can already construct an nsIRenderingContext
given a Windows DC.
> Aren't nsIDeviceContexts immutable? Can't we just have a GetDeviceContext method
> on nsIRenderingContext? I think you can already construct an nsIRenderingContext
> given a Windows DC.

InitFromDC() is an alternative to Init(), which is a step in the *creation* of
the DeviceContext.  nsIDeviceContexts are basically immutable but already can
hold an HDC.  It is necessary to do this to be able to print to windowless device 

As to GetDeviceContext() on nsIRenderingContext, the device context represents
the device, font information and scaling, which you get from the HDC on some
platforms, and the nsNativeDeviceContext can already be stored in the
DeviceContext--this really is the appropriate place to put the device context
initialization.  Further, rendering contexts are kinda throwaway classes that
get created on the fly and destroyed every time, so unless you're rendering
right away it's useless to create one.  The HWND is only there in the device
context to *get* the HDC anyway (and to do multi-head stuff apparently, which if
it really requires an HWND ought to be done in the widget), so really the device
context just represents the HDC.
Comment on attachment 123758 [details] [diff] [review]
Patch

roc, would you mind reviewing?	This seems more up your alley than anyone else,
and it's the sort of patch that probably requires a couple of sr's to look at.

I forgot one more thing I did with the patch: I disabled scrollbars entirely in
print contexts to avoid another instance of the "anonymous scrollbars get
created in the screen pres context whenever you print" problem.
Attachment #123758 - Flags: review?(roc+moz)
Sure. Can this wait till after 1.4 ships or are you on some kind of alternative
timeline?

There's code in nsGfxScrollFrame::CreateAnonymousContent that disables
scrollbars for printing already. If that doesn't do what you want, why don't you
just whack that? Alternatively if your new mechanism is what you need, please
remove the code in nsGfxScrollFrame::CreateAnonymousContent.
I don't need it for 1.4, no.  I *would* like to get it in within the next two
weeks, because the printing stuff depends on it and may add more code that will
make the patch more confusing (the printing stuff will get more discussion
shortly, I'm writing up a more detailed design doc).

I'll whack the existing code to deal with extra scrollbar creation, my patch
probably does make it unnecessary.
Status: NEW → ASSIGNED
+  /**
+   * The final width of the document (should be the same as the viewport width),
+   * in device pixels
+   */

Are you sure? Because of absolute positioning and other things, layout can place
elements to the right of the width constraint.

+   * See http://bugscape.nscp.aoltw.net/show_bug.cgi?id=23810

Don't bother including this.

I assume that nsI*Presentation is intended to be entirely read-only, i.e., no
user interaction with the presented document. The comments in the header should
note this. In particular it needs to be clear that this interface is not to be
used by general-purpose browser embedders. Actually I think I'd prefer a
different name, say nsIDocumentRenderer or something like that.

How about we drop nsIPresentation.idl for now, until we've decided what's
happening with printing?

+#ifndef __EMBED_PRESENTATION_H
+#define __EMBED_PRESENTATION_H

Should be __PRESENTATION_H

Your nsCSSFrameConstructor changes don't turn off scrollbars for
listboxes/comboboxes. (I guess only listboxes matter because you can't popup a
combobox popup.) How are you planning to do that?

I don't much like your scrolling implementation that moves the canvas view; it
breaks view manager invariants that might be important. Can't clients just
render at a different offset with different clipping set? Why do they need this
nsIPresentation scrolling method?

In nsRenderingContextWin::Init
+    if (usenative) {
Shouldn't this be "if (!usenative)"?

I don't understand the gymnastics with nsIDeviceContext. Wouldn't it be more
natural to pass an nsIRenderingContext into the presentation and use that to
draw on?
> >* The final width of the document (should be the same as the viewport width),
> Are you sure? Because of absolute positioning and other things, layout can
> place elements to the right of the width constraint.

Since I'm grabbing the view bounds, it really should include the full potential
width, but I thought I remembered testing that.  Maybe it's a remnant from when
I was grabbing frame bounds, which would not include overflow area.  I will
retest and modify the comments as necessary.

> I assume that nsI*Presentation is intended to be entirely read-only, i.e., no
> user interaction with the presented document.  The comments in the header
> should note this. In particular it needs to be clear that this interface is
> not to be used by general-purpose browser embedders. Actually I think I'd
> prefer a different name, say nsIDocumentRenderer or something like that.

This is true at the moment, but I would like in the future to make Gecko itself
create the presentation with it (right now that logic is spread out all over the
code; this has the benefit of sewing it back together again).  It's not intended
for embedders though, no.

> How about we drop nsIPresentation.idl for now, until we've decided what's
> happening with printing?

Fine with me, the logic in this patch is the main point.

> Your nsCSSFrameConstructor changes don't turn off scrollbars for
> listboxes/comboboxes. (I guess only listboxes matter because you can't popup a
> combobox popup.) How are you planning to do that?

Actually I *prefer* to show scrollbars if possible for listboxes.  Otherwise
they just look like a <div> with a border.  Your recent changes made listboxes
not cause these problems though, IIRC.  I'll retest to make sure.

> I don't much like your scrolling implementation that moves the canvas view; it
> breaks view manager invariants that might be important. Can't clients just
> render at a different offset with different clipping set? Why do they need
> this nsIPresentation scrolling method?

I am trying as much as humanly possible to "paint from the root" here.  Actually
if I wasn't sometimes painting to a different DeviceContext we could probably
send a normal paint event and have it work perfectly.  If scrolling is doing
something wrong I'll fix it.

> In nsRenderingContextWin::Init
> +    if (usenative) {
> Shouldn't this be "if (!usenative)"?

Nope.  I think I got thrown by that at one point too.  usenative means "grab the
DC from the window using GetNativeData()".  I'll make the comments more verbose
here.

> I don't understand the gymnastics with nsIDeviceContext. Wouldn't it be more
> natural to pass an nsIRenderingContext into the presentation and use that to
> draw on?

I'm not entirely sure I understand your question: who would create the
nsIDeviceContext needed by the PresContext?  You still need one of those to
reflow.  Basically, my philosophy here is, the device context represents the
device and all relevant information including color depth and such.  The device
in this case is a bitmap.  We want the device context, therefore, to represent
the bitmap if possible.  My patch makes that possible, at least on Windows, by
allowing an nsIDeviceContext that does not have an HWND.

Also, currently the creation of an nsIRenderingContext is very
platform-specific:
http://lxr.mozilla.org/seamonkey/source/gfx/src/windows/nsDeviceContextWin.cpp#293
.  Whoever created the RenderingContext would have to use platform-specific
logic.  In an initial implementation of this I did that and it looked unwieldy;
my conclusion was that it was cleaner to leave platform-specific logic in gfx
and keep the interface as generic as possible.
(the other stuff I'll fix soon)
> Since I'm grabbing the view bounds, it really should include the full potential
> width

The canvas view bounds should give you the actual width of the document, yes, so
you are returning the correct document width. But the canvas can overflow the
viewport horizontally, so the comment is incorrect.

> Your recent changes made listboxes not cause these problems though

No, my recent changes turned off listboxes for printing and print preview. If
you create another presentation where listboxes have scrollbars again, you will
recreate the same bug(s) that I was trying to avoid (and as hyatt explained,
can't be ultimately be fixed without cloning the document).

> I am trying as much as humanly possible to "paint from the root" here.

What do you mean? As far as I can tell, the client can fake scrolling using the
painting APIs you already have.

>> In nsRenderingContextWin::Init
>> +    if (usenative) {
>> Shouldn't this be "if (!usenative)"?

Let me be more precise. in nsRenderingContextWin::~nsRenderingContextWin(),
     ::ReleaseDC((HWND)mDCOwner->GetNativeData(NS_NATIVE_WINDOW), mMainDC);
     NS_RELEASE(mDCOwner);
   }
+  else if (mContext)
+  {
+    PRBool usenative;
+    mContext->SupportsNativeWidgets(usenative);
+    if (usenative) {
+      ((nsDeviceContextWin*)mContext)->MaybeReleaseDC(mDC);
should be "!usenative".

I guess I don't understand precisely what nsIRenderingContext and
nsIDeviceContext are trying to abstract. I *thought* an nsIDeviceContext
represented the characteristics of a particular device --- dpi, colors,
available fonts, and so on, but was not specific to a particular drawing surface
such as a particular window or offscreen memory area.
> The canvas view bounds should give you the actual width of the document, yes,
> so you are returning the correct document width. But the canvas can overflow
> the viewport horizontally, so the comment is incorrect.

Yeah, that's what I was trying to say, guess I wasn't clear.  The view includes
the overflow area while the frame does not.  I'll retest and change the comment.

> > No, my recent changes turned off listboxes for printing and print preview.
> > If you create another presentation where listboxes have scrollbars again,
> > you will recreate the same bug(s) that I was trying to avoid (and as hyatt
> > explained, can't be ultimately be fixed without cloning the document).

Well, I will test listboxes and ensure that they don't cause this problem; it's
been months, so I don't know exactly what I tested and what not, but it would
have been silly of me not to test listboxes.  Maybe I was silly :)

If worse comes to worse, I'll be happy to clone the document, but that will
create a whole *world* of new problems.

> > I am trying as much as humanly possible to "paint from the root" here.

> What do you mean? As far as I can tell, the client can fake scrolling using
> the painting APIs you already have.

Which API?  The APIs--not counting the Scroll*() APIs--don't control what is
painted, only where it will be painted on the canvas.

> Let me be more precise. in nsRenderingContextWin::~nsRenderingContextWin(),
> +    if (usenative) {
> +      ((nsDeviceContextWin*)mContext)->MaybeReleaseDC(mDC);
> should be "!usenative".

This also, in the destructor, should be if (usenative).  It's a pair with
Init().  You only want to release the DC from the RenderingContext if you
*created* it in the RenderingContext.  And both should happen only when
SupportsNativeWidgets() is true--that means that nsIWidgets are going to be
real, valid entities with actual HWND's behind them.  If that happens, we grab
the HDC from it in Init() and we release it in the destructor.

My code for doing that is more convoluted than it needs to be though; I am going
to try and null-check HDC.  I suspect I did not do that because I was untrusting
of what I would get back for the HDC when I called SupportsNativeWidgets() and
it failed.  If there was a real reason I don't recall it; testing will reveal it.

> I guess I don't understand precisely what nsIRenderingContext and
> nsIDeviceContext are trying to abstract. I *thought* an nsIDeviceContext
> represented the characteristics of a particular device --- dpi, colors,
> available fonts, and so on, but was not specific to a particular drawing
> surface such as a particular window or offscreen memory area.

I think that's accurate; but since any particular window or offscreen memory
area can have different dpi, colors, available fonts, and so on, on Windows we
need the HDC to get that information from.  (We only use the HWND for two
purposes in DeviceContext: to get the HDC and to do multi-monitor support.)  For
windows we typically get the HDC from the HWND, but for a bitmap, the only way
to get an HDC is from the HBITMAP directly.  We either pass the HBITMAP in (a
possibility but not as flexible), or we actually initialize a DC with am HDC.
Comment on attachment 123758 [details] [diff] [review]
Patch

Taking this off my queue for now. Note that tetron is going to submit a new
gfx+widget implementation that allows headless rendering of a document into a
buffer, which takes care of many of the use cases of this code
Attachment #123758 - Flags: review?(roc)
roc, where does your "Visual Regression Tests" [1] work fit in with this bug? I
suppose landing that would resolve this, right?

[1] http://weblogs.mozillazine.org/roc/archives/2005/03/visual_regressi.html
It's different from this approach but there isn't really a need for what jkeiser
was doing here.
Status: ASSIGNED → RESOLVED
Closed: 19 years ago
Resolution: --- → WONTFIX
Product: Core → Core Graveyard
Component: Layout: Misc Code → Layout
Product: Core Graveyard → Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: