65701 - Need a way to dump the view system back buffer from script

Status: RESOLVED WONTFIX

(Core :: Web Painting, enhancement, P1)

Description

[Hixie (not reading bugmail)]

In the View system we have a back-buffer that we send all of our screen-bits 
to. What would be really cool for our automated testing of layout, CSS and DOM
would be to be able to trigger, from JS, a dump of this back-buffer that we 
could then compare to baseline copies.

So for example you could lay a table out, grab a back-buffer dump, move a row 
out of the table into a DIV using the DOM, take another back-buffer dump, move 
it back, take a back-buffer dump, all from JS. Then, using a baseline/compare
system, the automation script could check that the dumps have not changed since 
the last run on that platform.

I propose that to make this possible from JS we should add a new method to 
either the Window or Document APIs (possibly only enabled in debug builds),
which would take a string to be used as a filename. This function would then
take a copy of the contents of the back-buffer and save them into a file with
the given name in a specific directory (maybe controlled by an environment 
variable or hardcoded to be the profile directory or something).

The file need not be in a valid image format, since it will never be examined 
directly. It is only going to be used to compare and create baselines. However,
if the format is known (e.g. if it is simply a dump which has three bytes per 
pixel with the red, green and blue colour values each taking a byte, or some 
such), I could write some code to convert the dumps to real files (e.g. PPMs).

I'd rather this was done internally, instead of using an external driver app,
for several reasons. One is that this is going to need to be done in between 
changes to the DOM, so we would need to signal the external app somehow, which
is intentionally hard from JS. A second reason is that none of these apps are
going to be XP, certainly not to the point that we are. Another is that these
apps will probably cost money, which means that QA who are not working on this
project full-time would not be able to afford to use the product.

Comment 1

[buster]

this is a great idea.  keep in mind that the results will only lend themselves
to comparisons on the same machine with a constant environment, since the
resulting image is highly dependent on the bit depth of the display, the window
size, what fonts are loaded, etc.  same problem as the reflow regression tests,
nothing new.

Comment 2

[Hixie (not reading bugmail)]

Absolutely, fonts are also a major issue with that. However I think that should
not be such a big problem, since commercial QA teams like Netscape's can 
dedicate lab machines for running these type of tests (it's not like they need
to be fast machines, since they are unattended tests) and personal users of this
feature would always be using the same machine (theirs) anyway.

Comment 3

[Kevin McCluskey (gone)]

Just a note: The backbuffer managed by the viewmanager is only a scratch area
for rendering the current paint request. If you want to take a snapshot of the
contents of a window you need to force an invalidate of the entire window, wait
for the paint message to be processed then dump the backbuffer. 

Comment 4

[Hixie (not reading bugmail)]

Oh, ouch. That's a problem -- one of the important things I was hoping to be
able to catch are bugs where we do not invalidate enough (e.g. with the outline
property when that is turned on).

Comment 5

[Kevin McCluskey (gone)]

Setting milestone to mozilla1.0.1.

"Ian" given that we can only dump the backbuffer after invalidating the entire
window is this still worth doing?

Comment 6

[Kevin McCluskey (gone)]

Clearing milestone.

Comment 7

[Kevin McCluskey (gone)]

Setting milestone to mozilla0.9.1

Comment 8

[Hixie (not reading bugmail)]

Well we *definitely* need *something* for QA. If we can't do it with the back
buffer maybe we can have a sort of internal cache of what has been painted, in
debug mode?

Comment 9

[Kevin McCluskey (gone)]

The backbuffer maintained by the view system really will not be useful for this 
purpose. Even if I keep a separate backbuffer for this purpose it will alter the 
very thing that we are testing, which is the accuracy of the rendering. 

To provide what is being asked for we would have to be able to copy the contents 
of display buffer per top-level window to an offscreen image, get the bits for 
the offscreen image, then write out the bits. 

I suggest adding the following method:

nsIWidget::WriteDisplayBuffer(nsString *aFileName)

The implementation may be able to live entirely in the cross-platform 
nsBaseWidget class if it's possible to copy the onscreen window's buffer to an 
offscreen drawingsurface. If not we would have to create per-platform code to 
do the transfer. We would need to be able to call GetBits on on the 
drawingsurface and cycle through the pixel buffer to write it to a file.

The WriteDisplayBuffer would have to be exposed as IDL'ed method in the 
DOMWindow interface to be accessible through JavaScript.

Comment 10

[Hixie (not reading bugmail)]

Sounds great!

Where would I start looking if I wanted to help with this?

Comment 11

[Robert O'Callahan (:roc) (email my personal email if necessary)]

See my comments in bug 68499.

One thing we could do is to maintain a per-viewmanager "master bitmap" for the 
entire document window, and have the rendering code copy the rendered bits into 
that bitmap as well as the screen. Then we could dump that bitmap to disk at 
will. To make that work we'd have to also make sure scrolling requests updated 
the master bitmap as well as the screen. Other than scrolling and 
double-buffered rendering, I think the only place we should be drawing directly 
onto the screen is with the caret, and we should be able to take care of that 
too. That will give you a solution to this bug.

Once we have this master bitmap working, we could actually use it to get rid of 
platform scrolling altogether, by doing a bitblt from the master bitmap during 
repaint instead of calling the platform scrolling support. That paves the way 
for eliminating almost all uses of native widgets and fixing the problems 
discussed in bug 68499. In this case, performance might or might not suck... The 
master bitmap can be cached and discarded when necessary, because if we don't 
have it, we can always just do a full repaint. If the underlying platform lets 
us keep it in video RAM, it would probably be fast. Otherwise, it would probably 
suck. We might be able to use it selectively, only when the view structure is 
complicated enough for native scrolling to look ugly.

Comment 12

[Kevin McCluskey (gone)]

Moving to Mozilla0.9.6

Comment 13

[Kevin McCluskey (gone)]

Bulk moving Mozilla1.01 bugs to future-P1. I will pull from the future-P1 list
to schedule bugs for post Mozilla1.0 milestones