52135 - [FIX]possible rounding error in gfx text control

Status: VERIFIED FIXED

(Core :: Layout: Form Controls, defect, P3)

Description

[buster]

I just fixed a subtle bug in the HR frame where a rounding error was causing an 
off-by-one measuring error.  On my system, p2t=15.00000 according to the 
debugger, but the code:
   nscoord(pt2)
returns 14, not 15.

So I replaced
  nscoord(p2t)
with
  nscoord onePixel = NSIntPixelsToTwips(1, p2t);  // get the rounding right
which gives the intended answer of 15 (on my system.)

You may want to inspect nsGfxTextControlFrame2::CalculateSizeStandard(), which 
uses the cast rather than the macro.  This won't show up on Windows because of 
an #ifdef _WIN32

Comment 1

[rods (gone)]

Yes, I struggled with this, but I thought I got it right, I will take a look. 
Thanks

Comment 2

[rods (gone)]

Yes, it isn't working right, needs to be fixed.


nominating. here is the diff:

Index: nsGfxTextControlFrame2.cpp
===================================================================
RCS file: /cvsroot/mozilla/layout/html/forms/src/nsGfxTextControlFrame2.cpp,v
retrieving revision 1.93
diff -r1.93 nsGfxTextControlFrame2.cpp
1354c1354,1355
<   charWidth = nscoord((float(charWidth) / p2t) + 0.5)*nscoord(p2t);
---
>   nscoord onePixel = NSIntPixelsToTwips(1, p2t);  // get the rounding right
>   charWidth = nscoord((float(charWidth) / float(onePixel)) + 0.5)*onePixel;

Comment 3

[buster]

r=buster.  right thing to do, very low risk.

Comment 4

[Kevin McCluskey (gone)]

Marking nsbeta3+.

Comment 5

[rods (gone)]

fixed

Comment 6

[ckritzer (gone)]

[buster|rods|mjudge], how would you suggest I verify?  Is this something that is 
easily viewed in a webdoc?  Thanks! -kritzer

Comment 7

[buster]

nope, this is a code-level fix, and it would take longer to fabricate a specific 
test case than it's worth.  The test case would depend on your platforms 
pixel/twip ratio, crap like that.  I'm verifying.

Comment 8

[rods (gone)]

No, just take buster's word on it.