[FIX]possible rounding error in gfx text control

VERIFIED FIXED

Status

()

P3
normal
VERIFIED FIXED
18 years ago
18 years ago

People

(Reporter: buster, Assigned: rods)

Tracking

Trunk
x86
Windows NT
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [nsbeta3+]Fix in Hand; need engineer input-ckritzer;)

(Reporter)

Description

18 years ago
I just fixed a subtle bug in the HR frame where a rounding error was causing an 
off-by-one measuring error.  On my system, p2t=15.00000 according to the 
debugger, but the code:
   nscoord(pt2)
returns 14, not 15.

So I replaced
  nscoord(p2t)
with
  nscoord onePixel = NSIntPixelsToTwips(1, p2t);  // get the rounding right
which gives the intended answer of 15 (on my system.)

You may want to inspect nsGfxTextControlFrame2::CalculateSizeStandard(), which 
uses the cast rather than the macro.  This won't show up on Windows because of 
an #ifdef _WIN32
(Assignee)

Comment 1

18 years ago
Yes, I struggled with this, but I thought I got it right, I will take a look. 
Thanks
Status: NEW → ASSIGNED
(Assignee)

Comment 2

18 years ago
Yes, it isn't working right, needs to be fixed.


nominating. here is the diff:

Index: nsGfxTextControlFrame2.cpp
===================================================================
RCS file: /cvsroot/mozilla/layout/html/forms/src/nsGfxTextControlFrame2.cpp,v
retrieving revision 1.93
diff -r1.93 nsGfxTextControlFrame2.cpp
1354c1354,1355
<   charWidth = nscoord((float(charWidth) / p2t) + 0.5)*nscoord(p2t);
---
>   nscoord onePixel = NSIntPixelsToTwips(1, p2t);  // get the rounding right
>   charWidth = nscoord((float(charWidth) / float(onePixel)) + 0.5)*onePixel;
Keywords: nsbeta3
Whiteboard: Fix in Hand
(Assignee)

Updated

18 years ago
Summary: possible rounding error in gfx text control → [FIX]possible rounding error in gfx text control
(Reporter)

Comment 3

18 years ago
r=buster.  right thing to do, very low risk.
Marking nsbeta3+.
Whiteboard: Fix in Hand → [nsbeta3+]Fix in Hand
(Assignee)

Comment 5

18 years ago
fixed
Status: ASSIGNED → RESOLVED
Last Resolved: 18 years ago
Resolution: --- → FIXED

Comment 6

18 years ago
[buster|rods|mjudge], how would you suggest I verify?  Is this something that is 
easily viewed in a webdoc?  Thanks! -kritzer

Updated

18 years ago
Whiteboard: [nsbeta3+]Fix in Hand → [nsbeta3+]Fix in Hand; need engineer input-ckritzer;
(Reporter)

Comment 7

18 years ago
nope, this is a code-level fix, and it would take longer to fabricate a specific 
test case than it's worth.  The test case would depend on your platforms 
pixel/twip ratio, crap like that.  I'm verifying.
Status: RESOLVED → VERIFIED
(Assignee)

Comment 8

18 years ago
No, just take buster's word on it.
You need to log in before you can comment on or make changes to this bug.