Closed Bug 1907936 Opened 2 years ago Closed 2 years ago

ClampDoubleToUint8 incorrectly rounds towards 0

Categories

(Core :: JavaScript Engine, defect, P1)

defect

Tracking

()

RESOLVED FIXED
130 Branch
Tracking Status
firefox130 --- fixed

People

(Reporter: anba, Assigned: anba)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

Previously reported at bug 745324. The patch attached to bug 745324 is incomplete and doesn't correctly handle the case when the fractional part is .5.

There are various ways how to implement half-to-even rounding, this C++
implementation was choosen because it's easy to translate into assembly
instructions.

  • Update the x86 implementation to use half-to-even rounding when available.
  • Use a single unified implementation for ARM32, because I didn't want to
    spend time trying to make sense of the HasVFPv3 code path which uses
    fixed floating point values.
  • ARM64 doesn't need any changes, because it was already using Fcvtns, which
    implements half-to-even rounding.
Blocks: sm-defects
Severity: -- → S3
Priority: -- → P1
Pushed by andre.bargull@gmail.com: https://hg.mozilla.org/integration/autoland/rev/e6d1732365c6 Correctly implement RoundHalfToEven in ClampDoubleToUint8. r=jandem
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → 130 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: