## Environment : Otoro phone, build 2012-10-03 us Taken from default.xml in b2g-distro: * "platform_build" revision= 795261940c8b11fb7dddd7a8e9dd8561fdc4fb64 * "gaia" revision= a3339652136fde9e5207e1bc1263a410c84f55fc * "releases-mozilla-central" revision= aecce9686d0b0e6ec25bb31e837eadace251a951 * "gonk-misc" revision= 202d2c62443c098b125e5d9d7b42226d98230e44 ## Repro : 1. launch calculator app 2. type in .001 /2 = ## Expected : 1. .0005 ## Actual : 1. .001 ## Note :
Oh. it's truncating at 3 decimal places and rounding up. :|
That doesn't sound good. Should we try to fix this before release?
This is a very basic change if we simply want to increase precision -- has a decision been made here yet?
This is quite a basic case for calculator. Nominate this as blocking.
blocking-basecamp: --- → ?
blocking+ calculator is not very useful without more precision than this
blocking-basecamp: ? → +
It looks like we might want to add the following test cases: Operations which involve: 1 - Positive very large scientific numbers, 1e+21 2 - Negative very large scientific numbers, -1e+21 3 - Positive large small scientific numbers, 1e-21 4 - Negative very small scientific numbers, -1e-21 I'll take a look at the code, but not sure if I'm the best person to solve this.
I have some ideas for this - taking unless someone really wants it?
Assignee: nobody → kgrandon
My thought on this is that instead of setting a fixed precision on all calculations, a maximum number character length should be set to something like 10. The final value of a calculation will give us a major hint as to what the user expects. For example, results could end up look like: 12.2937294 1.23948329 18239.1234 21932194.2 It's hard to mistake the "scale" that any of those results are measured on. No one gets to a huge number by accident, and no one gets to a decimal-heavy result without knowing that that's the type of scale that will be returned. I'm happy to take this if people agree.
Created attachment 690678 [details] Github pull request pointer Dale - Seen that you've done some work on Calculator - maybe you could take a look at this?
Attachment #690678 - Flags: review?(dale)
flagging for a testcase for this. John shih, can you add a testcase?
Driver retriage: We cannot hold the whole release back for this fix. Get review and land with approval once all P1 bugs are fixed.
blocking-basecamp: + → -
I think this constitutes data loss. Let's make sure to fix this for v1, regardless of blocking-basecamp (but as a lower priority as Dietrich notes).
tracking-b2g18: --- → +
Comment on attachment 690678 [details] Github pull request pointer Huge apologies for dropping the ball on this commit, I missed a whole bunch of emails and didnt notice it until checking my requests. As mentioned in the other thread, I think the implementation of the number parser is cleaner than a straight regex + conversion over all the exponents, however that doesnt fix the precision issue nor is it tested. I have proposed Andrea merges your tests + formatNumber fix into that patch and they can both go in together.
Attachment #690678 - Flags: review?(dale) → review-
This commit was combined with #819166 as they both overlapped in what they were fixing. Merged in: https://github.com/daleharvey/gaia/commit/77a7a987566a6834307b981ce78c15943c4f1615
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Component: Gaia::Calculator → Gaia::Calculator
Product: Boot2Gecko → Boot2Gecko Graveyard
You need to log in before you can comment on or make changes to this bug.