Closed Bug 1504181 Opened 3 years ago Closed 3 years ago

Signature verification fails because of scientific notation

Categories

(Firefox :: Remote Settings Client, enhancement)

enhancement
Not set
normal

Tracking

()

RESOLVED FIXED
Firefox 65
Tracking Status
firefox65 --- fixed

People

(Reporter: leplatrem, Assigned: leplatrem)

References

Details

Attachments

(1 file)

If the data contains some float numbers (eg. 0.000002942), the canonical JSON differs between the server and the client, which leads to signature verification failure.

The server serializes it as `2.942e-06` and the client as `0.000002942`.
Mimic python for small float numbers
It may be interesting to see that we also ran into this issue in Normandy. We had thought to modify the server or the API to accommodate the discrepancy instead of modifying Firefox. We discussed it a bit here: https://github.com/mozilla/normandy/issues/1210
> Are there other users of CanonicalJSON besides remote-settings and normandy?
> 

Nope.

> I don't feel good about trying to force the formatting to be specifically the Python one. Maybe we should standardize Canonical JSON somewhere.
> 

Well yes, writing a standard is a good idea...

For example, I was super sad to see `1.3e+04` instead of `1.3e+4`...

> We had thought to modify the server or the API to accommodate the discrepancy instead of modifying Firefox

That was my intention too. But I didn't know what to set for the number of significant/fractional numbers... 

>>> format(0.000068272)
'6.8272e-05'

>>> format(decimal.Decimal(0.000068272), ".8g")
'0.000068272000'

>>> format(decimal.Decimal(0.000068272), "f")
'0.00006827199999999999957024654495540971765876747667789459228515625'

Whereas in Firefox the change generic (but ugly I admit)
Assignee: nobody → mathieu
The change here is simple enough that I'm ok with it. I wanted to provide additional context, in case it were useful.
From the linked bugs, and per communication on IRC, Normandy still has this incompatibility, and since both Kinto and Normandy are written in Python, I guess this fixes the problem. I feel like this is papering over a huge mess of underdefined behavior about what "Canonical JSON" actually means, but maybe that's OK. Having this fix in Firefox master might not cover us as well as changing both Normandy and Kinto, but it's definitely better than nothing.
I am considering a solution for both the client and server.

The scientific notation should not have any leading zero according to these specs:

- https://tools.ietf.org/html/draft-rundgren-predictable-serialization-for-json-00#section-3.3
- https://www.ecma-international.org/ecma-262/6.0/#sec-tostring-applied-to-the-number-type

I will update this patch, and we can deploy a fix on the server (PR https://github.com/Kinto/kinto-signer/pull/275)
We will be able to deploy a fix on the server only. The patch for this bug does not contain any code change, only tests. Hence no need to uplift required :)
Pushed by mleplatre@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/233f5ec75924
Force scientific notation in CanonicalJSON r=glasserc
https://hg.mozilla.org/mozilla-central/rev/233f5ec75924
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 65
See Also: → 1659419
You need to log in before you can comment on or make changes to this bug.