775798 - Remove "\u000a" escaping from the sync2.0 spec

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Server: Sync, defect)

Description

[Ryan Kelly [:rfkelly]]

The spec currently defines application/newlines output as:

  application/newlines: each BSO is sent as a separate JSON object on its own line. Newlines in the body of the BSO object are replaced by ‘\u000a’.

I don't think this is necessary.  Any newlines in the BSO payload must already be escaped as "\n" for it to be valid JSON.  The only place literal newlines could occur is as insignificant whitespace in the structure of the JSON, in which case we're better off just deleting them than trying to add extra bytes to escape them.

I propose we remove all discussion of the '\u000a' escaping.  In the section on uploading application/newlines format data, we can add a note along the lines of "make sure your json library doesn't add newlines in an attempt to make the output human-readable".

Comment 1

[Toby Elliott [:telliott]]

I think you're correct. As the payloads are currently constructed, a newline is impossible. I don't think you even need to call out the lack of escaping - we'll reject the payload upon submission.

Comment 2

[Gregory Szorc [:gps]]

+1 to removal.

If we find some client absolutely needs newlines because a JSON encoder is inserting them, we can add the feature back in.

Comment 3

[Ryan Kelly [:rfkelly]]

Attachment: patch removing u000a escape of newlines

Docs updated in https://github.com/mozilla-services/docs/commit/6f592c80f21e6996ee87bda9d4cd269979e6fa48

Code patch attached.

Comment 4

[Ryan Kelly [:rfkelly]]

https://github.com/mozilla-services/server-syncstorage/commit/379c1a8992873ceab425f4c9191b1fb9f624af66

Comment 5

[Richard Newman [:rnewman]]

Surely you could translate newlines into straight \n in the JSON payload? After all, they must be occurring outside of a literal string, so we can do whatever we like…