Find a way to introduce new buchets past 2^53

VERIFIED FIXED in 2015-09-01

Status

P1
enhancement
VERIFIED FIXED
3 years ago
3 years ago

People

(Reporter: mat, Assigned: mat)

Tracking

Avenir
2015-09-01
Points:
---
Dependency tree / graph

Details

(Whiteboard: [ktlo])

(Assignee)

Description

3 years ago
With the current system, the maximum number of features (aka "buchets") we can pass to the API from fireplace is 53. This is because fireplace encodes the features into an integer, and the maximum safe integer in JavaScript is 2^53, also available as Number.MAX_SAFE_INTEGER in modern browsers with ES6.

It might appear to work if the resulting number is small enough but fail as soon as too many features are detected as present.

See also: http://blog.vjeux.com/2010/javascript/javascript-max_int-number-limits.html

We need to do something, because with the feature detection we need to add for bug 1172486 we're hitting that limit, and we won't be able to add new feature requirements until this is solved.

Various ideas (keep in mind we need to support the old system no matter what, for older versions of our package - parsing the version number to determine how to decode the string will be important on the API side):
- Use 2 integers (never combine them arithmetically)
- Play with negative numbers (we can use from 0 to -2^53, Number.MIN_SAFE_INTEGER)
- Keep the feature profile as a string of 1s and 0s
- Keep the feature profile as a string, but compress it
(Assignee)

Updated

3 years ago
See Also: → bug 1172486
Severity: normal → enhancement
Priority: -- → P2
Whiteboard: [ktlo]
Priority: P2 → P1
- send feature profile as a base64 string, decode it into a typed array. Stick '=' on the end to distinguish it from oldstyle buchets.
(In reply to Allen Short [:ashort] from comment #1)
> - send feature profile as a base64 string, decode it into a typed array.
> Stick '=' on the end to distinguish it from oldstyle buchets.

seems like it'd that'd be quite long.

what about just

10101101001
1,0,1,0,1,1,0,1,00,1

?
i'm saying that could be encoded as "BWk=".
(Assignee)

Comment 4

3 years ago
Encoding the number in base 64 you mean ? But then, we'd end up having the issue are trying to avoid: manipulating a number that's greater than 2^53 in JavaScript.

10101101001... format is one of my proposals, yeah. It'd be long - since we currently have 53 flags to pass, and it's only going to get worse - but at least it's simple...
decode it into a typedarray and you can deal with it in chunks, no need to represent it as a single number.
(Assignee)

Comment 6

3 years ago
Ah, I think I see it now, but to be efficient you'd need to pack the bits in groups of 8 yourself, since there is no TypedArray, working on bits, right ? Otherwise you end up using a Uint8Array and wasting 7 bits of space for each byte, and when turning it into base64 ending up with something longer than the equivalent raw string of 0s and 1s (with btoa(String.fromCharCode.apply(null, new Int8Array(arr))) I transform an array of 53 features in a string of 72 chars)
(Assignee)

Comment 7

3 years ago
Answering myself: it turns out other people have solved this before :)

There are lots of cool BitField implementations using TypedArrays, grouping bits like I suggest in comment 6. Take this gist for instance: https://gist.github.com/panzi/56bbd884e00a5cc548dc

An array of alternating true/false values is reduced to "VVVVVVVVFQ==", not bad! My favorite proposal so far, thanks for suggesting it, :ashort.
(Assignee)

Updated

3 years ago
Assignee: nobody → mpillard
(Assignee)

Updated

3 years ago
Status: NEW → ASSIGNED
(Assignee)

Updated

3 years ago
Depends on: 1188357
(Assignee)

Updated

3 years ago
Blocks: 1195704
(Assignee)

Comment 9

3 years ago
Fixed in https://github.com/mozilla/fireplace/commit/69eb6392d7efe352eac2fec829eb3c76fea2a430

STR:
- Make sure to update your mkt-dev package to version 20150825.134039
- Follow the same steps as https://bugzilla.mozilla.org/show_bug.cgi?id=1188357#c2
- If possible test with multiple FxOS versions - the ones before 2.0 should have less features shown in the "View Feature Profile Information" page
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → 2015-09-01
(Assignee)

Comment 10

3 years ago
Note: like bug 1188357, this is not part of the weekly push, we'll make a new prod package when we need one.

Comment 11

3 years ago
Verified as fixed following steps from comment #9 on different FxOS versions, including versions before 2.0 (less features were shown as green)
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.