1067490 - Make navigator.mozSettings available to privileged and/or all apps

Status: RESOLVED WONTFIX

(Core :: DOM: Core & HTML, defect)

Description

[Wil Clouser [:clouserw]]

One of the requirements for upcoming partners (in the 2.1 time frame) is to be able to identify specific device manufacturers in order to determine apps to be offered and revenue sharing.  I spoke with the metrics group and they are using deviceinfo.product_model and deviceinfo.hardware to determine that currently.

I'm told deviceinfo comes from mozSettings (https://developer.mozilla.org/en-US/docs/Web/API/Navigator.mozSettings) which is a certified API.  This bug is a request to make this API available (at least on a read-only basis) to at least privileged apps, but ideally all apps.

I'm marking this as blocking-b2g:2.1? because we need this for revenue split and to determine what apps to show our partner's devices.

I'm also marking as feature-b2g:2.1 because there is no 2.1? flag.  I'll alert folks to this though as it's non-standard to skip the nomination.

Comment 1

[Caitlin Galimidi [:c8o]]

Candice - 
Let us know how you want us to track this request, we've nominated blocking and feature 2.1 - but realize we've missed many deadlines. Almstrom will work with any partners directly this week and next.

David A - 
Without using any partner names, can you check over my user story, above, for accuracy?

Comment 2

[[:fabrice] Fabrice Desré]

We can already grant read-only access to settings for privileged apps like we do for wallpaper: https://mxr.mozilla.org/mozilla-central/source/dom/apps/PermissionsTable.jsm#418

What we need is the exact list of settings you want (look at https://mxr.mozilla.org/mozilla-central/source/b2g/chrome/content/settings.js#166 for what's likely relevant to you) and argue with Paul about getting that in, with or without a prompt.

Comment 3

[Wil Clouser [:clouserw]]

Thanks Fabrice.  I got .product_model and .hardware from the metrics group.  They mentioned their detection had devolved into a blob of regex's already and have started a discussion on standardizing it in our Mozilla Certification Plan (see https://wiki.mozilla.org/FirefoxOS/Metrics#Standardization_of_Device_info and, I assume, an email thread somewhere).

So, I'll say at a minimum .product_model and .hardware, but is there a reason not to make all of them available? From that link:

>       'deviceinfo.os'
>       'deviceinfo.previous_os'
>       'deviceinfo.software'
>       'deviceinfo.platform_version'
>       'deviceinfo.platform_build_id'
>       'deviceinfo.hardware': 
>       'deviceinfo.firmware_revision'
>       'deviceinfo.product_model'

Paul - what do you think?

Comment 4

[Candice Serran (:cserran)]

Confirmed with Fabrice on remaining work needed for whitelisting the settings - feature flag changed to 2.1.

Comment 5

[Wil Clouser [:clouserw]]

Looking at the PermissionsTable.jsm file that Fabrice linked to we'll also need to figure out the names of these and then add them to the Marketplace manifest, make a new package, and get that in 2.1 (I filed bug 1067532 to get that on the radar).

Comment 6

[[:fabrice] Fabrice Desré]

Jonas, would you prefer tweaking the UA string instead of exposing these settings to privileged apps?

Comment 7

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

My understanding is that we for various other reasons need to put a device-model-identifier in the UA string. We've done that for most or all FirefoxOS releases so far as I understand it.

So we should just do that and use that to solve this too.

Comment 8

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

So sounds like this is WONTFIX. For 2.1 we should do the same thing that we've done in previous releases, which essentially means that a device identifier is added by the partner. I don't know exactly how this has been done, but likely by modifying one of the prefs that control the UA string. I'd recommend talking to Lawrence Mandel and/or Gerv who has handled this for previous releases.

Comment 9

[Paul Theriault [:pauljt] (no longer reading bugmail)]

I wrote a comment here yesterday but apparently didn't submit it sorry. Seems like Jonas has proposed an alternative anyways. But Ill summarize my comment below.
 
 - Exposing the deviceinfo.* settings readonly to privilged apps sounds ok to me. Its not much more sensitive that navigator.oscpu
 - Exposing write access is dangerous in case Gaia apps don't sanitize the settings values
 - Write access to settings might mean that apps use this as a channel to communicate as Gaia does. This would be bad
 - If at all, I would recommend this to be for privileged apps only, and without a prompt. ( how you could ask the use about this in a meaningful way?)


Longer term:
 - are we planning on adding a new permission per setting? Sounds clunky
 - maybe we should make a whitelist of "safe" settings which can be read from privileged/regular apps (fingerprinting will always be a risk though)
 - write access worries me for the reasons above. Might be more appropriate to provide mediated access to set settings (e.g. settings app web activities that take "suggested settings" arguments that the user has a chance to review before applying.

Comment 10

[Caitlin Galimidi [:c8o]]

wil and jonas will need to chat this one through.
this is not a simple wontfix without discussion.

Comment 11

[Wil Clouser [:clouserw]]

(In reply to Jonas Sicking (:sicking) from comment #8)
> So sounds like this is WONTFIX. For 2.1 we should do the same thing that
> we've done in previous releases, which essentially means that a device
> identifier is added by the partner. I don't know exactly how this has been
> done, but likely by modifying one of the prefs that control the UA string.
> I'd recommend talking to Lawrence Mandel and/or Gerv who has handled this
> for previous releases.

This is exactly what we're trying to avoid.  Non-standard strings added to the UA by partners (and/or other random people along the path) devolve into a maintenance headache.  There is a reason the first hit on google for "user agent detection" goes to *our documentation site* where we specifically recommend *against* using the user agent to serve up different content to different visitors (https://developer.mozilla.org/en-US/docs/Browser_detection_using_the_user_agent).  Closing this bug is essentially saying "do as we say, not as we do."

This also affects more than just the Marketplace.  User agent is used for every site the phone will visit which means encouraging tampering with that string could lead to difficulties across the web.  The metrics team will tell you they see typos, misspellings and syntax errors all day long in user agents, including FirefoxOS user agents -- already, and we just shipped this thing.

Not exposing a standard and consistent way of communicating device information is repeating the same mistakes we made with browsers 10 years ago and putting more of a burden on web developers everywhere.  This bug should be reopened.

Comment 12

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

I agree that modifying the UA sucks. A lot. But whether marketplace uses the UA or not, we are forced to modify the UA string to add a device model identifier. We've done this for every FirefoxOS release so far I'm fairly sure.

The android browser set a precedence of including the a device model identifier in the UA string. And since then a large part of the industry has come to depend on it, so FirefoxOS has had little choice but to do the same thing.

At this point we continue to push against websites using this identifier. But mainly since most do it as a way to detect device capabilities. I.e. the main badness isn't using the UA string, but rather relying on the looking at the device model in order to detect device capabilities.

I don't see why reading the device model from a setting will be a smaller maintainability headache than reading the device model from the UA string? Either way it seems like you'll have to keep a database mapping device model to partner?


I agree that if we weren't modifying the UA anyway that the settings solution seems like a better solution. But given that using the UA will be possible anyway, why not use that preexisting solution?

Comment 13

[Wil Clouser [:clouserw]]

Yeah, we'll still have a mapping of model to partner.

The engineering difference is minimal.  The setting will be simpler because we'd store the entire string instead of doing a regex on it and when we're certifying builds from OEMs we can say things like: 

> deviceinfo.product_model is 'Foo'

instead of vaguer things like:

> 'Foo' is somewhere in the User Agent string but not a part of any other strings

Even when we have the best of intentions UA detection becomes a crapshoot of regex to maintain.  Looking for /;Foo\s/ seems easy, and then someone adds a space between the semi-colon or calls it FooPrime and breaks things.  Even /Foo/ is a nice idea, and then someone else adds Foo as a workaround for something totally unrelated to us.  (Sound unrealistic?  See Seamonkey adding "Firefox" to it's UA a couple years ago...)

Settings can be more restricted (in definition) and show an effort to improve the current practice of using the UA.

Comment 14

[Caitlin Galimidi [:c8o]]

Is there a difference in ownership of activity here between mozilla owned tasks and tasks for the manufacturing partner?

- who sets the UA string, moz or manufacturer? 
- who sets the deviceinfo.product_model setting, moz vs manu?

Comment 15

[David Almström]

Echoing Caitlin, wouldn't it be better if we were in control how this was defined and used/shared? 

I.e. device.info could have a lot of information and we could create a few APIs that requests that the device will share that information to a whitelist of websites (i.e. marketplace.firefox.com or, as in another usecase, to marketplace.mozillaadexchange.com) which will seriously limit the ability for 3rd party to fingerprint

Comment 16

[[:fabrice] Fabrice Desré]

(In reply to Caitlin Galimidi [:c8o] from comment #14)
> Is there a difference in ownership of activity here between mozilla owned
> tasks and tasks for the manufacturing partner?
> 
> - who sets the UA string, moz or manufacturer? 

The default one is set by gecko "automatically", using gecko's version number. But it's easy to override by using a preference, and I think all the shipping devices I've seen had a custom one.
(I didn't check the tarako based devices yet).

> - who sets the deviceinfo.product_model setting, moz vs manu?

This is set as part of the gonk subsystem build. It's up to the OEM to devide what to use there.

(In reply to David Almström from comment #15)
> Echoing Caitlin, wouldn't it be better if we were in control how this was
> defined and used/shared? 
> 
> I.e. device.info could have a lot of information and we could create a few
> APIs that requests that the device will share that information to a
> whitelist of websites (i.e. marketplace.firefox.com or, as in another
> usecase, to marketplace.mozillaadexchange.com) which will seriously limit
> the ability for 3rd party to fingerprint

So long for a level playing field with such a whitelist :(
I'm not happy either with adding the model info to the UA string, but at least it would 'benefit' everyone.

Comment 17

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Since it sounds like some people have missed it, I first want to say that no matter what we do in this bug, the UA string will be changed *anyway* in order to expose a device model identifier.


So that said:

I think we're making a big deal out of a small issue here.

There's very small differences between the different solutions.

Either solution will be in control of the OEM.

Either solution will require looking up the device identifier in some form of database in order to figure out which partner it maps to.

Neither solution is particularly "good for the web". Modifying the UA string has many downsides, but it is industry standard. Relying on the settings API is non-standard and I can't guarantee that it will remain forwards compatible with future releases.

We can not control where this information is sent given that we will be modifying the UA string. The question we are debating here is if we should give an *additional* avenue for the marketplace to get this, already available, information.

Both solutions are relatively easy to implement. Either a small change is required on the device in order add a few settings to the permission table. Or a small amount of code will need to be written in the marketplace in order to parse the device model from the UA string.


Using the UA string has the advantage that it works on all the releases that we have done so far. That and the fact that we're past FL are the main reasons I'd recommend going with the UA solution (as it doesn't require client side changes other than ones that will already be done).

I'm happy to in future releases expose a device model identifier on the navigator object as a proper API (this should obviously be standardized). We can tie that to the UA string so that the two always match and we never expose additional bits of fingerprinting. That way we can also ensure that when OEMs add a device model identifier to the UA string, that they do so in a consistent way.


So all that said, if marketplace still feel like they prefer to use the settings API I can write up a patch to cover the device side pieces. Like I said, it's pretty simple. But I'd need help from someone else to fight for getting it uplifted to the Aurora branch.

Comment 18

[Wil Clouser [:clouserw]]

> I'm happy to in future releases expose a device model identifier on the
> navigator object as a proper API (this should obviously be standardized). We
> can tie that to the UA string so that the two always match and we never
> expose additional bits of fingerprinting. That way we can also ensure that
> when OEMs add a device model identifier to the UA string, that they do so in
> a consistent way.

This is my favorite part of what you said.

> So all that said, if marketplace still feel like they prefer to use the
> settings API I can write up a patch to cover the device side pieces. Like I
> said, it's pretty simple. But I'd need help from someone else to fight for
> getting it uplifted to the Aurora branch.

I'd rather use an API, (I'd rather use a standardized API!), but I also don't want to fight to put this in Aurora.  I can wontfix.  Who is the contact for Mozilla certification that we can work with to ensure the UA is easily parsable?  ni?caitlin who I think will know

Comment 19

[Caitlin Galimidi [:c8o]]

So - I'll open bug for the next steps

1 - determine UA string for TAKO ( Almstrom )
2 - enable marketplace to detect TAKO devices based on UA string ( Marketplace > General )
3 - Publish mdn how-to doc / dev doc needed (Chris Mills)
3 - Add UA String Criteria for TAKO to the Certification Checklist  ( Tracking > KWard )

Comment 20

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

(In reply to Caitlin Galimidi [:c8o] from comment #19)
> 1 - determine UA string for TAKO ( Almstrom )

I already filed https://bugzilla.mozilla.org/show_bug.cgi?id=1072049

> 3 - Publish mdn how-to doc / dev doc needed (Chris Mills)

I *think* we already have this somewhere. Lawrence Mandel should know.