Create an API for detecting manifest field/value support

RESOLVED FIXED in Firefox 32, Firefox OS v2.0

Status

Core Graveyard
DOM: Apps
RESOLVED FIXED
5 years ago
7 months ago

People

(Reporter: dbuc, Assigned: fabrice)

Tracking

unspecified
mozilla33
Dependency tree / graph

Firefox Tracking Flags

(blocking-b2g:-, firefox31 wontfix, firefox32 fixed, firefox33 fixed, b2g-v2.0 fixed, b2g-v2.1 fixed)

Details

(Whiteboard: [dependency: marketplace])

Attachments

(2 attachments)

(Reporter)

Description

5 years ago
As the app manifest evolves over time, it will likely add, remove, and modify the fields and values it supports. In addition, different versions of the implementing UA/system (Firefox v__, FxOS v__) may or may not recognize or implement support for various fields and value signatures.

To ensure apps and app-installing apps (Firefox Marketplace) are able to feature detect for these differences, we need an API that allows developers to inquire as to the support of field or value signatures. Below is a proposed API that allows for maximum flexibility, followed by a series of use cases that highlight the issues - some we face now, some that are very likely to surface in the future.

API:

  navigator.apps.testManifest(Manifest_JSON_Object);

Description :

  The `testManifest` method accepts a manifest JSON object or fragment, which is tested against the platform environment's ability to support the fields and values it passes. The return value from `testManifest` is a boolean, indicating that the object and all its fields and values are supported - passing an ignored or spurious field or value would fail this check.
I think when I brought this up in IRC, we were discuss this first on dev-webapi about what we should do here before filing a bug.

See https://groups.google.com/forum/#!topic/mozilla.dev.webapi/XP6z0idaerw that a stale discussion on this right now.

Comment 2

5 years ago
Some important use cases:

- There has been discussion of changing the `icons` field's expected type in order to support platform/resolution-specific icon sizes. This could be tested:

testManifest({icons: ["url", "url"]})
testManifest({icons: {"60": "url", "128": "url"}})

- The `origin` field has broken backwards compatibility with FXOS 1.0*. This could have easily been tested as follows:

testManifest({origin: "app://foo.bar"})

- Support for deprecated fields could be tested similarly:

testManifest({some_deprecated_field: "foo"})

- If a `version` field is added to the manifest (as has been discussed on dev-webapps), a simple test for whether the new manifest version is supported:

testManifest({version: "2"})
Matt - take this discussion to dev-webapi. When a conclusion is reached, we can move forward with the bug.
(Reporter)

Comment 4

5 years ago
(In reply to Jason Smith [:jsmith] from comment #3)
> Matt - take this discussion to dev-webapi. When a conclusion is reached, we
> can move forward with the bug.

I already talked with Jonas about the need, and we reached agreement that filing a bug is the right step at this point. This is not the same thing as the other API we discussed weeks ago. This API is a defacto need based on the manifest model - empirically speaking, there are no substitutes or equivalents to serve the API gap detailed in the bug description, as the following use-case highlights:

Sometime in the future a field is added to the manifest that allows developers to declare and utilize a feature or capability that has no DOM manifestation in the developer's app context. For the sake of example, let's say this is a `wallpaper` field that takes an array of image URLs --> { wallpaper: [ img1, img2, ... ] } A user's selection of new wallpaper for their homescreen does not occur within the developer's app - instead, it is purely represented to the user by chrome-level system UI.

For any feature activated or manipulated outside of an app's detectable code scope (which will be the case for various manifest fields), there is absolutely 0 ability to ascertain whether or not the current environment supports such a field and its associated functionality.
You and Jonas are not the only people involved in discussions like these - there's spec owners who need to reach agreement on the approach to be used. Take it to dev-webapi and come back when the spec owners (Mounir & Macros) and developer lead (fabrice) reach agreement on an approach. I don't want this bug turning into a 200 comment bug.
(Reporter)

Comment 6

5 years ago
(In reply to Jason Smith [:jsmith] from comment #5)
> You and Jonas are not the only people involved in discussions like these -
> there's spec owners who need to reach agreement on the approach to be used.
> Take it to dev-webapi and come back when the spec owners (Mounir & Macros)
> and developer lead (fabrice) reach agreement on an approach. I don't want
> this bug turning into a 200 comment bug.

Here's the post on dev-webapi I did, basically a regurgitation of this bug: https://groups.google.com/forum/#!topic/mozilla.dev.webapi/WTYxKIAJr0A

If we don't have the ability to test support for manifest fields and value signatures, we're screwed. If the folks on dev-webapi grok and support this *requirement*, I'll view it as a inconsequential bonus - if they don't, we'll need to escalate and to get it handled regardless.

I won't beat around the bush: We either provide a way to detect this and make the app purchase/install flow work, or users will throw their phones at the nearest wall when they realize they've purchased fundamentally incompatible software...

...then, once they realize we could have prevented it from the beginning, they'll pick up the phone they just threw at a wall, and throw it at us.
Checking in here, as the Marketplace team is looking for confirmation that this will land in the 1.3 development timeframe. Getting this bug fixed will allow us to prevent users from installing apps that are incompatible with the user's devices. (Want to avoid user-agent sniffing workarounds, such as the ones suggested in bug 895474.)

Let me know if there are any open questions and we're excited to see this become a reality - much appreciated.
Blocks: 895474
(Assignee)

Comment 8

5 years ago
I don't see any reason that could not be ready for 1.3. We need to decide on the specifics of the API though (yeah bikesheding!).
Assignee: nobody → fabrice
blocking-b2g: --- → 1.3?
Marcos, are you concerns/questions mentioned in dev-webapi solved?
Flags: needinfo?(mcaceres)
It's still not clear to me what problem this API solves on the client (though I see value on the server/market place). IIRC, the examples I got on the mailing list didn't show how this solved, um, "real world" problems; but just showed detecting "support" for things in the manifest - it was also unclear what "support" actually means, etc.

Additionally, the APIs didn't really explain what the developer was supposed to do about the lack of support for something. I'm sure there must be a clear need for this - but I'm obviously missing something.
Flags: needinfo?(mcaceres)
(Reporter)

Comment 11

5 years ago
(In reply to Marcos Caceres [:marcosc] from comment #10)
> It's still not clear to me what problem this API solves on the client
> (though I see value on the server/market place). IIRC, the examples I got on
> the mailing list didn't show how this solved, um, "real world" problems; but
> just showed detecting "support" for things in the manifest - it was also
> unclear what "support" actually means, etc.
> 
> Additionally, the APIs didn't really explain what the developer was supposed
> to do about the lack of support for something. I'm sure there must be a
> clear need for this - but I'm obviously missing something.

Marcos, you are failing to grasp 2 central concepts here:

1) There are, and will be, manifest keys that *have no detectable representation* in the app's scope. This means that the developer *cannot know* if the OS is or is not doing something with those directives.

2) You keep boomeranging back to the idea that an app should just detect ALL the things and degrade well (despite the fact there are cases they simply can't, see #1 above), well there's a problem with that:

Other "installing apps" have 0 ability to run the app's code prior to install, thus all the detection that you want to happen (again, some of which is not even possible), can't occur, because the app is not yet installed. It places any web page tasked with making a decision about users and installation of apps in an impossible position that requires uninvoked code omniscience. The result of this is users paying for apps that will not work on their device, that we could have had the ability to know.

I don't know an easy way to say this other than to agree with Marcos --> "but I'm obviously missing something." - never a truer word was spoken.

At some point, this API can no longer be up for debate - a loud minority's (of one?) inability to understand the issue in fundamental ways cannot block us from proceeding with a crucial need that fixes a crucial breakdown in feature detection.

We may, in fact, need to escalate this. You make the call as to whether that is necessary - either way, we're moving towards a solution.
(In reply to Daniel Buchner [:dbuc] from comment #11)
> Marcos, you are failing to grasp 2 central concepts here:
> 
> 1) There are, and will be, manifest keys that *have no detectable
> representation* in the app's scope. This means that the developer *cannot
> know* if the OS is or is not doing something with those directives.

I completely understand that (about no detectable representation) - but you haven't explained which keys in particular and what the developer could do about those or how it affects the application? Having a generalized solution may turn out to be unhelpful (or solvable in other ways), which is why I keep asking for *actual* examples where this is a *real* issue. 

> 2) You keep boomeranging back to the idea that an app should just detect ALL
> the things and degrade well (despite the fact there are cases they simply
> can't, see #1 above), 

Note that the responder in #1 had the same problem as me:

"I try to figure what kind of problem you try to solve but I'm not sure. Is it about feature detection or interoperability? Do you have concrete use cases to understand the issue?"

So at least one other person is confused by this and was asking for concrete cases. 

#1 mentions fullscreen and orientation. How does not having fullscreen stop and application from working? What if the user chooses to drop out of fullscreen? Can't the app simply request to go to fullscreen with the fullscreen API even if it doesn't start in fullscreen? Shouldn't we address orientation support checking in the orientation lock API instead? 

> well there's a problem with that:
> 
> Other "installing apps" have 0 ability to run the app's code prior to
> install, thus all the detection that you want to happen (again, some of
> which is not even possible), can't occur, because the app is not yet
> installed. It places any web page tasked with making a decision about users
> and installation of apps in an impossible position that requires uninvoked
> code omniscience. The result of this is users paying for apps that will not
> work on their device, that we could have had the ability to know.

based on exactly which fields? The examples in #2 are also not particularly helpful (as there is not much a developer can do about icons even after install - the icons might 404 also?... the URI issue is larger, but could be worked around - what problems are we seeing in the store? who is being affected by it?).

> I don't know an easy way to say this other than to agree with Marcos -->
> "but I'm obviously missing something." - never a truer word was spoken.
> 
> At some point, this API can no longer be up for debate - a loud minority's
> (of one?) inability to understand the issue in fundamental ways cannot block
> us from proceeding with a crucial need that fixes a crucial breakdown in
> feature detection.
>
> We may, in fact, need to escalate this. You make the call as to whether that
> is necessary - either way, we're moving towards a solution.

You guys are free to add whatever you want - like you said, I'm just one person trying to understand what you are doing and maybe help you not spend time on solving the wrong problem. I said everything I needed to say on the list about this subject). 

I'm really curious to see what you can actually solve with this API (as it looks quite complicated). I don't know what escalating it would mean? ask Jonas? Fabrice? but if they say it is ok, then that's good enough for me.
(Assignee)

Comment 13

5 years ago
Can we cool down please?

One example of undetectable property is the 'chrome' property: If the runtime doesn't support it, the user may end up installing an app that lacks navigation controls to the point of being unusable. No big deal for a free app, more annoying for a paid one. So it would be nicer if the Marketplace could either hide this app, or display a big warning "You may have issues navigating in this app".

Other properties undetectable are 'origin' and 'role'. The first one is useful to whitelist domains with CORS, the second is used by gaia to manage replaceable keyboards and homescreens. I would argue that we want 'origin' to be detectable, but not necessarily 'role', since its usefulness will depend of the apps but not of the platform.

We can bikeshed the api, but something simple like :
  apps.mgmt.supports(['chrome', 'origin', 'crazy_prop']) == ['chrome', 'origin'] would probably do.

I will trust Jonas & Marcos to come up with a nice api.
Thanks, Fabrice. Just wanted to say this all (comment 13) sounds perfect.

Comment 15

5 years ago
(In reply to Fabrice Desré [:fabrice] from comment #13)
> Can we cool down please?
> 
> One example of undetectable property is the 'chrome' property: If the
> runtime doesn't support it, the user may end up installing an app that lacks
> navigation controls to the point of being unusable. No big deal for a free
> app, more annoying for a paid one. So it would be nicer if the Marketplace
> could either hide this app, or display a big warning "You may have issues
> navigating in this app".
> 
> Other properties undetectable are 'origin' and 'role'. The first one is
> useful to whitelist domains with CORS, the second is used by gaia to manage
> replaceable keyboards and homescreens. I would argue that we want 'origin'
> to be detectable, but not necessarily 'role', since its usefulness will
> depend of the apps but not of the platform.
> 
> We can bikeshed the api, but something simple like :
>   apps.mgmt.supports(['chrome', 'origin', 'crazy_prop']) == ['chrome',
> 'origin'] would probably do.
> 
> I will trust Jonas & Marcos to come up with a nice api.

Well said.

One thing that would also be necessary is the ability to test that the values are supported, as well. For instance, consider if the "chrome" field is updated to support additional information:

{"chrome":{"navigation":true, "context_menus":true}}

A user agent that only supports "chrome.navigation" would need to be differentiated from a user agent that supports both.

In a perfect world, my ideal "christmas wishlist solution" would be an api like apps.mgmt.supports({...}) where {...} is an actual manifest object (the parsed JSON for a "minimum viable manifest") and the API then comes back with either a true or false (true being "yes, i support every single thing you passed to me" and false being "there's thing here that i don't recognize or are not supported"). One nice thing would be that with the ability to pass a "full manifest", we can detect support for manifest versions (mozilla, w3c, etc.) without having to do weird sniffing or tests.
(In reply to Fabrice Desré [:fabrice] from comment #13)
> I would argue that we want 'origin' to be detectable, but not necessarily 'role', since its
> usefulness will depend of the apps but not of the platform.

role will need to be detectable so the Marketplace app can tell the API not to show replacement keyboard apps to users on >1.3.  

If we're enumerating the missing features here, 'redirects' is undetectable too.  (It can be worked around but it requires using the browser permission which makes the app a whole lot more potentially dangerous so not to be encouraged)
(In reply to Andrew Williamson [:eviljeff] from comment #16)
> role will need to be detectable so the Marketplace app can tell the API not
> to show replacement keyboard apps to users on >1.3.  

I meant <1.3 of course.
Thanks for the clarification Fabrice - was quite helpful. 

This API should be exclusively available to the marketplace application (as a privileged API) - but not generally exposed to other packaged/hosted applications. With regards to the API design, I think it should be kept to an absolute minimum to avoid complexity - thus, it should only cover what Fabrice mentioned in comment #13 - and should be synchronous . The API should only take a single string at a time, so, for example:

```
apps.mgmt.supports('chrome'); // returns true|false
```

Then the marketplace can just implement its own looping around that (and extend it to add Promises or whatever it needs for its own logic).

```
function checkSupport(listOfFeatures){
  var support = {supported: [], unsupported: []};

  function processSupportList(item){
    (apps.mgmt.supports(item)) ? support.supported.push(item) : support.supported.push(item); 
  }

  listOfFeatures.forEach(processSupportList);
  return support; 
}

var whatCanIuse = checkSupport("chrome", "foo", "bar"); 

warnUserAbout(whatCanIuse.unsupported).then(function(){updateUIWithSupportedStuff(whatCanIuse.supported)});
```

If you guys need WebIDL for this, I'm happy to provide it.

The stuff that Matt suggests in #15 would be useful if we find that the above is not sufficient to address the problem. We can find this out fairly quickly if this is going to be prototyped.

Comment 19

5 years ago
(In reply to Marcos Caceres [:marcosc] from comment #18)
> Thanks for the clarification Fabrice - was quite helpful. 
> 
> This API should be exclusively available to the marketplace application (as
> a privileged API) - but not generally exposed to other packaged/hosted
> applications.

The Marketplace is not special. We are not a certified app (and have no plans to become one). I don't see any reason why this API should not be exposed to other applications.

This must also be accessible to the Marketplace when it is visited via the web (marketplace.firefox.com), so it should not be a privileged API.

> With regards to the API design, I think it should be kept to
> an absolute minimum to avoid complexity - thus, it should only cover what
> Fabrice mentioned in comment #13

As I mentioned in comment #15, it must be able to inspect fields that are not in the root of the manifest. If it cannot inspect non-root fields, the API does nobody any good.

> - and should be synchronous. The API
> should only take a single string at a time, so, for example:
> 
> ```
> apps.mgmt.supports('chrome'); // returns true|false
> ```
> 
> Then the marketplace can just implement its own looping around that (and
> extend it to add Promises or whatever it needs for its own logic).
> 
> ```
> function checkSupport(listOfFeatures){
>   var support = {supported: [], unsupported: []};
> 
>   function processSupportList(item){
>     (apps.mgmt.supports(item)) ? support.supported.push(item) :
> support.supported.push(item); 
>   }
> 
>   listOfFeatures.forEach(processSupportList);
>   return support; 
> }
> 
> var whatCanIuse = checkSupport("chrome", "foo", "bar"); 
> 
> warnUserAbout(whatCanIuse.unsupported).
> then(function(){updateUIWithSupportedStuff(whatCanIuse.supported)});
> ```
> 
> If you guys need WebIDL for this, I'm happy to provide it.
> 
> The stuff that Matt suggests in #15 would be useful if we find that the
> above is not sufficient to address the problem. We can find this out fairly
> quickly if this is going to be prototyped.
(Reporter)

Comment 20

5 years ago
(In reply to Matt Basta [:basta] from comment #19)
> (In reply to Marcos Caceres [:marcosc] from comment #18)
> > Thanks for the clarification Fabrice - was quite helpful. 
> > 
> > This API should be exclusively available to the marketplace application (as
> > a privileged API) - but not generally exposed to other packaged/hosted
> > applications.
> 
> The Marketplace is not special. We are not a certified app (and have no
> plans to become one). I don't see any reason why this API should not be
> exposed to other applications.

Agreed - this cannot, and should not, be a privileged API. It reveals nothing but boolean into on feature support and does not allow for execution of any APIs or features. This API would be invaluable for any other app that provides an app installation experiences, for instance:

- Nvidia could choose to create a Tegra Zone app that offered search and install of games
- Amazon could create an alternate app marketplace to install apps
- Apps with companion websites could choose whether to prompt for install or not, based on detection of features (maybe an app requires the keyboard functionality in later versions of the OS and on their website they don't prompt for install of the app unless the user's env supports that option).

> This must also be accessible to the Marketplace when it is visited via the
> web (marketplace.firefox.com), so it should not be a privileged API.
> 
> > With regards to the API design, I think it should be kept to
> > an absolute minimum to avoid complexity - thus, it should only cover what
> > Fabrice mentioned in comment #13
> 
> As I mentioned in comment #15, it must be able to inspect fields that are
> not in the root of the manifest. If it cannot inspect non-root fields, the
> API does nobody any good.

The API should accept any depth of manifest key/value to ensure we can detect options correctly if they are added, removed, modified, or their values are extended. Here's a pseudo code example of what we could see in the future:

Let's say we add a 'wallpaper' key that allows an app to add an array of URLs to images it offers as wallpapers related to their app "wallpaper": [ "url-1, "url-2", ... ], these wallpapers would then be made available in the Firefox OS Wallpaper selection screen. For a relevant example, consider a sports team's app that also comes with some wallpapers of the players, team logo, etc (I have one on my phone that does this). The 'wallpaper' key is a wild success, apps start including wallpapers with their apps. As time goes on, we move to accept both simple URL strings and objects, with size and resolution specifics --> "wallpaper": [ {"low_dpi": "url-1", "high_dpi": "url-1"}, "url-2", ... ]. This API should be able to take the following and tell me if the values are recognized:

// If the user's env supports the 'wallpaper' key, 
// and the passing of both URL strings and complex 
// wallpaper objects, the API should return true.

navigator.manifestSupports({ "wallpaper": [ {"low_dpi": "url-1", "high_dpi": "url-1"}, "url-2", ... ] });

This ability will provide for any and all future use-cases we face, and provide developers with greater control over when they can show users apps for installation. I believe everyone on the planet generally hates when a website slaps you with a "Download our app!" screen, even more when you're not using the platform their app is made for, or when you do go to install it, you find out it isn't actually compatible with your device or environment --> this API will help prevent these facepalm user flows.


> > - and should be synchronous. The API

I'm not sure what the right call is on this one - I guess sync is fine?

Comment 21

5 years ago
A more urgent use case for "deep field" support is the discussions that have been going on with the "icons" field. Support for multiple resolutions and pixel densities and vector icons, if they broke backwards-compatibility, would be undetectable with a simple test for root fields.
Hi All,

Let me try to focus the discussion here a bit.

First off, I definitely agree that this can't be a privileged API.

We've always designed the Apps API such that any website can act as an "app store". Both to allow developers to self-host their apps, but also to allow competing marketplaces.

Hence this API definitely needs to be exposed everywhere.

I really do like the idea of an API like:

if(navigator.manifestSupports({ ... manifest here ... }) {
  <display app to user>
}

The nice thing with this approach is that it in theory fixes two of the fundamental issues that hasFeature had:

1. hasFeature had the problem that it was very coarse grained. That meant that if you had implemented 90% of a spec, you still couldn't detect support for any of it (even if the last 10% were not very useful).
2. hasFeature has the problem that it required a separate code-path just to support hasFeature which makes it hard to remember to keep that code path up-to-date.

With this API we can test individual properties and even individual values. And we can implement the install code such that we drop any properties/values that we don't have support for. That way forgetting to update manifestSupports, the feature won't work at all, which will be quickly detected.

That said, manifestSupports has two problems of its own:

1. The design of the API means that it's very manifest specific. I.e. we can't use the API to also do feature detecting of privileged APIs.
2. The fact that the API is so finegrained means that it's going to be very tedious to maintain. I.e. we'll have to write code to validate each property and constantly update that. Almost all the time this will be separate code from the code to actually implement the property behavior.

So an easier place to start might be to do something like this:

navigator.hasFeature("manifest.chrome.navigation");

that is going to be way easier to implement, and also allows us to expand to

navigator.hasFeature("api.SMSManager.sendMMS");

Where needed, we can add support for additional arguments, so something like

navigator.hasFeature("manifest.role", "keyboard");

but we'd only do this on a case-by-case basis as needed.

For strings like "api.SMSManager.sendMMS" we could populate the database backing this automatically based on what functions and attributes we have in our WebIDL files.

For strings like "manifest.foo.bar" we can make sure the install code drops any manifest properties that hasFeature returns false for. That will ensure that we remember to keep the database up-to-date.
Putting this functionality on navigator.hasFeature() WFM, and addresses most of my concerns.

Comment 24

5 years ago
From what I can tell, that suits our needs quite well.

My only concern is checks that involve changes rather than additions/deletions. For example, the manifest's `messages` field might someday become an object instead of an array of objects. In that case, how do you think the best way of detecting this might look?
(In reply to Matt Basta [:basta] from comment #24)
> From what I can tell, that suits our needs quite well.
> 
> My only concern is checks that involve changes rather than
> additions/deletions. For example, the manifest's `messages` field might
> someday become an object instead of an array of objects. In that case, how
> do you think the best way of detecting this might look?

In such situations, the easiest course would be to void changing the types of members that we support and instead add new manifest members. If a member justifies a change to its data type, then it likely warrants a new name too.
Not a committed feature for 1.3, so not blocking.
blocking-b2g: 1.3? → -
(In reply to Jason Smith [:jsmith] from comment #26)
> Not a committed feature for 1.3, so not blocking.

Fabrice - I thought this was definitely landing in 1.3?  This affects what apps the Marketplace can offer to what devices.
Flags: needinfo?(fabrice)

Comment 28

5 years ago
(In reply to Jonas Sicking (:sicking) Vacation Nov23-Dec2. Please ni? if you want feedback from comment #22)
> First off, I definitely agree that this can't be a privileged API.
I'd like to raise a potential concern with privacy (tell me if this is the wrong place to have this discussion, I'm happy to move it somewhere else). This API may lead to fingerprinting. I see the threat in one of 2 ways:
1) Apps based on the FirefoxOS Manifest can be installed on a variety of platforms which have very different support sets. Admittedly, the amount of information this part provides can be recovered in lots of other ways (I guess navigator.userAgent or equivalent)
2) Users might be given the option to disable system-wise support for something, like hide their phone has a camera or support a given feature. In this context, the API can fingerprint given what's available.

I don't know how serious or likely this threat is, but felt like raising it since it hadn't been so far.

> We've always designed the Apps API such that any website can act as an "app
> store". Both to allow developers to self-host their apps, but also to allow
> competing marketplaces.
> 
> Hence this API definitely needs to be exposed everywhere.
It's an interesting reasoning and I agree that allowing competing marketplaces is one of the most important feature of the FirefoxOS/Web apps ecosystem.
However, I wonder if it *requires* the API from being non-privileged. For instance, it could be agreed that a marketplace app can only exist if available in one other existing marketplace or installed by default in a phone. That might enable a very high-standard of quality among marketplaces and their marketplace apps. Granted, there are few incentives for a marketplace to encourage another marketplace...
In any case, I feel that from "allowing competing marketplaces" to "the API needs to be non-privileged", the move is not obvious.


> The nice thing with this approach is that it in theory fixes two of the
> fundamental issues that hasFeature had:
> 
> 1. hasFeature had the problem that it was very coarse grained. That meant
> that if you had implemented 90% of a spec, you still couldn't detect support
> for any of it (even if the last 10% were not very useful).
> 2. hasFeature has the problem that it required a separate code-path just to
> support hasFeature which makes it hard to remember to keep that code path
> up-to-date.
Lack of standard and accurate semantics was a big fundamental issue too and this proposal doesn't solve it.
This issue leads to some competitors returning true in some cases, but not having proper support to run some apps (what happened with hasFeature too).
The problem is that there is a disconnect between what the platform actually allows and the returned boolean. FirefoxOS is making a serious effort to wire up the returned boolean and what's implemented in the platform, but will other implementations do the same?

I don't know how to solve this problem really. Attach a minimum test suite to the spec? It sounds like a lot of effort, but at least, it provides a minimum of sanity to what the boolean means. The test suite can co-evolve with reality of implementations.
(Assignee)

Comment 29

5 years ago
(In reply to Wil Clouser [:clouserw] from comment #27)
> (In reply to Jason Smith [:jsmith] from comment #26)
> > Not a committed feature for 1.3, so not blocking.
> 
> Fabrice - I thought this was definitely landing in 1.3?  This affects what
> apps the Marketplace can offer to what devices.

Not blocking != not landing. But for sure we're getting late there. If we go the navigator.hasFeature() route as exposed in comment 22, which seems the current consensus there's a reasonable chance we can land something.
Flags: needinfo?(fabrice)
(Reporter)

Comment 30

5 years ago
(In reply to David Bruant from comment #28)
> (In reply to Jonas Sicking (:sicking) Vacation Nov23-Dec2. Please ni? if you
> > We've always designed the Apps API such that any website can act as an "app
> > store". Both to allow developers to self-host their apps, but also to allow
> > competing marketplaces.
> > 
> > Hence this API definitely needs to be exposed everywhere.
> It's an interesting reasoning and I agree that allowing competing
> marketplaces is one of the most important feature of the FirefoxOS/Web apps
> ecosystem.
> However, I wonder if it *requires* the API from being non-privileged. For
> instance, it could be agreed that a marketplace app can only exist if
> available in one other existing marketplace or installed by default in a
> phone. That might enable a very high-standard of quality among marketplaces
> and their marketplace apps. Granted, there are few incentives for a
> marketplace to encourage another marketplace...
> In any case, I feel that from "allowing competing marketplaces" to "the API
> needs to be non-privileged", the move is not obvious.

What if I'm a regular ol' website that offers an app which requires a certain manifest feature, and I want to check if it iss supported before showing my app's install UI to the user? If this was a privileged API, I'd need to flash user with a prompt for access this API, even if it turns out their current env doesn't support the feature/app. This means I would be forced to nag all users, even ones who would/could never install my app. It's almost the API-permission-nag-UI equivalent of this: http://idontwantyourfuckingapp.tumblr.com/

> This issue leads to some competitors returning true in some cases, but not
> having proper support to run some apps (what happened with hasFeature too).
> The problem is that there is a disconnect between what the platform actually
> allows and the returned boolean. FirefoxOS is making a serious effort to
> wire up the returned boolean and what's implemented in the platform, but
> will other implementations do the same?

If the API dictates the test is to be integrated into the actual manifest parse-flow, the "someone could lie" case should be a fringe issue. Regardless, the idea that "someone could lie" is not a great reason in itself, consider the following:

- If a site can't perform this detection, app installation could fail, or the app may not run correctly.

- If a site can perform this detection, and the platform lies (against the spec's mandated test process), app installation could fail, or the app may not run correctly.

- If a site can perform this detection, and the platform is honest, app installation is more likely to only happen for users who can install the app, and the app is more likely be installed on devices/envs where it can operate correctly.

Considering the logical outcomes presented above (assuming the premise "the platform could lie" actually happens), there's still only one option that lessens the chance of incompatible installation attempts, and broken app experiences: implement this API. The "the platform could lie" argument throws the baby (the ability to mitigate and dramatically reduce bad install/user experiences of apps) out with the bathwater (the mere *possibility* a platform could lie).
(In reply to Daniel Buchner [:dbuc] from comment #30)
> What if I'm a regular ol' website that offers an app which requires a
> certain manifest feature, and I want to check if it iss supported before
> showing my app's install UI to the user? If this was a privileged API, I'd
> need to flash user with a prompt for access this API, even if it turns out
> their current env doesn't support the feature/app. This means I would be
> forced to nag all users, even ones who would/could never install my app.
> It's almost the API-permission-nag-UI equivalent of this:
> http://idontwantyourfuckingapp.tumblr.com/

Privileged API != permission prompt.  There are both unprivileged and privileged APIs that do and don't flash permission prompts.  If it was a privileged API then it just wouldn't be available to regular ol' website, and would only be available to privileged (packaged) apps - something that our own Marketplace doesn't even satisfy!
(Reporter)

Comment 32

5 years ago
(In reply to Andrew Williamson [:eviljeff] from comment #31)
> (In reply to Daniel Buchner [:dbuc] from comment #30)
> > What if I'm a regular ol' website that offers an app which requires a
> > certain manifest feature, and I want to check if it iss supported before
> > showing my app's install UI to the user? If this was a privileged API, I'd
> > need to flash user with a prompt for access this API, even if it turns out
> > their current env doesn't support the feature/app. This means I would be
> > forced to nag all users, even ones who would/could never install my app.
> > It's almost the API-permission-nag-UI equivalent of this:
> > http://idontwantyourfuckingapp.tumblr.com/
> 
> Privileged API != permission prompt.  There are both unprivileged and
> privileged APIs that do and don't flash permission prompts.  If it was a
> privileged API then it just wouldn't be available to regular ol' website,
> and would only be available to privileged (packaged) apps - something that
> our own Marketplace doesn't even satisfy!

I'm not sure what your point is, are you it's OK that this be implemented as a privileged API? If so, how does that address the fact that a site with an app, or apps, needs to be able to detect and offer only the apps that will work for the user's environment? Surely you're not advocating a flow where the user would hit a site, be prompted to install an app installer that asks for this permission alone, then run that installer to find out what actual apps they can/can't install? I feel like I may have completely misinterpreted your response (if so, sorry for being dense), please clarify if that's the case.

Comment 33

5 years ago
The Marketplace can't use this API if it's privileged. If it's a privileged API (I can't see any reasoning for making it one--though I could be missing something), it doesn't satisfy any of our use cases and will have been a complete waste of time.
(In reply to Daniel Buchner [:dbuc] from comment #32)
> (In reply to Andrew Williamson [:eviljeff] from comment #31)
> > Privileged API != permission prompt.  There are both unprivileged and
> > privileged APIs that do and don't flash permission prompts.  If it was a
> > privileged API then it just wouldn't be available to regular ol' website,
> > and would only be available to privileged (packaged) apps - something that
> > our own Marketplace doesn't even satisfy!
> 
> I'm not sure what your point is, are you it's OK that this be implemented as
> a privileged API? 

my point was you appeared to have conflated APIs that have prompts with the type of app (unprivileged/privileged) the API requires.  If you see https://developer.mozilla.org/en-US/Apps/Developing/App_permissions both types of app have PROMPT and ALLOW (i.e no prompt).  I'm not particularly okay with it being a privileged API as it limits it to privileged apps, and Marketplace isn't a privileged app.

> If so, how does that address the fact that a site with an
> app, or apps, needs to be able to detect and offer only the apps that will
> work for the user's environment? 

well, it wouldn't.  

> Surely you're not advocating a flow where
> the user would hit a site, be prompted to install an app installer that asks
> for this permission alone, then run that installer to find out what actual
> apps they can/can't install? 

No, that would be crazy!

If we think fingerprinting is a big enough privacy issue (and tbh I don't) then it could be non privileged and PROMPT which would still allow Marketplace (and random websites) to feature detect but not without alerting the user that its going on and giving them opt-out.

Comment 35

4 years ago
Replying to some comments from the previous issue that got moved to this (issue 873599). 

(In reply to Daniel Buchner [:dbuc] from comment #1)

What I want for a CPU core count API isn't something that only works with installed apps--core count is hardly sensitive information, especially when I can timing attack it out of you anyways (I wrote that timing attack stuff you linked to). I don't deeply participate in any web standards organizations, but I really think you or someone else here should bring up standardizing navigators.cores.

(In reply to Jonas Sicking (:sicking) from comment #2)
> * Number cores in the hardware is different from number of cores available
> to a given page/app. The number of cores available to a given page/app
> depends what other things are running in the background.

I don't think there should be an interface designed around available cores. Desktop apps that attempt to fully utilize all cores (e.g. 7zip LZMA2/xz (compression), Handbrake (video encoding/compression), etc.) make enough threads for every core, even if some are at high load. The OS handles scheduling so you don't have to worry what cores are at what load (at least for highly parallelizable algorithms).

An "available cores" API would require event subscription, greatly increased worker management complexity and overhead, and could possibly interfere with itself unless you manage your threadpool just right.


As to whatever happens with this bug, I hope CPU core count access is NOT included in a privileged API, as I've proven that it is already public information (although I currently have to beat it out of your browser with wasted CPU cycles).

Comment 36

4 years ago
(In reply to comment #28)
> (In reply to Jonas Sicking (:sicking) Vacation Nov23-Dec2. Please ni? if you
> want feedback from comment #22)
> > First off, I definitely agree that this can't be a privileged API.
> I'd like to raise a potential concern with privacy (tell me if this is the
> wrong place to have this discussion, I'm happy to move it somewhere else). This
> API may lead to fingerprinting. I see the threat in one of 2 ways:
> 1) Apps based on the FirefoxOS Manifest can be installed on a variety of
> platforms which have very different support sets. Admittedly, the amount of
> information this part provides can be recovered in lots of other ways (I guess
> navigator.userAgent or equivalent)

Yeah, as far as I can see, the unique information that this API provides is already available from the user agent string, so we're not adding any new entropy bits.  Based on that, I'm not sure if there is a fingerprinting concern here.

CCing Sid to check my sanity here.
Flags: needinfo?(sstamm)
I read through most of this bug (sorry, ran out of energy) and did not draw the same conclusion that data(this feature) == data(user agent string).  Seemed more like this feature exposes more data.  Which is probably fine.

Things unclear to me that could help me wrap my head around it:
* Who/what can access this API (and obtain data)?  I'm guessing "any installed app", but am uncertain.
* What data is accessible via the API?  Seems like a capabilities list of sorts, but I'm not sure I fully get it.
(In reply to Sid Stamm [:geekboy or :sstamm] from comment #37)
> Things unclear to me that could help me wrap my head around it:
> * Who/what can access this API (and obtain data)?  I'm guessing "any
> installed app", but am uncertain.

Any website, but could be PROMPT (so a doorhanger like GPS).  There doesn't seem to be a consensus if installed and privileged apps are also PROMPT or ALLOW (i.e. no prompt).

> * What data is accessible via the API?  Seems like a capabilities list of
> sorts, but I'm not sure I fully get it.

It returns what parts of the manifest are supported by the device/OS, to account for changes in the specification over time.
Thanks :eviljeff.  Sounds pretty sane to me -- agreeing with Ehsan -- so long as there's a pref to change the policy so any "ALLOW" to "PROMPT" at runtime so addons that want to reduce fingerprintability can require prompts for all or can disable the API entirely.
Flags: needinfo?(sstamm)

Comment 40

4 years ago
Great, thanks!  So, given the above, can we move forward here?  It's not clear to me what the latest state of the proposals in this bug are...
We should not prompt unless we are protecting user-private data. As far as I can tell there is no user-private data here. Prompting when not needed is harmful as it causes prompting exhaustion and gets users used to being prompted about harmless things that they don't care about, thus they are less likely to take the time to provide the right answer when it actually matters.

If this API only exposes data that could be deduced by looking at the UA-string (which I think and hope is the case), then I don't see any harm in exposing it to all websites without any prompting. Our default should be that webpages get the same APIs as apps, unless security or privacy requires otherwise.

Comment 42

4 years ago
(In reply to Jonas Sicking (:sicking) from comment #41)
> We should not prompt unless we are protecting user-private data. As far as I
> can tell there is no user-private data here. Prompting when not needed is
> harmful as it causes prompting exhaustion and gets users used to being
> prompted about harmless things that they don't care about, thus they are
> less likely to take the time to provide the right answer when it actually
> matters.

Agreed.

> If this API only exposes data that could be deduced by looking at the
> UA-string (which I think and hope is the case), then I don't see any harm in
> exposing it to all websites without any prompting. Our default should be
> that webpages get the same APIs as apps, unless security or privacy requires
> otherwise.

Given the fact that everybody seems to agree that there is nothing exposed by this API which could not be inferred from the UA string already, it seems like we can proceed without prompting.

Daniel, does this satisfy the market place needs?  Is there anything else that you guys might need for your use case which won't be addressed by this proposal?
Flags: needinfo?(dbuchner)
(Reporter)

Comment 43

4 years ago
I need one of the Marketplaces devs to check this out to ensure the solution fits. It was Basta, but I'm guessing Cvan has the most experience at the moment - hopefully he'll chime in here, regardless I'll touch base with him tomorrow.
Flags: needinfo?(dbuchner)

Comment 44

4 years ago
I think the API suggested in comment #22 is fine, so long as the concern from comment #24 is addressed.

Comment 45

4 years ago
(In reply to Matt Basta [:basta] from comment #44)
> I think the API suggested in comment #22 is fine, so long as the concern
> from comment #24 is addressed.

Does comment 25 address that sufficiently?

Comment 46

4 years ago
(In reply to :Ehsan Akhgari (needinfo? me!) (slow responsiveness, emailacopolypse) from comment #45)
> Does comment 25 address that sufficiently?

As a principle, sure. It'd be a whole lot better if someone made a commitment to not change the type of manifest members, though.

Comment 47

4 years ago
(In reply to Matt Basta [:basta] from comment #46)
> (In reply to :Ehsan Akhgari (needinfo? me!) (slow responsiveness,
> emailacopolypse) from comment #45)
> > Does comment 25 address that sufficiently?
> 
> As a principle, sure. It'd be a whole lot better if someone made a
> commitment to not change the type of manifest members, though.

I'd think that's something that we can ensure.  Fabrice, what do you think?
Flags: needinfo?(fabrice)
(Reporter)

Comment 48

4 years ago
(In reply to :Ehsan Akhgari (needinfo? me!) (slow responsiveness, emailacopolypse) from comment #47)
> (In reply to Matt Basta [:basta] from comment #46)
> > (In reply to :Ehsan Akhgari (needinfo? me!) (slow responsiveness,
> > emailacopolypse) from comment #45)
> > > Does comment 25 address that sufficiently?
> > 
> > As a principle, sure. It'd be a whole lot better if someone made a
> > commitment to not change the type of manifest members, though.
> 
> I'd think that's something that we can ensure.  Fabrice, what do you think?

It would be optimal if the API could detect and report support for value signatures, stuff like acceptable value types and addition/removal/modification of compatible values. If this API cannot detect those things, we would need to ensure the signatures don't change, because we would have no way to detect compatibility.
(Reporter)

Comment 49

4 years ago
Sorry, my last comment was the response to the info request - ignore this.
Flags: needinfo?(fabrice)
We're not planning on changing semantics of existing properties of the manifest. Possibly we'll add more support for richer values, like richer filter types for webactivities. But simply expressing what types are supported won't do much there.

So lets keep things simple for now and worry about making the API richer once it's actually needed so we types of things we need to be able to express.

Comment 51

4 years ago
(In reply to Jonas Sicking (:sicking) from comment #50)
> We're not planning on changing semantics of existing properties of the
> manifest. Possibly we'll add more support for richer values, like richer
> filter types for webactivities. But simply expressing what types are
> supported won't do much there.
> 
> So lets keep things simple for now and worry about making the API richer
> once it's actually needed so we types of things we need to be able to
> express.

That sounds good to me!
There's been a bunch of discussions in various places about when navigator.hasFeature() should return what. And also about whether something like navigator.mozContacts should return undefiend, null or a working API. Or when the navigator.mozContacts property simply shouldn't exist. Likewise about when the various interface objects of APIs should be exposed on the global or not.

For some reason posts to dev-webapi isn't showing up in Google Groups, so I can't link to a proposal there. Posting my full proposal here instead.



Regarding when to expose an API like navigator.mozContacts:

We should "expose the API" when the "platform supports the API".

Where "platform supports the API" means something like "Both device hardware and UA software supports the API, and if API requires a permission to be enumerated and signed for in an app manifest, that that has been done."

And where "expose the API" means to expose both entry points in the form of navigator.foo, as well as interface objects on the global.

In other words, if the device is lacking the necessary hardware to make an API work, we should hide both any entry points on the navigator, as well as any interface objects. Likewise, if the API is one that requires a permission to be enumerated in a manifest and then signed for, but the page isn't loaded from a package where that has happened, we should hide the API. And obviously if we haven't implemented the API it won't exist.

However, the user having to approve usage of the API though a security dialog, but having explicitly answered "no" in such a dialog, does *not* mean that we should hide the API. Then the API should exist, but should return asynchronous errors when used.

The logic here is that if a page couldn't possibly use an API, no matter what the user does (short of buying new hardware or installing new software) then we should hide the API. This signals to the page "you'll have to do without this API".

However if the page could possibly get access to the API, if only granted access by the user, then we should expose the API. This way the page could surface UI to the user saying "you can't use this part of the app because you've denied me access to the necessary APIs". This helps with the case when a user has chosen "no" and "remember this decision" in a security dialog, but then forgotten about it.

This also makes the "API not implemented" and "hardware lacking support" behavior act the same, making the page have fewer behaviors to deal with. It also means that we can add APIs to signed-for apps without worrying about polluting the public web.


Regarding what navigator.hasFeature(X) should return:

It should return true if the "platform" could support the API if the page had been granted the sufficient permission.

Where "platform" means "device hardware and UA software".

So if there is any theoretical possibility for a page to get sufficient permissions to have the API work on the given device hardware and UA software, then hasFeature should return true.
Whiteboard: [depency marketplace]
Whiteboard: [depency marketplace] → [depedency marketplace]
Whiteboard: [depedency marketplace] → [dependency marketplace]
Whiteboard: [dependency marketplace] → [dependency: marketplace]
What are the next actions here (if any)?
If I could summarize (correct me if I'm wrong):
Implement comment #22 as part of Bug 1009645 (I think)
navigator.hasFeature("manifest.foo.bar")
where it doesn't return a field/value pair but just tests to see if the the manifest feature is supported with and that:
* the database needs to be kept up to date
* the manifest field should not change type or functionality in the future.
* It should not be privileged.

So should this be added to Bug 1009645?

Comment 55

4 years ago
Which manifest fields is MarketPlace interested in detecting, and what are the use cases?  (I want to make sure that there is no other way of detecting those features elsewhere in the platfrom.)

Note that the getFeature() API implemented in bug 1009645 requires a permission which will only be granted to the MarketPlace app for now, because of the privacy concerns in exposing the amount of the memory on the device to non-privileged code (the "hardware.memory" part of the API.)
Flags: needinfo?(dbialer)
(In reply to :Ehsan Akhgari (lagging on bugmail, needinfo? me!) from comment #55)
> Note that the getFeature() API implemented in bug 1009645 requires a
> permission which will only be granted to the MarketPlace app for now,
> because of the privacy concerns in exposing the amount of the memory on the
> device to non-privileged code (the "hardware.memory" part of the API.)

I think I missed where there was talk about a permission being needed to use the feature detection API - one of the original goals was to enable non-privileged content to be able to detect APIs that may be privileged.  When you say it will 'only be granted to Marketplace' - is that only the app?  Not the website?  How is this limitation enforced?
Flags: needinfo?(ehsan)

Comment 57

4 years ago
(In reply to Andrew Williamson [:eviljeff] from comment #56)
> (In reply to :Ehsan Akhgari (lagging on bugmail, needinfo? me!) from comment
> #55)
> > Note that the getFeature() API implemented in bug 1009645 requires a
> > permission which will only be granted to the MarketPlace app for now,
> > because of the privacy concerns in exposing the amount of the memory on the
> > device to non-privileged code (the "hardware.memory" part of the API.)
> 
> I think I missed where there was talk about a permission being needed to use
> the feature detection API - one of the original goals was to enable
> non-privileged content to be able to detect APIs that may be privileged. 
> When you say it will 'only be granted to Marketplace' - is that only the
> app?  Not the website?  How is this limitation enforced?

I tried to get this past the privacy team (see the lengthy discussions on bug 983799 and its dependencies).  The issue is that the privacy team is not OK with exposing "hardware.memory" through this API because of possible fingerprinting issue, so we decided to expose it behind a permission which we can grant to the MarketPlace app (but not the website yet) in order to unblock the MarketPlace app.  We still need to figure out how to expose this to non-privileged web.

The limitation will be enforced through the MarketPlace review process.  I asked around to figure out who to talk to in order to enforce that, and I think I need to talk to Lisa Brewster about that (and please correct me if she's not the right person), but I've been waiting for the Gecko work to finish before I reach out to her.
Flags: needinfo?(ehsan)
(In reply to :Ehsan Akhgari (lagging on bugmail, needinfo? me!) from comment #57)
> I tried to get this past the privacy team (see the lengthy discussions on
> bug 983799 and its dependencies).  The issue is that the privacy team is not
> OK with exposing "hardware.memory" through this API because of possible
> fingerprinting issue, so we decided to expose it behind a permission which
> we can grant to the MarketPlace app (but not the website yet) in order to
> unblock the MarketPlace app.  We still need to figure out how to expose this
> to non-privileged web.

okay.  Well I won't attempt to re-hash the privacy discussions but it is limiting how useful it will be, especially for non-Mozilla app-stores.  And Marketplace will actually need to become a proper packaged app too (currently its just a non-privileged shim app with essentially an iframe to the website)

> The limitation will be enforced through the MarketPlace review process.  I
> asked around to figure out who to talk to in order to enforce that, and I
> think I need to talk to Lisa Brewster about that (and please correct me if
> she's not the right person), but I've been waiting for the Gecko work to
> finish before I reach out to her.

I report to Lisa, so you can consider your reaching out delegated to me :)
(In reply to :Ehsan Akhgari (lagging on bugmail, needinfo? me!) from comment #55)
> Which manifest fields is MarketPlace interested in detecting, and what are
> the use cases?  (I want to make sure that there is no other way of detecting
> those features elsewhere in the platfrom.)

Fx1.1
chrome|navigation
origin
redirects

Fx2.0
precompile
role=input (i.e. 3rd party keyboards)

Fx??
role=lockscreen

(maybe more)
I think Andrew captured these correctly - though I don't know about the role and redirects field. The role field seems to be undocumented at https://developer.mozilla.org/en-US/Apps/Build/Manifest and not clear if redirects is a new addition to a particular version (also seems undocumented if it was introduced later than Fx1.0)and therefore something that needs to be detected on the platform.

Comment 61

4 years ago
(In reply to comment #58)
> (In reply to :Ehsan Akhgari (lagging on bugmail, needinfo? me!) from comment
> #57)
> > I tried to get this past the privacy team (see the lengthy discussions on
> > bug 983799 and its dependencies).  The issue is that the privacy team is not
> > OK with exposing "hardware.memory" through this API because of possible
> > fingerprinting issue, so we decided to expose it behind a permission which
> > we can grant to the MarketPlace app (but not the website yet) in order to
> > unblock the MarketPlace app.  We still need to figure out how to expose this
> > to non-privileged web.
> 
> okay.  Well I won't attempt to re-hash the privacy discussions but it is
> limiting how useful it will be, especially for non-Mozilla app-stores.  And
> Marketplace will actually need to become a proper packaged app too (currently
> its just a non-privileged shim app with essentially an iframe to the website)

Yes, I understand that.  The current situation is unfortunate.  We should try again in the future.

> > The limitation will be enforced through the MarketPlace review process.  I
> > asked around to figure out who to talk to in order to enforce that, and I
> > think I need to talk to Lisa Brewster about that (and please correct me if
> > she's not the right person), but I've been waiting for the Gecko work to
> > finish before I reach out to her.
> 
> I report to Lisa, so you can consider your reaching out delegated to me :)

Oh awesome!  :-)

Comment 62

4 years ago
(In reply to comment #59)
> (In reply to :Ehsan Akhgari (lagging on bugmail, needinfo? me!) from comment
> #55)
> > Which manifest fields is MarketPlace interested in detecting, and what are
> > the use cases?  (I want to make sure that there is no other way of detecting
> > those features elsewhere in the platfrom.)
> 
> Fx1.1
> chrome|navigation
> origin
> redirects
> 
> Fx2.0
> precompile
> role=input (i.e. 3rd party keyboards)
> 
> Fx??
> role=lockscreen
> 
> (maybe more)

Thanks, this is helpful!

Comment 63

4 years ago
(In reply to comment #60)
> I think Andrew captured these correctly - though I don't know about the role
> and redirects field. The role field seems to be undocumented at
> https://developer.mozilla.org/en-US/Apps/Build/Manifest and not clear if
> redirects is a new addition to a particular version (also seems undocumented if
> it was introduced later than Fx1.0)and therefore something that needs to be
> detected on the platform.

I don't know the answers to these questions right now, will need to check the code.  But I just realized something.  So us adding these names to getFeature() at this point is going to be basically unconditionally true, because all of the items in comment 59 are already supported features.  Is this the issue that you're trying to address?  Or are we trying to provide ways to feature detect the upcoming changes to the manifest format in the future?
(In reply to David Bialer [:dbialer] from comment #60)
> I think Andrew captured these correctly - though I don't know about the role
> and redirects field. The role field seems to be undocumented at
> https://developer.mozilla.org/en-US/Apps/Build/Manifest 

yeah, not sure why its missing.  afaik role was originally only use for system apps - to hide them from the homescreen - then got overloaded later so I guess the lack of documenting was an oversight.

> and not clear if
> redirects is a new addition to a particular version (also seems undocumented
> if it was introduced later than Fx1.0)and therefore something that needs to
> be detected on the platform.

redirects was added in the same version as origin (1.1)
(In reply to :Ehsan Akhgari (lagging on bugmail, needinfo? me!) from comment #63)
> I don't know the answers to these questions right now, will need to check
> the code.  But I just realized something.  So us adding these names to
> getFeature() at this point is going to be basically unconditionally true,
> because all of the items in comment 59 are already supported features.  Is
> this the issue that you're trying to address?  Or are we trying to provide
> ways to feature detect the upcoming changes to the manifest format in the
> future?

I suppose yes, they would be true - or true on FxOS at least.  Android and Desktop would presumably return false for the role, and role:input may be false in some circumstances (see bug 962656).
Even though they would be always be true, it would still be better to have the support for them there - we want to avoid the kind of fuzzy logic that says 'if getFeature exists then precompile does too'.  And this would be the way upcoming manifest changes would be detected also.
Flags: needinfo?(dbialer)
(In reply to Andrew Williamson [:eviljeff] from comment #59)
> (In reply to :Ehsan Akhgari (lagging on bugmail, needinfo? me!) from comment
> #55)
> > Which manifest fields is MarketPlace interested in detecting, and what are
> > the use cases?  (I want to make sure that there is no other way of detecting
> > those features elsewhere in the platfrom.)
> 
> Fx1.1
> chrome|navigation
> origin
> redirects
> 
> Fx2.0
> precompile
> role=input (i.e. 3rd party keyboards)
> 
> Fx??
> role=lockscreen

Can you provide more details about how you plan to use the information provided by getFeature for these manifest fields?  It'll help know if there is a different/existing way to get this information.  Thanks.
Flags: needinfo?(awilliamson)
(In reply to Andrew Overholt [:overholt] from comment #66)
> Can you provide more details about how you plan to use the information
> provided by getFeature for these manifest fields?  It'll help know if there
> is a different/existing way to get this information.  Thanks.

One way we'd use it: to determine if a device is capable of supporting apps that require <feature> to be useful, ensuring that Marketplace users aren't presented with apps completely useless to them [1]. We do that by passing with each API request a profile [2] of features that the user agent does and does not support. We use that profile as query filters [3] when retrieving apps.

So, a somewhat trivial example: we'd want to hide third-party keyboards if role=input is not supported. That would result in a 0 in the appropriate position in the feature profile bitstring, which would result in an additional filter in our queries.

[1] http://firefox-marketplace-api.readthedocs.org/en/latest/topics/features.html
[2] http://firefox-marketplace-api.readthedocs.org/en/latest/topics/search.html#feature-profile-label
[3] https://github.com/mozilla/zamboni/blob/master/mkt/constants/features.py#L317-L409
Flags: needinfo?(awilliamson)

Comment 68

4 years ago
(In reply to Chuck Harmston [:chuck] from comment #67)
> (In reply to Andrew Overholt [:overholt] from comment #66)
> > Can you provide more details about how you plan to use the information
> > provided by getFeature for these manifest fields?  It'll help know if there
> > is a different/existing way to get this information.  Thanks.
> 
> One way we'd use it: to determine if a device is capable of supporting apps
> that require <feature> to be useful, ensuring that Marketplace users aren't
> presented with apps completely useless to them [1]. We do that by passing
> with each API request a profile [2] of features that the user agent does and
> does not support. We use that profile as query filters [3] when retrieving
> apps.

Yes, that makes perfect sense.  This is what I had in mind when designing navigator.getFeature().

> So, a somewhat trivial example: we'd want to hide third-party keyboards if
> role=input is not supported. That would result in a 0 in the appropriate
> position in the feature profile bitstring, which would result in an
> additional filter in our queries.

We discussed that in another bug which I can't find right now with Tim, and IIRC the conclusion was that we should move the pref which determines whether gaia supports third party keyboards to Gecko and then have the API be hidden behind that pref and also feature detectible based on the same pref.  I believe that will fix the third party keyboard use case.
Ehsan, Andrew, and I just spoke via videoconferencing to clear up any lingering confusion here.

- the marketplace goal of not exposing apps that won't work on a device isn't always possible with DOM APIs
- the problem with manifest parsing is that gecko and gaia just discard bogus values so it's hard to say when an error will be thrown
- also, manifest values being "correct" doesn't guarantee the underlying code will be present or functional so having getFeature return true for them is likely more harmful than helpful
- it's worth adding certain manifest fields to getFeature (ex. chrome, origin, redirects) since they're not possible to detect via other, more preferable methods
- Ehsan cautioned that getFeature is very gecko-specific and is unlikely to be implemented by other engines/UAs in (at least) its current form

We agreed for the future:

- we'll consider each manifest field on a case-by-case basis to ensure there's no better way to detect device capability before adding it to getFeature as a last resort
- we should ensure those adding manifest fields are careful about making their functionality detectable via reliable methods

Worth adding to getFeature:
- chrome|navigation
- origin
- redirects

Not likely to be needed in getFeature:
- precompile (fraught with other potential reasons it won't work and is a first-time startup optimization)

More investigation required (not yet landed in FxOS):
- role=lockscreen

Comment 70

4 years ago
Needinfo-ing myself so that I keep this on my radar...  Will try to get to it next week.
Flags: needinfo?(ehsan)
What's the timescale on the implementation of this API?

Comment 72

4 years ago
(In reply to Andrew Williamson [:eviljeff] from comment #71)
> What's the timescale on the implementation of this API?

Bug 1009645 is still lingering...  It's not clear if this will make it for 2.1 at this point.
status-b2g-v2.0: --- → ?
blocking-b2g: - → 2.0?
Feature request, which means this isn't a blocker.
blocking-b2g: 2.0? → -
status-b2g-v2.0: ? → ---
Blocks: 1035380

Comment 74

4 years ago
I documented the API in <https://wiki.mozilla.org/WebAPI/Navigator.hasFeature#.22manifest.22_namespace>.  For the implementation, as there is no clean way of tying this to the code that consumes these flags from the manifest, we probably have a static list of these values and resolve the promise to true if we encounter one of them.

Note that the manifest.precompile name will not tell you whether the asm.js precompilation will work or not on the device, as that can also depend on other factors such as the amount of free disk space available.
Flags: needinfo?(ehsan)

Comment 75

4 years ago
Reuben, are you interested in taking a stab at this too?
Flags: needinfo?(reuben.bmo)
I think we will need another manifest field supported.
manifest.role.value

where value=input or lockscreen (not sure what else keyboard?).  This seems to be evolving.

I understand this is a new field and will be in 2.0.
Andy W may know more details.
Flags: needinfo?(awilliamson)
(In reply to David Bialer [:dbialer] from comment #76)
> I think we will need another manifest field supported.
> manifest.role.value
> 
> where value=input or lockscreen (not sure what else keyboard?).  This seems
> to be evolving.

I'm not clear if its role=homescreen (for replaceable homescreens) or role=lockscreen (for lockscreen widgets), or which will be implemented for which version.  Are there any bugs to refer to yet?

For role=input (3rd party keyboards) I believe it will be detected a different way, via the 'api.window.Navigator.mozInputMethod' feature (bug 962656). We could have role=input as an alternative way for completeness (so all the valid value of role are detectable), but its not strictly needed afaik.
Flags: needinfo?(awilliamson)
(In reply to :Ehsan Akhgari (lagging on bugmail, needinfo? me!) from comment #75)
> Reuben, are you interested in taking a stab at this too?

Yep. Though I might not be the best assignee if this has to get to 2.0.
Flags: needinfo?(reuben.bmo)

Comment 79

4 years ago
(In reply to Andrew Williamson [:eviljeff] from comment #77)
> (In reply to David Bialer [:dbialer] from comment #76)
> > I think we will need another manifest field supported.
> > manifest.role.value
> > 
> > where value=input or lockscreen (not sure what else keyboard?).  This seems
> > to be evolving.
> 
> I'm not clear if its role=homescreen (for replaceable homescreens) or
> role=lockscreen (for lockscreen widgets), or which will be implemented for
> which version.  Are there any bugs to refer to yet?

I'm not sure.

> For role=input (3rd party keyboards) I believe it will be detected a
> different way, via the 'api.window.Navigator.mozInputMethod' feature (bug
> 962656). We could have role=input as an alternative way for completeness (so
> all the valid value of role are detectable), but its not strictly needed
> afaik.

We might need to discuss these issues on a case by case basis as they come up.  Let's not extend the scope of this bug, if you have any other fields you'd like to detect I'd appreciate if you could file separate bugs.  Thanks!
(Assignee)

Comment 80

4 years ago
Created attachment 8452870 [details] [diff] [review]
manifest-feature.patch

We really don't have many things to test right now...
Attachment #8452870 - Flags: review?(ehsan)

Comment 81

4 years ago
Comment on attachment 8452870 [details] [diff] [review]
manifest-feature.patch

Review of attachment 8452870 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/base/Navigator.cpp
@@ +1521,5 @@
> +    "manifest.origin"
> +  , "manifest.redirects"
> +#ifdef MOZ_B2G
> +  , "manifest.chrome.navigation"
> +  , "manifest.role.input"

We shouldn't accept role.input here.  But we should accept precompile.  (You need to fix the test too.)
Attachment #8452870 - Flags: review?(ehsan) → review+
Backed out for mochitest-2 desktop and mochitest-4 android failures: https://tbpl.mozilla.org/php/getParsedLog.php?id=43470568&tree=B2g-Inbound

https://hg.mozilla.org/integration/b2g-inbound/rev/08adad9d0ddf
Flags: needinfo?(fabrice)
https://hg.mozilla.org/mozilla-central/rev/19c2c4b4993b
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla33
Comment on attachment 8452870 [details] [diff] [review]
manifest-feature.patch

Approval Request Comment
[Feature/regressing bug #]:
[User impact if declined]: Users install apps that don't work.  Developers get bad reviews because apps don't work.
[Describe test coverage new/current, TBPL]:
[Risks and why]: As discussed in the b2g-drivers meeting, we were going to finish this patch and then evaluate if we could take it for 2.0.  As it stands today, the Marketplace cannot determine if an app will run on your device or not.
[String/UUID change made/needed]:
Attachment #8452870 - Flags: approval-mozilla-aurora?
Comment on attachment 8452870 [details] [diff] [review]
manifest-feature.patch

This is not the patch that landed on m-c.

Fabrice - Unless I've missed something I think you need at attach the patch that actually landed.
Attachment #8452870 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora-
Flags: needinfo?(fabrice)
(In reply to Wil Clouser [:clouserw] from comment #86)
> [Describe test coverage new/current, TBPL]:
Tests are included in the patch.

I confirmed with Ehsan that this additional information is only available to the marketplace app. I will approve as soon as I know what patch I should approve.
(Assignee)

Comment 89

4 years ago
Created attachment 8454695 [details] [diff] [review]
19c2c4b4993b


Approval Request Comment
[Feature/regressing bug #]:
[User impact if declined]: Users install apps that don't work.  Developers get bad reviews because apps don't work.
[Describe test coverage new/current, TBPL]:
Tests are part of the patch.
[Risks and why]: As discussed in the b2g-drivers meeting, we were going to finish this patch and then evaluate if we could take it for 2.0.  As it stands today, the Marketplace cannot determine if an app will run on your device or not.
[String/UUID change made/needed]: None
Attachment #8454695 - Flags: approval-mozilla-aurora?
Flags: needinfo?(fabrice)
Comment on attachment 8454695 [details] [diff] [review]
19c2c4b4993b

Thanks Fabrice. Aurora+
Attachment #8454695 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
status-b2g-v2.0: --- → fixed
status-b2g-v2.1: --- → fixed
status-firefox31: --- → wontfix
status-firefox32: --- → fixed
status-firefox33: --- → fixed
Blocks: 1047100
No longer blocks: 1047100

Updated

3 years ago
Blocks: 1217187

Updated

7 months ago
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.