Closed Bug 910150 Opened 11 years ago Closed 7 years ago

[Sensor] There is no way to enable acceleration or gyro individually from Web API

Categories

(Firefox OS Graveyard :: General, defect)

ARM
Gonk (Firefox OS)
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: mchen, Unassigned)

References

Details

Currently when gaia listen to DeviceMotion event, Gecko will enable acceleration and gyro sensors in the same time and there is no way to enable one of them only. This introduced the issue on wasting power because Web App may need to listen one of them only. Ex: Teeter app in Android will use acceleration sensor's data to control the game but in FxOS we will enable both of acceleration and gyro sensors. That is not a good case for saving power. Ex: Nexus 4 had both of these sensors.
I think that we should it like android. In android, we have SensorManager Class which allows to listen for any particular sensor type. If firefox, we don't have anything like that. @Marco, are you thinking to implement something like this as webapi : http://developer.android.com/reference/android/hardware/SensorManager.html
Flags: needinfo?(mchen)
We are discussing it on the "dev-webapi@lists.mozilla.org". According to this is standard Web API, we need to find out a proposal and summit to W3C.
Flags: needinfo?(mchen)
Hi Doug, I think that since the way to listen devicemotion event is a standard in W3C now so we can't modify it. So maybe we can add a new DOM element called "SensorManager" which is used to control the sensor's life cycle and the way for listening sensor data is still from devicemotion. Ex: interface SensorManager { attribute DOMString sensorList; boolean enableSensor(DOMString aSensorName); boolean disableSensor(DOMString aSensorName); }; May I know your suggestion? Thanks.
Flags: needinfo?(doug.turner)
Did you look at the w3c geolocation or deviceapi mailing lists? I think it was from rick waldron had some good ideas. Rick, do you want to point Marco in the right direction?
Flags: needinfo?(doug.turner)
(In reply to Marco Chen [:mchen] (PTO, 09/16, 09/18~09/22) from comment #0) > Currently when gaia listen to DeviceMotion event, Gecko will enable > acceleration and gyro sensors in the same time and there is no way to enable > one of them only. This introduced the issue on wasting power because Web App > may need to listen one of them only. > > Ex: Teeter app in Android will use acceleration sensor's data to control the > game but in FxOS we will enable both of acceleration and gyro sensors. That > is not a good case for saving power. > > Ex: Nexus 4 had both of these sensors. As I've stated in the past[0][1][2], the existing window-targeted sensor events (eg. DeviceMotion) are a poor design for JavaScript programs that must run in browsers. Semantically, the "window" object should not be the target of the events, it has nothing to do with sensing the physical world. There is also no reasonable way to represent more then one of any kind of sensor in a JavaScript program. The case reported here is more evidence to support designing and implementing a replacement programming model for device sensory APIs—ultimately to deprecate and abandon the existing design. Yes, it will take time. Yes, implementors may balk. The problem that's faced here is the result of an over-engineered design: "orientation" and "motion" are always calculated by fusing the values of more then one sensor. In this case when only part of the data is needed (ie. the acceleration) the entire device must suffer the power consumption needs of both the accelerometer and the gyroscope (and likely the magnetometer). As reported, Accelerometer and Gyro are related, but not enough to warrant mutual inclusion by default. In the past I've proposed a design that uses constructors to produce sensor instance objects on an _as-needed_ basis. The following is an overview of that proposed solution. ----------------- Sensory APIs are defined in a top level Sensor object, whose properties are constructors for specific device sensors: Sensor { Accelerometer Gyroscope Magnetometer Light Proximity Temperature (device, not environment) ... } - If a specific sensor is not available for a given device, the constructor's value is null. - Each constructor produces an instance object that represents a device sensor. - Each instance includes either of the following: - A single readonly "value" property for sensors that produce a single value. - Proximity: value (cm) - Light: value (lx) - Temperature: value (celsius) - Multiple readonly properties for sensors that produce multiple values - Accelerometer: x, y, z (g, where 1g = 9.8m/s^2) - Gyroscope: x, y, z (rad/s) - Magnetometer: x, y, z (gauss) - Each instance includes a "threshold" property that is used by the implementation to determine the threshold at which the "change" (see below) event is fired. The value of the "threshold" property can be any number which is representable in JavaScript. - Specific instances will have additional "calibration" data properties, to be defined. - Each instance object inherits EventTarget - The "data" event (needs further definition). - The event must fire on the sensor instance object. - The event should fire on a constant, implementation independent interval, however the interval is recommended to match the current frame rate of requestAnimationFrame (this needs more attention). - The "change" event (needs further definition). - The event must fire on the sensor instance object. - The event should fire whenever there is a significant change in the sensor value. The definition of "significant change" is when the sensor's value changes between two "data" events and the difference of that change is greater then then value specified by the "threshold" property. This approach solves the reported issue by giving a browser based JavaScript program explicit control over device sensory observation. Here are the solutions to the two reported example cases: > Ex: Teeter app in Android will use acceleration sensor's data to control the game but in FxOS we will enable both of acceleration and gyro sensors. That is not a good case for saving power. var accel = new Sensor.Accelerometer(); // Set a threshold for x, y, z changes accel.threshold = 0.5; accel.addEventListener("change", function() { // "move" the screen when the value of any of the following has // changed > 0.5 between two data readings: // // - this.x // - this.y // - this.z // }); > Ex: Nexus 4 had both of these sensors. var accel = new Sensor.Accelerometer(); var gyro = new Sensor.Gyroscope(); var orientation; // Update orientation only when significant changes occur... accel.addEventListener("change", function() { orientation = calculate(accel, gyro); }); gyro.addEventListener("change", function() { orientation = calculate(accel, gyro); }); requestAnimationFrame(function loop() { // update the game world with "orientation" World.from(orientation); requestAnimationFrame(loop); }); With all due respect, the solution in comment 3 only makes the existing problem worse by acknowledging its flaws and covering them up. [0] http://lists.w3.org/Archives/Public/public-device-apis/2012Dec/0060.html [1] http://lists.w3.org/Archives/Public/public-device-apis/2013Jan/0023.html [2] http://lists.w3.org/Archives/Public/public-device-apis/2013Jan/0037.html
Hi all, Thanks for all your reply and I will register into w3c geolocation and deviceapi mailing lists. Hi Rick, May I know the status of your proposal? Is it near approved or under discussion? Thanks.
(In reply to Marco Chen [:mchen] (PTO, 09/16, 09/18~09/22) from comment #6) > Hi all, > > Thanks for all your reply and I will register into w3c geolocation and > deviceapi mailing lists. > > Hi Rick, > > May I know the status of your proposal? Is it near approved or under > discussion? There was discussion and ultimately rejected with the excuse that implementors will not want to replace their current implementation.
(In reply to Rick Waldron from comment #7) > (In reply to Marco Chen [:mchen] (PTO, 09/16, 09/18~09/22) from comment #6) > There was discussion and ultimately rejected with the excuse that > implementors will not want to replace their current implementation. Hi Rick, Their excuse is one of my reason to proposal SensorManager on comment 3 and no matter what we need a way to solve the issue here. 1. A simple and compatible way is to keep the sensor API now and add another mechanism to control sensors individually. 2. To proposal a entire new and non-compatible way into W3C. About the second way, I also like the proposal from Rick but the real problem is that "it seems to be not accepted by W3C currently". So could you re-consider the first way to solve the problem now? Even we used MozSensorXXX to solve it we still keep the w3c sensor API. May I know your suggestion?
(In reply to Marco Chen [:mchen] (PTO, 09/16, 09/18~09/22) from comment #8) > (In reply to Rick Waldron from comment #7) > > (In reply to Marco Chen [:mchen] (PTO, 09/16, 09/18~09/22) from comment #6) > > There was discussion and ultimately rejected with the excuse that > > implementors will not want to replace their current implementation. > > Hi Rick, > > Their excuse is one of my reason to proposal SensorManager on comment 3 and > no matter what we need a way to solve the issue here. > > 1. A simple and compatible way is to keep the sensor API now and add another > mechanism to control sensors individually. > 2. To proposal a entire new and non-compatible way into W3C. > > About the second way, I also like the proposal from Rick but the real > problem is that "it seems to be not accepted by W3C currently". So could you > re-consider the first way to solve the problem now? Even we used > MozSensorXXX to solve it we still keep the w3c sensor API. > > May I know your suggestion? Issues I can see with the original proposed solution: - How will program code know which sensor(s) in the list are currently enabled or disabled? Given the following from the DeviceMotion spec: > The information provided by the events is not raw sensor data, but rather high-level data > which is agnostic to the underlying source of information. Common sources of information > include gyroscopes, compasses and accelerometers. - How will program code know which sensors to disable? Device "A" might be providing data from Accelerometer and Gyro, while device "B" uses Accelerometer, Gyro and Compass. If a program disables "gyro", then device A and device B are going to have observably different data from DeviceMotion events. - 3rd party code might rely on DeviceMotion data being the expected fusion of Accelerometer and Gyro data (or Accelerometer, Gyro and Compass|Magnetometer). - eg. I might be developing a game and I disable "gyro" for some specific feature of my game, but simultaneously relying on some 3rd party game framework that uses DeviceMotion and expects the data to appear as the specification describes. - What happens when that same framework releases a new version that forces the Gyro back into enabled state? The data received by _my_ program will become observably different from the data that is expected when the Gyro is disabled. - Why is the list a DOMString? Is this a space separated string? - This is more of a nit, but why does every member of "SensorManager" also have the word "sensor" in it? Isn't it clear enough by the object's name that those members are "sensor" specific tasks? eg. interface SensorManager { attribute DOMString sensorList; boolean enableSensor(DOMString aSensorName); boolean disableSensor(DOMString aSensorName); }; vs. interface SensorManager { attribute DOMString list; boolean enable(DOMString name); boolean disable(DOMString name); };
(In reply to Rick Waldron from comment #9) > (In reply to Marco Chen [:mchen] (PTO, 09/16, 09/18~09/22) from comment #8) > - How will program code know which sensor(s) in the list are currently > enabled or disabled? Maybe we can add more one function to get the status of sensor. (thanks for this suggestion) > - How will program code know which sensors to disable? Device "A" might be > providing data from Accelerometer and Gyro, while device "B" uses > Accelerometer, Gyro and Compass. If a program disables "gyro", then device A > and device B are going to have observably different data from DeviceMotion > events. SensorManager.list will return what sensors this device supports so user know what sensors can be controlled by enable/disable Web APIs. I think the all sensor types in DeviceMotion should be enabled by default (following the current spec) then user can disable anyone based on SensorManager.list. > - 3rd party code might rely on DeviceMotion data being the expected fusion > of Accelerometer and Gyro data (or Accelerometer, Gyro and > Compass|Magnetometer). This can be based on Window. The window should know the status from SensorManager then filter out the event which is enabled by another window. > > - Why is the list a DOMString? Is this a space separated string? Yes, it just return an array of sensors. > - This is more of a nit, but why does every member of "SensorManager" also > have the word "sensor" in it? Isn't it clear enough by the object's name > that those members are "sensor" specific tasks? > interface SensorManager { > attribute DOMString list; > boolean enable(DOMString name); > boolean disable(DOMString name); > }; You are right. --------------------------------------------------- I am not familiar with Web API and it's rule so thanks for your great suggestion here. Now it will be // By default, sensors are all enabled corresponding to a DevceXXXEvent. // User should explicitly to disable one of sensors by this Web API. interface SensorManager { attribute DOMString list; boolean enable(DOMString name); boolean disable(DOMString name); boolean isEnabled(DOMString name); };
Flags: needinfo?(waldron.rick)
Hi Doug, After checking with Rick, his proposal was rejected by device api group already. And the reason is the same with Comment 3. So may I know that to add a new SensorManager for controlling life cycle of each sensor is a good way or not? Thanks.
Flags: needinfo?(doug.turner)
(In reply to Marco Chen [:mchen] (Summit 10/03 ~10/08) from comment #10) > (In reply to Rick Waldron from comment #9) > > (In reply to Marco Chen [:mchen] (PTO, 09/16, 09/18~09/22) from comment #8) > > - How will program code know which sensor(s) in the list are currently > > enabled or disabled? > Maybe we can add more one function to get the status of sensor. (thanks for > this suggestion) > > > - How will program code know which sensors to disable? Device "A" might be > > providing data from Accelerometer and Gyro, while device "B" uses > > Accelerometer, Gyro and Compass. If a program disables "gyro", then device A > > and device B are going to have observably different data from DeviceMotion > > events. > SensorManager.list will return what sensors this device supports so user > know what sensors can be controlled by enable/disable Web APIs. > I think the all sensor types in DeviceMotion should be enabled by default > (following the current spec) then user can disable anyone based on > SensorManager.list. This still doesn't address the issue that I've asked about. How will an application's program code know which sensors to disable when the _same_ app is run on different devices? What happens when the web application is run on Android? iPhone? > > > - 3rd party code might rely on DeviceMotion data being the expected fusion > > of Accelerometer and Gyro data (or Accelerometer, Gyro and > > Compass|Magnetometer). > This can be based on Window. The window should know the status from > SensorManager then filter out the event which is enabled by another window. I don't see how this answers my question. > > > > > - Why is the list a DOMString? Is this a space separated string? > > Yes, it just return an array of sensors. Can you clarify this answer? I asked if it was a "space separated string" and you said "Yes", but then you said "it just return an array of sensors", which is not a string and is not what DOMString returns. Is this an array of DOMString(s)? > > > - This is more of a nit, but why does every member of "SensorManager" also > > have the word "sensor" in it? Isn't it clear enough by the object's name > > that those members are "sensor" specific tasks? > > interface SensorManager { > > attribute DOMString list; > > boolean enable(DOMString name); > > boolean disable(DOMString name); > > }; > > You are right. > > --------------------------------------------------- > > I am not familiar with Web API and it's rule so thanks for your great > suggestion here. > Now it will be > > // By default, sensors are all enabled corresponding to a DevceXXXEvent. > // User should explicitly to disable one of sensors by this Web API. > interface SensorManager { > attribute DOMString list; > boolean enable(DOMString name); > boolean disable(DOMString name); > boolean isEnabled(DOMString name); > }; Great, glad to see this addressed. I'm still concerned that it's completely unclear what relationship is, between DeviceMotion and SensorManager.enable|disable(...). If DeviceMotion values are the fusion of an accelerometer and a gyroscope on a given device, won't SensorManager.disable("gyroscope") change the resultant values of a DeviceMotion event? If I write an physical world animation library that relies on DeviceMotion values being what DeviceMotion is specifies they will be, what happens when this library is used by a FirefoxOS app that is disabling one of the sensors for some other purpose?
Flags: needinfo?(waldron.rick)
Hi Rick, Thanks for the feedback. ------------------------------------ 1. attribute DOMString list; -> sequence<DOMString> getSensorsList(); Change a way for web content to get supported sensors list on a given device. /* return an array of sensor identifiers, e.g. [ "acceleration", "gyro", "light" ] */ 2. User story: device: DeviceMotion will have both acceleration and gryo events. case 1: App 1: a. enabling to receive DeviceMotion event. b. expect to receive both acceleration and gyro events. case 2: App 1: a. enabling to receive DeviceMotion event. b. call SensorManager.disable("gyro"). c. expect to receive acceleration event only. note: device really disabled the power of gyro sensor. case 3: App 1: a. enabling to receive DeviceMotion event. b. expect to receive both acceleration and gyro events. App 2: a. enabling to receive DeviceMotion event. b. call SensorManager.disable("gyro"). c. expect to receive acceleration event only. note: device didn't really disable the power of gyro sensor. ---------------------------- Can this answer your question? Thanks.
Flags: needinfo?(waldron.rick)
Hi Tapas, I know that you also raise the request for this issue so could you give me a hand here? The device equipped both acceleration and gyro sensors is nexus 4 (what I had now) but according to embedded battery I can't measure the power difference between enabling both of them and just acceleration sensors. So could you help to give a experiment for these difference. I think the power difference is not only came from sensors themselves but also the CPU idle time. Maybe the measurement can be based on an average power consumption of a period of time. Thanks.
Flags: needinfo?(tkundu)
(In reply to Marco Chen [:mchen] (Business Trip 10/21 ~ 10/23) from comment #13) > Hi Rick, > > Thanks for the feedback. > > ------------------------------------ > > 1. attribute DOMString list; -> sequence<DOMString> getSensorsList(); > Change a way for web content to get supported sensors list on a given > device. > > /* return an array of sensor identifiers, e.g. > [ "acceleration", "gyro", "light" ] > */ Great > 2. User story: > device: DeviceMotion will have both acceleration and gryo events. > > case 1: > App 1: a. enabling to receive DeviceMotion event. > b. expect to receive both acceleration and gyro events. Confirmed > > case 2: > App 1: a. enabling to receive DeviceMotion event. > b. call SensorManager.disable("gyro"). > c. expect to receive acceleration event only. > > note: device really disabled the power of gyro sensor. Confirmed > > case 3: > App 1: a. enabling to receive DeviceMotion event. > b. expect to receive both acceleration and gyro events. > > App 2: a. enabling to receive DeviceMotion event. > b. call SensorManager.disable("gyro"). > c. expect to receive acceleration event only. > > note: device didn't really disable the power of gyro sensor. This is a problem. App 2 is expecting the measurement value to _only_ include an acceleration measurement, but it will receive the fusion of both gyroscopic (orientation) measurement and acceleration.
Flags: needinfo?(waldron.rick)
(In reply to Rick Waldron [:rwaldron] from comment #15) > (In reply to Marco Chen [:mchen] (Business Trip 10/21 ~ 10/23) from comment > > > > case 3: > > App 1: a. enabling to receive DeviceMotion event. > > b. expect to receive both acceleration and gyro events. > > > > App 2: a. enabling to receive DeviceMotion event. > > b. call SensorManager.disable("gyro"). > > c. expect to receive acceleration event only. > > > > note: device didn't really disable the power of gyro sensor. > > This is a problem. App 2 is expecting the measurement value to _only_ > include an acceleration measurement, but it will receive the fusion of both > gyroscopic (orientation) measurement and acceleration. So the implementation of browser engine should be responsible to record the sensor's power status of each app (window) then filter out unnecessary data. ex: Case 3 -> window 1 -> app1 -> acceleration event -> window 2 -> app 2 Sensors -> window 1 -> app1 -> gyro event -> window 2 (be filtered here so not send to app 2)
(In reply to Marco Chen [:mchen] (Business Trip 10/21 ~ 10/23) from comment #16) > (In reply to Rick Waldron [:rwaldron] from comment #15) > > (In reply to Marco Chen [:mchen] (Business Trip 10/21 ~ 10/23) from comment > > > > > > case 3: > > > App 1: a. enabling to receive DeviceMotion event. > > > b. expect to receive both acceleration and gyro events. > > > > > > App 2: a. enabling to receive DeviceMotion event. > > > b. call SensorManager.disable("gyro"). > > > c. expect to receive acceleration event only. > > > > > > note: device didn't really disable the power of gyro sensor. > > > > This is a problem. App 2 is expecting the measurement value to _only_ > > include an acceleration measurement, but it will receive the fusion of both > > gyroscopic (orientation) measurement and acceleration. > > So the implementation of browser engine should be responsible to record the > sensor's power status of each app (window) then filter out unnecessary data. > > ex: Case 3 > > -> window 1 -> app1 > -> acceleration event > -> window 2 -> app 2 > Sensors > -> window 1 -> app1 > -> gyro event > -> window 2 (be filtered here so not send to > app 2) I think there's been some misunderstanding by both of us. I wasn't originally considering the multiple window or multiple app cases because I assumed they would all be treated as unique realms. Please ignore my previous response. Here is my earlier concern, restated: If I'm writing a game that includes a JS physics library (3rd party) that was designed for use with DeviceMotion data (as it is specified here http://dev.w3.org/geo/api/spec-source-orientation.html#devicemotion), but my _own_ game code has shut off the gyro (same window, same app), then the JS library will be receiving unexpected data and may behave incorrectly or produce incorrect physics.
Hi Rick, This is a good point. But I think since they are belonged to the same app then they should handle such kind of issues by themselves.
(In reply to Marco Chen [:mchen] (Business Trip 10/21 ~ 10/23) from comment #18) > Hi Rick, > > This is a good point. But I think since they are belonged to the same app > then they should handle such kind of issues by themselves. Are you implying that application authors should change library code?
(In reply to Rick Waldron [:rwaldron] from comment #19) > Are you implying that application authors should change library code? I think since the app author tried to use a third party library then he needs to know the requirement of this library. If he know that 3rd party library will need both acceleration and gyro then he can't call SensorManager to disable one of them.
(In reply to Marco Chen [:mchen] (Business Trip 10/21 ~ 10/23) from comment #20) > (In reply to Rick Waldron [:rwaldron] from comment #19) > > Are you implying that application authors should change library code? > > I think since the app author tried to use a third party library then he > needs to know the requirement of this library. If he know that 3rd party > library will need both acceleration and gyro then he can't call > SensorManager to disable one of them. This is simply incorrect. A library consumer should not have to be concerned with the implementation details of the library.
Hi Rick, A library should provide it's minimal requirement for people who want to use it. Ex: library needs both of acceleration and gyro sensor data for processing or XXX memory footprint or others. Or we can't guarantee this runtime or device can let library work well. Based on this requirement, I still think app author should know the capability of any libraries it used.
(In reply to Marco Chen [:mchen] from comment #14) > Hi Tapas, > > I know that you also raise the request for this issue so could you give me a > hand here? > > The device equipped both acceleration and gyro sensors is nexus 4 (what I > had now) but according to embedded battery I can't measure the power > difference between enabling both of them and just acceleration sensors. So > could you help to give a experiment for these difference. > > I think the power difference is not only came from sensors themselves but > also the CPU idle time. Maybe the measurement can be based on an average > power consumption of a period of time. > > Thanks. Hi Marco, Do you want the exact numbers for each sensor's (gyro and acceleration) power consumption ? I need to discuss with my team to get the numbers. I do not know much about embedded battery. But i knew that we should not enable multiple sensors if it is not required for application. it will drain battery fast if you enable multiple sensors simultaneously,
Flags: needinfo?(tkundu)
(In reply to Tapas Kumar Kundu from comment #23) > Do you want the exact numbers for each sensor's (gyro and acceleration) > power consumption ? > I need to discuss with my team to get the numbers. Hi, I didn't only want to know the power consumption of each sensors themselves. I would like to know how they impact the entire system's power consumption. Ex: The each sensor event will break CPU's idle time then effect the power saving number by CPU Idle mechanism. So if possible, I may need to measure Fixed condition: 1. disable off peripheral's power (ex: ril, wifi, BT). 2. Set screen brightness to a certain value (ex: 200) 3. Set dim timeout to more then 1 minutes. Measurement: 1. Maybe use power monitor to count and calculate the average current within one minutes. Different condition: 1. Enabling acceleration sensor only. 2. Enabling acceleration and gyro sensor in the same time. > I do not know much about embedded battery. But i knew that we should not > enable multiple > sensors if it is not required for application. it will drain battery fast if > you enable multiple sensors simultaneously, I can't agree you any more.
Flags: needinfo?(mchen)
Flags: needinfo?(doug.turner)
Firefox OS is not being worked on
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.