Last Comment Bug 692955 - (camera) camera support for desktop <input accept=image/*> tag.
(camera)
: camera support for desktop <input accept=image/*> tag.
Status: NEW
: dev-doc-needed
Product: Firefox
Classification: Client Software
Component: General (show other bugs)
: unspecified
: All All
: -- normal with 18 votes (vote)
: ---
Assigned To: [:fabrice] Fabrice Desré
:
Mentors:
http://people.mozilla.com/~fdesre/cap...
: 451674 (view as bug list)
Depends on: 700232 749625 507749 508063 508082 565843 567323 621915 643692 718031
Blocks: 528886 webapi 741393
  Show dependency treegraph
 
Reported: 2011-10-07 15:10 PDT by [:fabrice] Fabrice Desré
Modified: 2015-04-17 07:50 PDT (History)
53 users (show)
dveditz: sec‑review?
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
part 1: backend (21.76 KB, patch)
2011-10-07 15:11 PDT, [:fabrice] Fabrice Desré
no flags Details | Diff | Splinter Review
part 2: front-end (34.57 KB, patch)
2011-10-07 15:12 PDT, [:fabrice] Fabrice Desré
no flags Details | Diff | Splinter Review
screenshot: capture picker (101.44 KB, image/png)
2011-10-07 15:16 PDT, [:fabrice] Fabrice Desré
no flags Details
screenshot: capture done! (103.94 KB, image/png)
2011-10-07 15:17 PDT, [:fabrice] Fabrice Desré
no flags Details
part 1 : gips backend (21.97 KB, patch)
2011-10-31 12:44 PDT, [:fabrice] Fabrice Desré
no flags Details | Diff | Splinter Review
part 2: capture picker UI (34.88 KB, patch)
2011-10-31 12:45 PDT, [:fabrice] Fabrice Desré
no flags Details | Diff | Splinter Review
part 3: xcode 4.2 support (637 bytes, patch)
2011-10-31 12:46 PDT, [:fabrice] Fabrice Desré
no flags Details | Diff | Splinter Review
gips backend updated for bitrot (23.64 KB, patch)
2012-01-03 13:08 PST, Randell Jesup [:jesup]
no flags Details | Diff | Splinter Review
capture UI updated for bitrot (38.72 KB, patch)
2012-01-03 13:09 PST, Randell Jesup [:jesup]
no flags Details | Diff | Splinter Review
gips backend updated for EXPORTS change (23.80 KB, patch)
2012-01-05 14:29 PST, Randell Jesup [:jesup]
no flags Details | Diff | Splinter Review
capture picker UI - updated for gkmedia m-c merge (38.68 KB, patch)
2012-01-20 13:05 PST, Randell Jesup [:jesup]
no flags Details | Diff | Splinter Review
Part 1: GIPS backend, updated for m-c merge (23.41 KB, patch)
2012-02-21 11:28 PST, Randell Jesup [:jesup]
no flags Details | Diff | Splinter Review
Part 2: capture picker UI, updated for m-c merge (38.45 KB, patch)
2012-02-21 11:29 PST, Randell Jesup [:jesup]
no flags Details | Diff | Splinter Review
Part 1: GIPS backend, updated for webrtc merge - non-functional (23.78 KB, patch)
2012-04-27 08:31 PDT, Randell Jesup [:jesup]
no flags Details | Diff | Splinter Review
Part 2: capture picker UI, updated for webrtc merge - non-functional (39.06 KB, patch)
2012-04-27 08:32 PDT, Randell Jesup [:jesup]
no flags Details | Diff | Splinter Review

Description [:fabrice] Fabrice Desré 2011-10-07 15:10:18 PDT
This bug is about provide support for still image capture from a camera using the <input accept=image/*> tag.

We use the gips library from the webrtc project
Comment 1 [:fabrice] Fabrice Desré 2011-10-07 15:11:30 PDT
Created attachment 565664 [details] [diff] [review]
part 1: backend

We create an async input stream that receives frames from gips.
Comment 2 [:fabrice] Fabrice Desré 2011-10-07 15:12:14 PDT
Created attachment 565666 [details] [diff] [review]
part 2: front-end

Showing the capture picker in a tab-modal dialog.
Comment 3 [:fabrice] Fabrice Desré 2011-10-07 15:16:54 PDT
Created attachment 565669 [details]
screenshot: capture picker
Comment 4 [:fabrice] Fabrice Desré 2011-10-07 15:17:17 PDT
Created attachment 565670 [details]
screenshot: capture done!
Comment 5 [:fabrice] Fabrice Desré 2011-10-07 15:18:14 PDT
These patches apply on top of the Alder project repository:
http://hg.mozilla.org/projects/alder
Comment 6 Justin Dolske [:Dolske] 2011-10-11 02:26:18 PDT
Is there a project/feature page for this? Also, you'll want to engage with UX to get a design for the front-end UI this adds.
Comment 7 [:fabrice] Fabrice Desré 2011-10-11 08:58:35 PDT
The feature page is here: https://wiki.mozilla.org/Platform/Features/Camera_API
Comment 8 Christopher Blizzard (:blizzard) 2011-10-11 14:57:22 PDT
*** Bug 451674 has been marked as a duplicate of this bug. ***
Comment 9 Ben Bucksch (:BenB) 2011-10-11 15:56:39 PDT
From dup bug:
Taking photos is extremely privacy sensitive and can, when used without the user noticing, represent a surveillance device inside my house. I personally don't feel comfortable seeing this feature *at all* in Firefox.

But, assuming this is accepted: Given the tricks we've seen with <input type="file"> (ask Jesse Ruderman), please be sure to have explicit consent from the user, in form of a dialog box that cannot possibly be accidentally clicked nor the user tricked into clicking. Inline/inpage approval is not OK.

<sarcasm>We could even do a "authorization=police" parameter, which skips the user confirmation.</sarcasm>
Comment 10 Randell Jesup [:jesup] 2011-10-11 23:05:34 PDT
Ben - see https://wiki.mozilla.org/Security/Discussions/WebRTC for the corollary to this bug - access to the camera/microphones for video and audio for WebRTC (and also for added security risk, screen-sharing).  Google has implemented much of this as Google Talk and Google+ Hangouts, and now Hangouts With Extras (which allows screen sharing).

Yes, this is *highly* security/privacy sensitive, and as part of this effort the IETF is looking at the security implications of the protocols and APIs (see ekr's draft from the above wiki page).
Comment 11 brunoais 2011-10-12 06:13:44 PDT
A nice way to do this is to have a request made to the user asking if you allow the loaded site to take a picture of you almost like the firefox's popup blocker.
Differences:
When you allow to be taken pictures, you allow the main page, not the frames.
Only the main page is allowed to take pictures. Any frames in it are always denied.
The page may request to take a picture whenever it wants but the picture is only taken when the user presses the <Enter> or the button to take the picture. That button is in a modal box made by the browser or the OS itself (like the file picker). The page has no control over that modal window.
Extra: The modal window should be useful for the user.
Allowing is by page load. Reloading a page means that the approval of allowing to take the picture is lost.
Comment 12 Ben Bucksch (:BenB) 2011-10-25 08:04:55 PDT
> A nice way to do this is to have a request made to the user asking if you
> allow the loaded site to take a picture of you almost like the firefox's popup blocker.
> The page has no control over that modal window.

+1

> when the user presses the <Enter>

That's subject to attacks (e.g. the "press enter game")
Comment 13 [:fabrice] Fabrice Desré 2011-10-31 12:44:18 PDT
Created attachment 570795 [details] [diff] [review]
part 1 : gips backend

updated gips backend
Comment 14 [:fabrice] Fabrice Desré 2011-10-31 12:45:27 PDT
Created attachment 570797 [details] [diff] [review]
part 2: capture picker UI

Fixes an osx only bug
Comment 15 [:fabrice] Fabrice Desré 2011-10-31 12:46:03 PDT
Created attachment 570798 [details] [diff] [review]
part 3: xcode 4.2 support
Comment 16 Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]] 2011-11-22 15:39:18 PST
removing sec-review-needed and marking the feature page instead
Comment 17 [:fabrice] Fabrice Desré 2011-11-28 17:09:22 PST
Mockups from Jennifer:
http://people.mozilla.com/~jboriss/specs/first_spec.png
Comment 18 Ben Bucksch (:BenB) 2011-11-29 08:59:50 PST
Looks great! And good from privacy perspective.

I also like that I can take a file from disk instead.
Comment 19 Robin Berjon 2011-11-30 08:09:11 PST
(In reply to Fabrice Desré [:fabrice] from comment #17)
> Mockups from Jennifer:
> http://people.mozilla.com/~jboriss/specs/first_spec.png

These look really nice and very promising.

Suggestion: it is not uncommon for websites to want pictures with a specific aspect ratio. For instance, profile pictures may have to be square, or portrait 4x3. This is often supported either with direct truncation (where the server side just cuts the picture without asking the user) or with some in-page script that allows the user to select which area to use, within constraints. A lot of those script solutions are a bit heavy handed, and aren't always very accessible. Since I note that you are already considering a way of selecting an area in the picture, it could be useful to allow for a constrained aspect ratio mode. This could be specified by the page author using one of the markup hints that we're looking into.

Other non-UI hints could be useful, such as indicating which size boundaries the author wants so that the UA could provide an already-resized image. Such small additions could really help authors.
Comment 20 Junior 2011-12-08 12:55:58 PST
Please, How I can to take a snapshot from a webcam in a desktop computer with Firefox?, What is the code supported?
Comment 21 Ian Melven :imelven 2011-12-08 13:01:19 PST
(In reply to Junior from comment #20)
> Please, How I can to take a snapshot from a webcam in a desktop computer
> with Firefox?, What is the code supported?

when this lands in desktop Firefox, you will be able to do this via <input accept=image/*> on a webpage.
Comment 22 Randell Jesup [:jesup] 2012-01-03 13:08:54 PST
Created attachment 585530 [details] [diff] [review]
gips backend updated for bitrot
Comment 23 Randell Jesup [:jesup] 2012-01-03 13:09:23 PST
Created attachment 585531 [details] [diff] [review]
capture UI updated for bitrot
Comment 24 Randell Jesup [:jesup] 2012-01-03 13:11:41 PST
Added URL for fabrice's test (perhaps you should put the code here)
Comment 25 Randell Jesup [:jesup] 2012-01-05 14:29:06 PST
Created attachment 586224 [details] [diff] [review]
gips backend updated for EXPORTS change
Comment 26 Randell Jesup [:jesup] 2012-01-20 13:05:36 PST
Created attachment 590312 [details] [diff] [review]
capture picker UI - updated for gkmedia m-c merge
Comment 27 Ted Mielczarek [:ted.mielczarek] 2012-01-23 04:51:25 PST
I pushed a couple of changes to alder:
http://hg.mozilla.org/projects/alder/rev/84616cd44046
http://hg.mozilla.org/projects/alder/rev/b25db6a3947e

With those changes, I was able to build with these patches applied and get working video capture on Windows.
Comment 28 Randell Jesup [:jesup] 2012-02-21 11:28:06 PST
Created attachment 599277 [details] [diff] [review]
Part 1: GIPS backend, updated for m-c merge
Comment 29 Randell Jesup [:jesup] 2012-02-21 11:29:05 PST
Created attachment 599278 [details] [diff] [review]
Part 2: capture picker UI, updated for m-c merge
Comment 30 Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]] 2012-03-21 14:38:21 PDT
need to schedule a full team review
Comment 31 Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]] 2012-04-02 12:11:23 PDT
Please follow the instructions on this page to schedule a review.
Comment 32 Randell Jesup [:jesup] 2012-04-27 08:31:52 PDT
Created attachment 619047 [details] [diff] [review]
Part 1: GIPS backend, updated for webrtc merge - non-functional
Comment 33 Randell Jesup [:jesup] 2012-04-27 08:32:10 PDT
Created attachment 619048 [details] [diff] [review]
Part 2: capture picker UI, updated for webrtc merge - non-functional
Comment 34 Randell Jesup [:jesup] 2012-04-27 08:35:15 PDT
These updated patches (probably the second one) are non-functional - no "Capture" button appears.  They compile clean, though! :-)  They are for running on top of the about-to-be-pushed update to alder from webrtc.org

Note You need to log in before you can comment on or make changes to this bug.