692955 - (camera) camera support for desktop <input accept=image/*> tag.

Status: NEW

(Firefox :: General, defect)

Description

[[:fabrice] Fabrice Desré]

This bug is about provide support for still image capture from a camera using the <input accept=image/*> tag.

We use the gips library from the webrtc project

Comment 1

[[:fabrice] Fabrice Desré]

Attachment: part 1: backend

We create an async input stream that receives frames from gips.

Comment 2

[[:fabrice] Fabrice Desré]

Attachment: part 2: front-end

Showing the capture picker in a tab-modal dialog.

Comment 3

[[:fabrice] Fabrice Desré]

Attachment: screenshot: capture picker

Comment 4

[[:fabrice] Fabrice Desré]

Attachment: screenshot: capture done!

Comment 5

[[:fabrice] Fabrice Desré]

These patches apply on top of the Alder project repository:
http://hg.mozilla.org/projects/alder

Comment 6

[Justin Dolske [:Dolske]]

Is there a project/feature page for this? Also, you'll want to engage with UX to get a design for the front-end UI this adds.

Comment 7

[[:fabrice] Fabrice Desré]

The feature page is here: https://wiki.mozilla.org/Platform/Features/Camera_API

Comment 9

[Ben Bucksch (:BenB)]

From dup bug:
Taking photos is extremely privacy sensitive and can, when used without the user noticing, represent a surveillance device inside my house. I personally don't feel comfortable seeing this feature *at all* in Firefox.

But, assuming this is accepted: Given the tricks we've seen with <input type="file"> (ask Jesse Ruderman), please be sure to have explicit consent from the user, in form of a dialog box that cannot possibly be accidentally clicked nor the user tricked into clicking. Inline/inpage approval is not OK.

<sarcasm>We could even do a "authorization=police" parameter, which skips the user confirmation.</sarcasm>

Comment 10

[Randell Jesup [:jesup] (needinfo me)]

Ben - see https://wiki.mozilla.org/Security/Discussions/WebRTC for the corollary to this bug - access to the camera/microphones for video and audio for WebRTC (and also for added security risk, screen-sharing).  Google has implemented much of this as Google Talk and Google+ Hangouts, and now Hangouts With Extras (which allows screen sharing).

Yes, this is *highly* security/privacy sensitive, and as part of this effort the IETF is looking at the security implications of the protocols and APIs (see ekr's draft from the above wiki page).

Comment 11

[brunoais]

A nice way to do this is to have a request made to the user asking if you allow the loaded site to take a picture of you almost like the firefox's popup blocker.
Differences:
When you allow to be taken pictures, you allow the main page, not the frames.
Only the main page is allowed to take pictures. Any frames in it are always denied.
The page may request to take a picture whenever it wants but the picture is only taken when the user presses the <Enter> or the button to take the picture. That button is in a modal box made by the browser or the OS itself (like the file picker). The page has no control over that modal window.
Extra: The modal window should be useful for the user.
Allowing is by page load. Reloading a page means that the approval of allowing to take the picture is lost.

Comment 12

[Ben Bucksch (:BenB)]

> A nice way to do this is to have a request made to the user asking if you
> allow the loaded site to take a picture of you almost like the firefox's popup blocker.
> The page has no control over that modal window.

+1

> when the user presses the <Enter>

That's subject to attacks (e.g. the "press enter game")

Comment 13

[[:fabrice] Fabrice Desré]

Attachment: part 1 : gips backend

updated gips backend

Comment 14

[[:fabrice] Fabrice Desré]

Attachment: part 2: capture picker UI

Fixes an osx only bug

Comment 15

[[:fabrice] Fabrice Desré]

Attachment: part 3: xcode 4.2 support

Comment 16

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

removing sec-review-needed and marking the feature page instead

Comment 17

[[:fabrice] Fabrice Desré]

Mockups from Jennifer:
http://people.mozilla.com/~jboriss/specs/first_spec.png

Comment 18

[Ben Bucksch (:BenB)]

Looks great! And good from privacy perspective.

I also like that I can take a file from disk instead.

Comment 19

[Robin Berjon]

(In reply to Fabrice Desré [:fabrice] from comment #17)
> Mockups from Jennifer:
> http://people.mozilla.com/~jboriss/specs/first_spec.png

These look really nice and very promising.

Suggestion: it is not uncommon for websites to want pictures with a specific aspect ratio. For instance, profile pictures may have to be square, or portrait 4x3. This is often supported either with direct truncation (where the server side just cuts the picture without asking the user) or with some in-page script that allows the user to select which area to use, within constraints. A lot of those script solutions are a bit heavy handed, and aren't always very accessible. Since I note that you are already considering a way of selecting an area in the picture, it could be useful to allow for a constrained aspect ratio mode. This could be specified by the page author using one of the markup hints that we're looking into.

Other non-UI hints could be useful, such as indicating which size boundaries the author wants so that the UA could provide an already-resized image. Such small additions could really help authors.

Comment 20

[Junior]

Please, How I can to take a snapshot from a webcam in a desktop computer with Firefox?, What is the code supported?

Comment 21

[Ian Melven :imelven]

(In reply to Junior from comment #20)
> Please, How I can to take a snapshot from a webcam in a desktop computer
> with Firefox?, What is the code supported?

when this lands in desktop Firefox, you will be able to do this via <input accept=image/*> on a webpage.

Comment 22

[Randell Jesup [:jesup] (needinfo me)]

Attachment: gips backend updated for bitrot

Comment 23

[Randell Jesup [:jesup] (needinfo me)]

Attachment: capture UI updated for bitrot

Comment 24

[Randell Jesup [:jesup] (needinfo me)]

Added URL for fabrice's test (perhaps you should put the code here)

Comment 25

[Randell Jesup [:jesup] (needinfo me)]

Attachment: gips backend updated for EXPORTS change

Comment 26

[Randell Jesup [:jesup] (needinfo me)]

Attachment: capture picker UI - updated for gkmedia m-c merge

Comment 27

[(not currently active) Ted Mielczarek]

I pushed a couple of changes to alder:
http://hg.mozilla.org/projects/alder/rev/84616cd44046
http://hg.mozilla.org/projects/alder/rev/b25db6a3947e

With those changes, I was able to build with these patches applied and get working video capture on Windows.

Comment 28

[Randell Jesup [:jesup] (needinfo me)]

Attachment: Part 1: GIPS backend, updated for m-c merge

Comment 29

[Randell Jesup [:jesup] (needinfo me)]

Attachment: Part 2: capture picker UI, updated for m-c merge

Comment 30

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

need to schedule a full team review

Comment 31

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

Please follow the instructions on this page to schedule a review.

Comment 32

[Randell Jesup [:jesup] (needinfo me)]

Attachment: Part 1: GIPS backend, updated for webrtc merge - non-functional

Comment 33

[Randell Jesup [:jesup] (needinfo me)]

Attachment: Part 2: capture picker UI, updated for webrtc merge - non-functional

Comment 34

[Randell Jesup [:jesup] (needinfo me)]

These updated patches (probably the second one) are non-functional - no "Capture" button appears.  They compile clean, though! :-)  They are for running on top of the about-to-be-pushed update to alder from webrtc.org

Comment 35

[Stephen Donner [:stephend] Not actively reading bugmail]

Mass-removing myself from cc; search for 12b9dfe4-ece3-40dc-8d23-60e179f64ac1 or any reasonable part thereof, to mass-delete these notifications (and sorry!)

Comment 36

[BugBot [:suhaib / :marco/ :calixte]]

The severity field for this bug is relatively low, S3. However, the bug has 18 votes and 50 CCs.
:fabrice, could you consider increasing the bug severity?

For more information, please visit auto_nag documentation.

Comment 37

[BugBot (nomail) [:suhaib / :marco/ :calixte]]

The last needinfo from me was triggered in error by recent activity on the bug. I'm clearing the needinfo since this is a very old bug and I don't know if it's still relevant.

Comment 38

[Nico Grunbaum [:ng, @chew:mozilla.org]]

fabrice, do you have any objections to removing the "camera" alias from this bug for the sake of Bugzilla search-ability of camera related bugs?

Comment 39

[[:fabrice] Fabrice Desré]

(In reply to Nico Grunbaum [:ng, @chew:mozilla.org] from comment #38)

fabrice, do you have any objections to removing the "camera" alias from this bug for the sake of Bugzilla search-ability of camera related bugs?

No, feel free to make the changes you need. I'm not even sure that <input accept=image/*> is still something we want so maybe we should just close this bug WONTFIX.

Comment 40

[brunoais]

I believe this is a very useful feature for firefox, special on smartphones.
It's quite common with native apps to include an option to just snap a photo when an image is wanted.
More and more as time goes it becomes more useful to have the option to just snap the photo directly into the web form.

Comment 41

[[:fabrice] Fabrice Desré]

(In reply to brunoais from comment #40)

I believe this is a very useful feature for firefox, special on smartphones.
It's quite common with native apps to include an option to just snap a photo when an image is wanted.
More and more as time goes it becomes more useful to have the option to just snap the photo directly into the web form.

I don't disagree, but I'm not the one making these decisions for Gecko/Firefox.