Closed Bug 674718 Opened 13 years ago Closed 7 years ago

FirefoxOS USB

Categories

(Core :: DOM: Device Interfaces, defect)

Product:
Core
Component:
DOM: Device Interfaces
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WONTFIX
Project Flags:
blocking-basecamp -

People

(Reporter: cjones, Unassigned)

References

Details

(Keywords: feature)

Attachments

(5 files, 2 obsolete files)

patch
50.55 KB, patch
Details | Diff | Splinter Review
Partial windows implementation
63.23 KB, patch
Details | Diff | Splinter Review
libusb.patch
885.70 KB, patch
Details | Diff | Splinter Review
0001-WebUSB-v1.patch
57.56 KB, patch
Details | Diff | Splinter Review
Software structure of 0001-WebUSB-v1.patch
24.68 KB, image/gif
Details 
Per discussion
 - offer an interface like window.addUSBHandler(matchClass, matchVendor, matchDevice) that allows content to "steal" a USB device when it's connected.  The match* arguments specify which devices the handler is interested in.  They might be "*" or undefined to mean "everything", otherwise specific device classes or IDs.
 - when the page first loads, or a new USB device is connected, throw a doorhanger-style popup to let the page bind the USB device.  (One option would be "Always allow".)

From here the details are a bit murkier ... when permission is granted by the user, content will receive a notification of some sort giving it the option of binding itself to the device.  The device would be represented by an abstraction like USBDevice, exposing (i) byte-buffer send/recv and (ii) control-message send/recv functionality.  (Ignoring reserved-bandwidth transfers for now.)  Both send and recv will be non-blocking.

It seems to make sense to make the WebUSB send/recv API match the WebSockets API as much as possible.
Let the bikeshedding begin!

(In reply to comment #0)
> Per discussion
>  - offer an interface like window.addUSBHandler(matchClass, matchVendor,
> matchDevice)

Would probably be better to offer addHandler(matchObj), where matchObj might be { class: "foo", vendor: "blah" }, etc.
Yeah, here is what my current implementation sketch does:

var usb = new USBManager(); // I don't want to stick more stuff on window
usb.onclaim = ...
usb.claimDevice({ class: subclass: protocol: idVendor: idProduct: })

onclaim gets called if a device is found and the user agrees

USB has a pretty well known standard set if idioms and APIs. I would stick to those for modeling the USBDevice class.
Another interesting implementation detail of this is to whether we want to use XPCOM or JSAPI. I started writing a proper XPCOM component for this, but I spent 4 hours just making the basic idl and C++ glue and uuidgen and fml. Then I got tired of that and switched to a simple JSAPI extension (similar to CTypes) and got something discovering devices and running claim handlers on them within 30 minutes. I think I am leaning to the latter now. The only drawback is that C++ code won't be able to use this as efficiently, but I think I am ok with that.
(In reply to comment #2)
> Yeah, here is what my current implementation sketch does:
> 
> var usb = new USBManager(); // I don't want to stick more stuff on window
> usb.onclaim = ...
> usb.claimDevice({ class: subclass: protocol: idVendor: idProduct: })
> 
> onclaim gets called if a device is found and the user agrees

So the onclaim handler returns true or false to indicate whether it wants to steal the device?  What happens after it returns true?  How does it get a handle to the USBDevice so that it can attach recv handlers etc.?  (Don't need to solve this ASAP, just want to make sure the problem is clear.)  It would be nice to pass a USBDevice handle to the onclaim handler, but the problem is there's no way to do a "const reference" in JS, so if the onclaim decides not to take the device, it's free to mutate the USBDevice however it wants before returning false, which sucks.

> USB has a pretty well known standard set if idioms and APIs. I would stick
> to those for modeling the USBDevice class.

Maybe ... but if they're idioms because of historical accident instead of an underlying fundamental difference from say sockets, we would need to carefully think whether we're better off targeting old-time USB devs used to libusb, or web authors used to other saner APIs.  I can see arguments for both.  Not something we need to resolve for a v0 prototype.
(In reply to comment #3)
> Another interesting implementation detail of this is to whether we want to
> use XPCOM or JSAPI. I started writing a proper XPCOM component for this, but
> I spent 4 hours just making the basic idl and C++ glue and uuidgen and fml.
> Then I got tired of that and switched to a simple JSAPI extension (similar
> to CTypes) and got something discovering devices and running claim handlers
> on them within 30 minutes. I think I am leaning to the latter now. The only
> drawback is that C++ code won't be able to use this as efficiently, but I
> think I am ok with that.

There's a lot of googunk with IDL stuff, but Gecko people speak that as a second language.  I think Gecko folks in general are less likely to speak JSAPI extension, so the experience for a Gecko person might be the opposite of yours.

What are the deeper tradeoffs?  I don't know JSAPI extensions well, so this is a question for you and others.  Which will be faster to maintain and evolve, all other things being equal (experience level with both approaches).  Which is less tied to SM internals.

Agree about not caring about C++ callers, for this API and all new ones from now on TBH.
If we don't care about C++, JSAPI is the way to go. The API is much cleaner. We don't have a way to generate JSAPI from idl for now, but we are working on that for the new DOM bindings. For small API surfaces this shouldn't make a difference.
What kind of C++ interface would be generated "under" the JSAPI?  If it's clean and sane and real C++ (like actual return values and so forth), I think everyone on the Gecko side would move to this system like yesterday.
OS: Linux → All
Hardware: x86_64 → All
Somewhat related: bug 604039. I'm putting together a prototype patch to expose joystick events to content, but it's much higher level than this.
Attached patch patch (obsolete) — — Splinter Review 
WIP. MacOSX only for now. Allow creation of a USBManager object that can be used to claim USB devices:

var usb = new USBManager();
usb.claimDevice(class, subClass, protocol, vendor, product)

If a matching device is found (and later on if security policies permit), fire usb.onattachdevice, handing in a device object.

The device object is not done yet. This isn't working yet. Just a code dump.
(In reply to comment #2)
> Yeah, here is what my current implementation sketch does:
> 
> var usb = new USBManager(); // I don't want to stick more stuff on window

You are sticking at least USBManager in window.

I can't believe it took 4 hours to do XPIDL/XPCOM stuff. That's either first-time startup cost or sandbagging.

We should not rush and then have to redo this. We have three big issues to get right:

1. Security model. Right now you're assuming a permission model as described in roc's blog? If so we could use some safe-by-default design, instead of hooking up powerful code and risking forgetting or messing up the call out to the permission manager.

2. API design, USB precedent vs. WebSockets vs. ? We are aiming at WebDevs first, so I lean toward the latter but it's just a starting bias, not decisive.

3. XPCOM vs. JSAPI. Gecko DOM hackers should weigh in. If we really want JSAPI glue from XPIDL (it can pile up to sizeable code footprint) let's do that now, instead of spending 30 minutes on handcoding. That will add up over time and risk bugs.

There may be another angle here: module system or no module system.

Let's take at least a weekend to get to some agreement.

/be
> You are sticking at least USBManager in window.

Or window.navigator, still trying to decide.

> 
> We should not rush and then have to redo this. We have three big issues to
> get right:
> 
> 1. Security model. Right now you're assuming a permission model as described
> in roc's blog? If so we could use some safe-by-default design, instead of
> hooking up powerful code and risking forgetting or messing up the call out
> to the permission manager.

Not sure what you mean. The permission model I have in mind is that you claim a very specific device and if that exists, a doorhanger lets the user approve the action.

> 2. API design, USB precedent vs. WebSockets vs. ? We are aiming at WebDevs
> first, so I lean toward the latter but it's just a starting bias, not
> decisive.

I think most USB drivers on the web will be written by USB devs, but I don't think the API would look much different either way.

> 3. XPCOM vs. JSAPI. Gecko DOM hackers should weigh in. If we really want
> JSAPI glue from XPIDL (it can pile up to sizeable code footprint) let's do
> that now, instead of spending 30 minutes on handcoding. That will add up
> over time and risk bugs.
> 
> There may be another angle here: module system or no module system.
> 
> Let's take at least a weekend to get to some agreement.

I went with XPCOM/XPConnect for now. A partical JSAPIIDL solution causes instanceof mismatches along the proto chain (EventTarget is needed in both worlds, one XPCOM thing and one via JSAPI). Not worth the hassle for now.

> 
> /be
(In reply to comment #11)
> > You are sticking at least USBManager in window.
> 
> Or window.navigator, still trying to decide.

FWIW, Jonas, Ben and I were thinking of using navigator for the WebAPI objects.
I have no strong opinion either way. I think there is some value of not further cluttering window.
Attached patch patch (obsolete) — — Splinter Review 
links, untested
Attachment #549550 - Attachment is obsolete: true
I'm going to do some research into what it'll take to do this stuff on Win32. I *think* it should all be possible in user-space but I'm not certain.
I believe there's a libusb port for windows too that might be of use here.
libusb is GPL, so we can't use that
(In reply to comment #17)
> libusb is GPL, so we can't use that

According to http://libusb.org/ libusb is LGPL.
Neither is compatible with the MPL.
libusb's synchronous interface is a bad match anyway, we don't want that
What we've done elsewhere is to run the synchronous API on a separate thread which then sends events to the main thread.

Of course, if we can get to an asynchronous API directly instead, that's better.
What use cases is this API meant to address? Why wouldn't those use cases be better addressed by a collection of bus-independent device class-specific APIs?

If a Web app is programmed to capture USB devices of class foo and I have a device of class foo but using Bluetooth (or a yet unknown future bus type), I'd be pretty annoyed for being excluded due to using a different bus than the one the app author expected.

Also, it seems to me that there's a risk of Web app developers choosing to capture the whole USB device in cases where a higher-level API exists and using the higher-level API would be friendlier to other browser tabs and to non-browser apps on the same operating system. (Consider USB pointing device capture compared to using the existing DOM mouse events.)
(In reply to comment #22)
> What use cases is this API meant to address? Why wouldn't those use cases be
> better addressed by a collection of bus-independent device class-specific
> APIs?

Those APIs are not mutually exclusive. A low-level USB API allows you to talk to anything USB, including devices with vendor-specific protocols (which are unfortunately very frequent). You _can_ use this interface to talk to a webcam, for example, but the preferred method will be to use the webcam interface which works with USB cameras and built-in ones. In some edges cases webapps might prefer USB after all because maybe its a specific camera that has additional functionality that is not exposed through the general interface (i.e. stereo camera or whatever).

> 
> If a Web app is programmed to capture USB devices of class foo and I have a
> device of class foo but using Bluetooth (or a yet unknown future bus type),
> I'd be pretty annoyed for being excluded due to using a different bus than
> the one the app author expected.

Bluetooth and USB device classes are not generally compatible. Trying to handle them uniformly would require us to spend a lot of effort create a uniform model on top of WebUSB and WebBluetooth. I would rather let web library authors innovate here. We offer the low-level API, and people can write javascript libraries with nice abstractions (i.e. PTP.js that can transfer pictures via usb and bluetooth).

> 
> Also, it seems to me that there's a risk of Web app developers choosing to
> capture the whole USB device in cases where a higher-level API exists and
> using the higher-level API would be friendlier to other browser tabs and to
> non-browser apps on the same operating system. (Consider USB pointing device
> capture compared to using the existing DOM mouse events.)

There is always a risk of people using inferior APIs for a task. You can use canvas to display text instead of HTML and CSS, and you use accessibility etc. I don't think we can stop that. There will be a specific and particularly loud warning if you try to talk to devices that are supported by the kernel (i.e. Mouse or block device). We will have to decide whether we ever allow to disconnect those from the kernel driver. For the primary mouse I would for example argue that we don't want to permit that. The same with mounted block device drivers. We can figure this stuff out along the way.
Attached patch patch — — Splinter Review 
Added some goop with bz's help to register the constructor for the USBManager. You can actually instantiate it now. Weeee.
Attachment #549555 - Attachment is obsolete: true
(In reply to comment #23)
> (In reply to comment #22)
> > What use cases is this API meant to address? Why wouldn't those use cases be
> > better addressed by a collection of bus-independent device class-specific
> > APIs?
> 
> Those APIs are not mutually exclusive. A low-level USB API allows you to
> talk to anything USB, including devices with vendor-specific protocols
> (which are unfortunately very frequent). You _can_ use this interface to
> talk to a webcam, for example, but the preferred method will be to use the
> webcam interface which works with USB cameras and built-in ones. In some
> edges cases webapps might prefer USB after all because maybe its a specific
> camera that has additional functionality that is not exposed through the
> general interface (i.e. stereo camera or whatever).

Isn't interfacing with a specific camera against the device-independent nature of the Web, though?

I guess having an USB API would allow Web apps to be innovative with a new hit device (like a Kinect level of hit device) before browsers and operating systems catch up with higher-level APIs for a new device class, but it seems bad if Web apps are able to create lock-in on the level of requiring a particular device from a particular vendor. Then anyone wishing to compete with that device needs to emulate it on the USB level.

> > If a Web app is programmed to capture USB devices of class foo and I have a
> > device of class foo but using Bluetooth (or a yet unknown future bus type),
> > I'd be pretty annoyed for being excluded due to using a different bus than
> > the one the app author expected.
> 
> Bluetooth and USB device classes are not generally compatible. Trying to
> handle them uniformly would require us to spend a lot of effort create a
> uniform model on top of WebUSB and WebBluetooth. I would rather let web
> library authors innovate here. We offer the low-level API, and people can
> write javascript libraries with nice abstractions (i.e. PTP.js that can
> transfer pictures via usb and bluetooth).

I wasn't suggesting that Bluetooth and USB were compatible. Rather, they don't need to be compatible if Web apps are working with higher-level APIs.

> > Also, it seems to me that there's a risk of Web app developers choosing to
> > capture the whole USB device in cases where a higher-level API exists and
> > using the higher-level API would be friendlier to other browser tabs and to
> > non-browser apps on the same operating system. (Consider USB pointing device
> > capture compared to using the existing DOM mouse events.)
> 
> There is always a risk of people using inferior APIs for a task. You can use
> canvas to display text instead of HTML and CSS, and you use accessibility
> etc. I don't think we can stop that.

There is a way and you mention it immediately: Preventing Web apps from using this API to capture a device that belongs to a class that already has (or can Real Soon Now have) higher-level APIs for it (pointing devices, keyboards, storage devices, webcams, microphones).

If you hand Web authors a footgun, the footgun will go off in wrong places. OTOH, banning apps from taking over mice or storage devices has a relatively small penalty on innovation (mainly preventing things like http://bellard.org/jslinux/ from offering generic USB bridging).

> There will be a specific and
> particularly loud warning if you try to talk to devices that are supported
> by the kernel (i.e. Mouse or block device). We will have to decide whether
> we ever allow to disconnect those from the kernel driver. For the primary
> mouse I would for example argue that we don't want to permit that. The same
> with mounted block device drivers. We can figure this stuff out along the
> way.
> Isn't interfacing with a specific camera against the device-independent
> nature of the Web, though?

There will always be a tension between innovation and standard abstractions. You mentioned kinect yourself. Its very very very hard (read: impossible) to design APIs that take into account future device innovation. We should enable the web to immediately take advantage of such devices, and quickly backfill with better high-level device abstractions. USB is painful to use. People won't rely on it as long there are better alternatives.

> I wasn't suggesting that Bluetooth and USB were compatible. Rather, they
> don't need to be compatible if Web apps are working with higher-level APIs.

Definitely. We want higher level APIs for common device classes. I just don't think its mutually exclusive.

> There is a way and you mention it immediately: Preventing Web apps from
> using this API to capture a device that belongs to a class that already has
> (or can Real Soon Now have) higher-level APIs for it (pointing devices,
> keyboards, storage devices, webcams, microphones).
> If you hand Web authors a footgun, the footgun will go off in wrong places.
> OTOH, banning apps from taking over mice or storage devices has a relatively
> small penalty on innovation (mainly preventing things like
> http://bellard.org/jslinux/ from offering generic USB bridging).

I think this is a valid view-point (I mostly agree with mice, a bit less with storage devices, but I am open to negotiations). We should discuss security policies in more detail once the basic driver functionality works.
Depends on: 680326
Finally managed to get most of the pieces in place and working for device detection. Right now, any time you call ClaimDevice, or any time a physical device is connected/disconnected from your machine, it'll run EnumConnectedDevices to rescan all your USB ports.

Left to do here:
Take out the debugging printfs
Figure out the correct way to add the link libraries it needs
Start firing the attach/detach DOM events correctly

Note that this patch won't work without bug 680326, at present.
Keywords: dev-doc-needed
Awwwww yeaaaaah! I've been idly poking at libusb for a while as a side project to do exactly this. Never spent enough time to bring it to fruition, though!

Relevant to the discussion at hand: class-specific devices work great when they exist, and are probably the preferred way to access some things. But there are a number of devices that don't really adhere to that, otherwise require custom drivers, and this make "use this device with the web" a rather painful experience. (Yes, that's largely due to OS-FAIL, but here we are.).

A few examples: a ambient "you've got mail" notification device, a toy rocket launcher (bug 683056), Arduino. I own all 3, and would like them to work with my browser regardless of OS.
How about a mini-iTunes-like add-on that downloads Fennec and syncs it to your Android phone? We want bundling/distro in mobile -- why not use our desktop Firefox channel?

/be
Blocks: 711613
Does the scope of this API allow a user agent to act as both a USB host and a USB client?

What I'm thinking is that a web app like a photo gallery may want to act as both depending on the user agent hardware. When you run the gallery app on a desktop PC, you might want to be able to transfer files from a USB client like a digital camera or media player using USB MSC, PTP or MTP. Conversely, when you run the app on a smartphone you might want the device to act as a USB client in order to transfer media files to/from a USB host.

One of the unresolved use cases for the Gallery app on B2G is how to transfer media files on and off the device using USB, Bluetooth etc. so an MTP.js implementation could be a route to achieving this.
I believe WebUSB is going to handle the USB host case and "USB file reading" is going to handle the USB client part. At least that seems to be Jonas' plan currently.
Is there any one try to make a patch for B2G gonk?
I think Jim Straus (jstraus on IRC) was looking at it a bit back in early December? That's the last I heard.
Not yet, there are many other high priority items for MWC so we are focusing on that for now. We will cycle back to this after february.
Also, when we come back to this work, we shouldn't need any gonk-specific code, in theory.
A couple of questions…

1) is there anyone working on this?

2) if (1==false) && is the priority low on this?

then:

I could work on this API for my final university thesis.
assinged to me for sec action to schedule a meeting
Whiteboard: [secr:curtisk]
Whiteboard: [secr:curtisk] → [sec-assigned:curtisk:749344]
No longer blocks: b2g-product-phone
Whiteboard: [sec-assigned:curtisk:749344] → [sec-assigned:curtisk:749344][b2g:blocking-]
blocking-basecamp: --- → -
Whiteboard: [sec-assigned:curtisk:749344][b2g:blocking-] → [sec-assigned:curtisk:749344]
It looks like this API is targeted at the smartphone being the USB Host, but in most cases, the Smartphone is a USB device connected to the PC as a Mass Storage Device or MTP and so on. Are there any proposed APIs for when the device is attached to a PC Host ? In most cases, you would like to allow users to choose the mode of operation. I also would like to know what kind of interfaces you guys would like to have from the linux kernel Gadget Framework (which I maintain) so we can make sure to have all that on the upcoming releases.

I have been thinking of a generic way to notify about cable/device status (disconnected, connected, suspending, suspended, resuming, enumerated, etc) so that USB charging can be implemented fairly easily, too.
#38 USB mounting is already sort of done for B2G, check out bug 737153. Most of the interaction for options will happen via the settings API. WebUSB itself is mostly concerned with host mode support.
#39 fair enough. Just thought that since this is targeting a smartphone (B2G), you ought to have a USB Peripheral-side API. Never mind then ;-)
Flags: sec-review?(curtisk)
Keywords: sec-review-needed
Whiteboard: [sec-assigned:curtisk:749344]
Is this API still under development? I've been experimenting with Chrome's chrome.usb API and it's really fun. I'd love to see this land in Firefox, especially if it was accesible from the drive-by web (instead of just apps, as it is in Chrome).

(Oculus Rift driver using chrome.usb: https://github.com/benvanik/vr.js/tree/master/experimental/usb-driver)
Unfortunately I don't believe this been touched in a long time, and will need some pretty serious overhauls since it's bitrotted a fair amount (especially now that we'll also need to worry about WebIDL, electrolysis, etc...). That said, the idea is still alive, it just needs some implementation resources.

From what I remember, our major blocker on the first iteration of this was the ability to detach and reattach devices on windows. I vaguely remember hearing that outside of that, it was ready to at least be reviewed, though I don't know how much "working" that was. With all the work the games team is doing lately and the attention it's getting, hopefully we can get some traction soon.
Yeah, my understanding from extensive research (and consulting lots of windows driver/kernel development experts) is that the windows USB stack is explicitly not designed for this kind of thing.

Driver signing and other mechanisms also interfere (it's possible to do user mode drivers on Windows, but it's limited in ways that make it difficult to use that stack directly for these purposes). In practice, this feature requires writing a kernel-mode filter that intercepts potentially all USB traffic, or building something elaborate that installs a custom driver signing cert and then generates signed 'drivers' for each device on demand so that JS content can interact with it. (This is what the libusb stuff available on windows does.) I'm also not certain it is possible for this to work unless you are running as Administrator, which limits its usefulness (probably for good reason, though).

Anyone who looks into this can feel free to ping me for advice or information (though I'm only an 'expert' to the extent that I was stuck doing research on this for a couple months...)
This would also be useful to enable using a FxOS device as a controller for an arduino. 

In Node, this:
https://github.com/voodootikigod/node-serialport
is used by:
https://github.com/rwldrn/johnny-five/

and then JS programs can drive an arduino. 

At a nodebots hack day, there was a proof of concept that used Chrome's Serial API to do something similar, so it just needed the chrome browser to work:
http://developer.chrome.com/apps/serial.html

If FxOS could do this, the FxOS devices are small enough to be attached to the arduino to allow more portable arduino setups. Right now, a laptop connected via USB is needed.

I tried a shot at updating the patch, but the bitrot and unfamiliarity with the changes in the codebase since initial patch meant I was not able to complete it on the arduino hack day. I was using the mozilla-b2g18 branch of gecko.

Where I got stuck was updating the nsUSBManager code: nsDOMEventTargetWrapperCache no longer exists. I was looking at the WebSocket.h/cpp as maybe a guide, but I am mostly just using pattern matching to do the patch update, and I think I have reached the limit of that without really understanding how the code should be worked.

So if there are some easy hints to give to a beginner that is new to the code, feel free to let me know. I expect there are not easy hints, and in that case, I am happy to provide feedback on any updated patch by trying it out on an FxOS device to see if it can be used to drive an arduino, as long at the patch could work with mozilla-b2g18.

While I can see windows support as being difficult, perhaps as a start it could just be for FxOS devices.
Yeah, not sure an FxOS port would be classified as easy. We don't have a libusbhost backend yet (unless that happened on android while I wasn't looking), so on top of unbitrotting, this requires integration with the base system all the way up through the DOM.
Are there any objections to using libusb since our license policy now allows for LGPL code to be dynamically linked?
Thats a question for the license wrangler. Gerv, how does LGPL work for us?
LGPL works for us as long as the code is in a discrete dynamically-linked library. If you want to do something other than that, at the very least it would require more analysis.

Gerv
Is this project still ongoing? I am interested in joining the development of WebUSB.
(In reply to Sam Lin from comment #50)
> Is this project still ongoing?

http://whatwg.github.io/serial/ seems relevant.
Is there any one who also like to contribute on this bug?  Sam had told to me that he would like to contribute his spare time on this bug.  Since this bug is not on the list of our milestone and not schedule stressing, I suggest to let Sam start implement this for b2g and Linux desktop along the road started by Andreas if no one have concern.
I'm willing to spend time on this. I work on the gaia side of things but been following this bug for a while. Who is the proper person to mentor this bug?
I can mentor this one. That said, there are a LOT of decisions to be made here before work begins, because I don't really get the general direction of this API at the moment.

We haven't had any decisions on the types of USB messages we're interested in handling, whether this should just ship bulk/interrupt messages, resembling a reduced driver like WinUSB, or whether we want class compliancy, what platforms we'll be supporting first, etc. We need some good use cases first before we can start on this. The security on it is going to be a very special nightmare too.

While many people use serial USB adapters these days, I don't think WebSerial and WebUSB overlap.
I have interest on this bug for a specific use case. I want to be able to talk to this kind of boards from JavaScript:

http://www.adafruit.com/products/563#Description

My goal is to port to Firefox OS a hardware accessory I developed for Android.

Kyle where do you suggest me to start looking to make this happen?
Flags: needinfo?(kyle)
Diego, check out http://github.com/whatwg/serial. It's not a panacea, but the project idea was borne out of "I wanna javascript my arduino"

The question of what arduino is going to look like is interesting since I believe they're using the HID raw report hack on their newer boards to bypass things like ftdi driver installation. That's not really serial anymore, heh.

That said, supporting the arduino would be supporting descriptor parsing and certain interrupt packets to deal with HID raw report reading, which is not "supporting USB", it's "supporting a tiny subset of USB". That's why I'm wanting better definitions of exactly what WebUSB is supposed to be before people run off to start coding.
Flags: needinfo?(kyle)
For those interested in WebSerial (which as :qdot points out may not be the same work as WebUSB), related discussion for more background:

https://groups.google.com/forum/#!topic/mozilla.dev.webapi/wykkibp6BKo/discussion
A big problem you have to address is how your goals for an API like WebUSB interact with platform constraints. If you're not interested in supporting Windows users (this seems unlikely), you have a lot more options because the OS X USB stack (Based on what Andreas Gal told me, at least) is extremely flexible and you can easily use libusb on Linux-based devices. The Windows USB model is extremely restrictive (especially if you are trying to work from user-mode) and each new Windows release makes driver signing a bigger headache. The kernel model for USB drivers and device driver installation, etc. is not designed to make WebUSB or something like it easy to implement. In practice you may end up having to write a USB filter driver that filters all USB traffic in order to be able to interact with a given device. IIRC this is already done by VMs for Windows (virtualbox, vmware, etc.), but it is not particularly well documented and you will not find any help on NT kernel development lists or MSDN.

If you're willing to constrain the API in particular ways you might be able to make it a relatively simple challenge. For example, if the model is closer to 'install this website as a driver for this physical device', I believe it would be fairly simple to implement that on all major platforms.
(In reply to Diego Marcos from comment #53)
> I'm willing to spend time on this. I work on the gaia side of things but
> been following this bug for a while. Who is the proper person to mentor this
> bug?

Would you also work on gecko side, or just gaia side apps?
Just like Kyle had mentioned, this bug is not for specific application of USB.  If you expect to work on serial connection between devices, you should go to WebSerial bug.

Sam would start this bug and I will mentor since we are at the same city.  He will start by exposing attach/detach and devices info messages.  Then, expose more information and APIs.  The target is to expose an API for implementing gadgets and device drivers.  This bug should be a meta bug.  I will open more bugs to bite them piece by piece.  So, people is easy to contribute their spare time on this subject.
(In reply to Thinker Li [:sinker] from comment #59)
> (In reply to Diego Marcos from comment #53)
> > I'm willing to spend time on this. I work on the gaia side of things but
> > been following this bug for a while. Who is the proper person to mentor this
> > bug?
> 
> Would you also work on gecko side, or just gaia side apps?
> Just like Kyle had mentioned, this bug is not for specific application of
> USB.  If you expect to work on serial connection between devices, you should
> go to WebSerial bug.
> 
> Sam would start this bug and I will mentor since we are at the same city. 
> He will start by exposing attach/detach and devices info messages.  Then,
> expose more information and APIs.  The target is to expose an API for
> implementing gadgets and device drivers.  This bug should be a meta buIg.  I
> will open more bugs to bite them piece by piece.  So, people is easy to
> contribute their spare time on this subject.

I can work wherever is needed. Gecko/Gaia/Gonk. My goal is having a gaia app talking to my hardware accesssory. I just want to better understand what are the misssing pieces and find a starting point. Which one is the WebSerial bug?
Attached patch libusb.patch — — Splinter Review 
I'm not actively working on this, but I hacked up a quick proof of concept using libusb that could be useful.  Currently all it does is print the list of connected devices to the terminal when you call USB.getDevices().  The libusb code comes from chromium's libusb and I ported over the bare minimum of their gyp file to our moz build system to get it to build on OSX. If you're on another platform you'll need to port more.

One thing I didn't realize at the time of creating this, is that libusb does not provide a way to notify you when a device is connected. We'd either have to poll or (ideally)hook into the various platform specific notifications.
I will start working on this bug. Since Thinker will be a mentor and he will divide this bug to some pieces, I will do this step by step.
Depends on: 948803
Ok, you're using FxOS terms, not Firefox Desktop terms, so it sounds like you want to do a FxOS USB implementation. That's a whole different deal, especially since it needs to deal with gadget/OTG stuff.

Can you make a FxOS USB API metabug that blocks this then? There's a ton of history about cross platform USB here already, I'd rather not see this bug hijacked for a single platform that didn't even really exist when the bug was started.
(In reply to Kyle Machulis [:kmachulis] [:qdot] from comment #63)
> Ok, you're using FxOS terms, not Firefox Desktop terms, so it sounds like
> you want to do a FxOS USB implementation. That's a whole different deal,
> especially since it needs to deal with gadget/OTG stuff.
> 
> Can you make a FxOS USB API metabug that blocks this then? There's a ton of
> history about cross platform USB here already, I'd rather not see this bug
> hijacked for a single platform that didn't even really exist when the bug
> was started.

I created a new bug to track USB API work specific to Firefox OS

https://bugzilla.mozilla.org/show_bug.cgi?id=949215

Who are the experts in the room that could give me some directions to get started?
Depends on: 949215
(In reply to K. Gadd (:kael) from comment #58)
> The Windows
> USB model is extremely restrictive (especially if you are trying to work
> from user-mode) and each new Windows release makes driver signing a bigger
> headache.

VirtualBox has generic USB bridging from a Windows 8.1 host to a Linux guest. How does that work? Can we use whatever mechanism VirtualBox uses on Windows (not the same GPL code but the same approach to Windows APIs)?
I believe VirtualBox uses a filter driver like I described, but the last time I looked I was unable to find the source for it in their repository. As long as you manage to sign the filter and make sure it's stable, it would be an adequate solution for most WebUSB use cases, I think.
I think USB support is only part of their "extension" which bundles proprietary stuff. They are not willing or cannot release the USB driver source code.
Any news about this?
Yes, I am acturally working on this and hopefully could submit a quick patch in a couple of weeks.
WOW i cheering for you!
I need this API for a project about Open Web Apps and Arduino!
(In reply to Sam Lin from comment #69)
> Yes, I am acturally working on this and hopefully could submit a quick patch
> in a couple of weeks.

Are you working on this on github or moz hg? Would love to see what you've got going so far.
(In reply to Sam Lin from comment #69)
> Yes, I am acturally working on this and hopefully could submit a quick patch
> in a couple of weeks.

BTW, you may want to post an intent to implement to the dev-platform and dev-webapi mailing lists, as many people that've worked on this before may be interested in commenting or helping. See the template email at:

https://wiki.mozilla.org/WebAPI/ExposureGuidelines
One obstacle to this feature that's important to keep in mind: Windows is the most difficult platform to implement this on, due to the signing & device support architecture it uses. Signed 'device for anything' drivers are very difficult (if not impossible) to implement, which means that the best solution might actually be a USB filter driver. USB filter drivers are *very* tricky to implement correctly and when I worked on this I was not able to find any open source examples of a functioning (let alone reliable) USB filter driver for Win32 that would work on modern (Win7, Win8, etc) Windows kernels.

Implementing this feature on Linux and MacOS may be enough to solve a subset of user challenges but it's worth making sure your design and approach will be possible to implement on Windows.

If memory serves, the best candidate for a Windows solution was to install a custom signing key for drivers that Firefox would use to sign a custom driver for each device that WebUSB was going to manipulate. IIRC the windows variant of libusb (used by various utilities like custom Android device flashing tools) does something like this, where it custom-signs a version of the libusb driver that targets a specific device - and if memory serves, we couldn't use windows libusb due to a license conflict.

Glad to answer questions and discuss design issues with anyone working on this.
Is there still an intention to make this accessible via driveby?
Yes, there is. I was expecting to submit a patch to here a month ago. However, due to some unexpected difficulties, the schedule is postponed. I will submit a patch to here as soon as I can.
(In reply to Sam Lin[:neopleo] from comment #75)
> Yes, there is. I was expecting to submit a patch to here a month ago.
> However, due to some unexpected difficulties, the schedule is postponed. I
> will submit a patch to here as soon as I can.

Enabling USB access via driveby seems quite dangerous. It's not really practical
to explain to the user what the security impact of directly accessing random USB
devices is. This feature needs real security analysis before it lands.

Doug, can you help arrange?
Flags: needinfo?(dougt)
Sorry that I misunderstood the meaning of "driveby".

The way that I implement WebUSB is folliwng the link 

https://wiki.mozilla.org/WebAPI/WebUSB which is probably not the way you mention in comment#74.
What I mean by "driveby" is that the API is accessible from the Web rather
than from installed apps (albeit ones written in JS). I don't see anything
on the page you link to about what kinds of JS would be able to access this
API.
there exists some NPAPI plugins who's sole purpose is to interact with USB devices.  I think that this API is probably a good thing as it allows us to kill of plugins (which is always a good thing).

HOWEVER, exposing this to the web assumes that you can figure out the how to build a safe system that keeps the user in control and safe.  Big assumption.  I also think that figuring that part out is a lot harder then the plumbing.

Moreover, the API you have listed in the bug and on our wiki is not enough.  Please take a look at what Google does for the ChromeBook:  https://developer.chrome.com/apps/usb.
Flags: needinfo?(dougt)
One consistently terrifying thing here is that even if you have a
permissions opt-in for 'USB', it's hard to present the breadth and
risk level of that USB opt-in to the user, because all you have is
vendor/product IDs (as shown in the manifest part of Google's
documentation) - one ID looks the same as another to the user. If you
require confirmation you can at least try to display the device
information there, but it's possible you won't have detailed enough
info - many devices on a given USB bus will have innocuous/confusing
names, like how my webcam actually shows up as a generic USB hub, with
the important USB devices under it.

An attacker could easily request access to my webcam's hub (which
would just show up as 'generic usb hub'), and by claiming it, gain
access to my camera and microphone. I wouldn't notice unless I were
already using them at the time. Once the attacker had access they
could use some of the known-in-the-wild attacks that allow modifying
or replacing a usb device's firmware in order to have control over my
machine long-term. Spooky.

Interestingly Google's API doesn't appear to provide notifications to
tell you that new devices have been connected, just an API to query
existing devices. That's something the API proposal on the mozilla
wiki gets mostly right (though I'd argue that their 'enumerate then
claim' API is still superior - you do absolutely want notifications
instead of polling or looping, though.)
(In reply to K. Gadd (:kael) from comment #81)
> One consistently terrifying thing here is that even if you have a
> permissions opt-in for 'USB', it's hard to present the breadth and
> risk level of that USB opt-in to the user, because all you have is
> vendor/product IDs (as shown in the manifest part of Google's
> documentation) - one ID looks the same as another to the user. If you
> require confirmation you can at least try to display the device
> information there, but it's possible you won't have detailed enough
> info - many devices on a given USB bus will have innocuous/confusing
> names, like how my webcam actually shows up as a generic USB hub, with
> the important USB devices under it.
> 
> An attacker could easily request access to my webcam's hub (which
> would just show up as 'generic usb hub'), and by claiming it, gain
> access to my camera and microphone. I wouldn't notice unless I were
> already using them at the time. Once the attacker had access they
> could use some of the known-in-the-wild attacks that allow modifying
> or replacing a usb device's firmware in order to have control over my
> machine long-term. Spooky.

I think this last is the important point. You might *intend* to grant
access to your WebCAM and *actually* grant access to your WebCam but
then find that the attacker has reprogrammed it and now has log
term access. Until we resolve this issue, I think allowing driveby
access is a nonstarter. Rather, this should be treated like installing
a new app.



> Interestingly Google's API doesn't appear to provide notifications to
> tell you that new devices have been connected, just an API to query
> existing devices. That's something the API proposal on the mozilla
> wiki gets mostly right (though I'd argue that their 'enumerate then
> claim' API is still superior - you do absolutely want notifications
> instead of polling or looping, though.)
any news for this?
I've done the git pull in my local source and got some unexpected errors.

I will fix these and upload a patch.
The latest Nightly can be built but can not be launched normally.
Hence, I can merge my code to the latest build and create a patch as soon as it get fixed.
I've followed the spec defined in the following link to create this API.

https://wiki.mozilla.org/WebAPI/WebUSB

The implementation is still buggy so apply this patch at your own risk. : )

=======================================================
The working environment:

OS: Ubuntu 12.10 64 bit
The commit number I did the git pull last time: 9764e19e067e8e5 (10/5)
=======================================================

=======================================================
When you launch the Nightly,


Current status:

The system can listen to the usb product ID. 
When you attach the usb device to the host, you can find out the console shows the ProductID: xxxx

1. You can create an object. 
   var usbm = new MozUSBManager();

2. Once you know your usb product id, you can do:
   usbm.claimDevice("xxxx"); which xxxx is your product id.
   If it is matched, you will see some messages shown on your console.

3. I've added part of addEventListener callback to the source but not yet fully implment that function.
   So DO NOT try the callback for now.
========================================================

TODOs:

1. Multi processes can listen to different usb devices.

2. Callback function implementation. (attach, detach)

3. Adding the descriptor as an argument in claimDevice function.
I've followed the spec defined in the following link to create this API.

https://wiki.mozilla.org/WebAPI/WebUSB

The implementation is still buggy so apply this patch at your own risk. : )

=======================================================
The working environment:

OS: Ubuntu 12.10 64 bit
The commit number I did the git pull last time: 9764e19e067e8e5 (10/5)
=======================================================

=======================================================
When you launch the Nightly,


Current status:

The system can listen to the usb product ID. 
When you attach the usb device to the host, you can find out the console shows the ProductID: xxxx

1. You can create an object. 
   var usbm = new MozUSBManager();

2. Once you know your usb product id, you can do:
   usbm.claimDevice("xxxx"); which xxxx is your product id.
   If it is matched, you will see some messages shown on your console.

3. I've added part of addEventListener callback to the source but not yet fully implment that function.
   So DO NOT try the callback for now.
========================================================

TODOs:

1. Multi processes can listen to different usb devices.

2. Callback function implementation. (attach, detach)

3. Adding the descriptor as an argument in claimDevice function.
Flags: sec-review?(curtisk) → sec-review?
Blocks: 1029961
No longer blocks: 1029961
any news for this API?
FYI, I have been working on a Unofficial Draft Specification for something like this:

https://reillyeon.github.io/webusb/

It is currently being implemented (behind a flag) in Chromium. Comments on the design are welcome and can be submitted through the GitHub issue tracker. Of note is the "Device Requirements" section which describes a protocol for devices to describe something akin to Content Security Policy which I hope will mitigate the concerns over drive-by attacks mentioned above.

The Windows driver loading issue can be resolved on Windows 8.1 and above by using Microsoft OS 2.0 Descriptors: http://go.microsoft.com/fwlink/p/?linkid=306681
So looking at: https://reillyeon.github.io/webusb/ in section 3 it says:

"To be supported by a page using this API a USB device MUST provide information to the UA about the origins authorized to connect to it and MAY also provide a landing page that the UA MAY direct the user to navigate to in order to interact with the device."

So can this information also come from someplace other than the device itself? Otherwise it seems kind of difficult to connect to anything that wasn't specifically designed to be accessed by the the "web".

I guess I'd like to be able to specify somehow that any device on my local network is allowed to access the USB device attached to this computer without the device having to be aware of the fact.
That is intentional. Devices shipping today are not secure enough to be accessed by the web. As mentioned in comment 81 user prompting for arbitrary devices is just not sufficient to mitigate the security risk.

For testing purposes the UA could provide a settings UI that overrides the WebUSB Descriptors (or lack there of) returned by a device. I'm still concerned malicious sites could social engineer their way around this.

For manufacturers who want to opt in the idea of a "public device registry" is that the owner of the vendor ID (which are assigned by the USB-IF) could provide mapping from vendor ID/product ID pairs to allowed origins. How such a registry would be managed is to be determined.
Assignee: gal → nobody
Component: General → DOM: Device Interfaces
Keywords: feature
(In reply to Reilly Grant from comment #91)
> For testing purposes the UA could provide a settings UI that overrides the
> WebUSB Descriptors (or lack there of) returned by a device. I'm still
> concerned malicious sites could social engineer their way around this.

Social engineering is a perfectly valid concern, but making this process overly complicated and/or obscure could seriously hurt adoption of the technology (as early devices definitely won't be shipping with built in WebUSB support, and all the devices out there obviously won't have any, either).

I can imagine a kind of add-on plug for the connector (as in, physical hardware, an inexpensive add-on between the device and the host machine) that could be used to serve the appropriate headers, and results in an express consent from the user side (basically a physical http://crossorigin.me/-equivalent). This would, on the other hand similarly hurt early adoption, and since this device would have to be either created (which requires skills) or ordered by the user, could also pose other security issues (MITM/spyware/rootkit code hiding in the firmware of the plug).

TL;DR: I think early platform adopters should be provided a sufficiently unconvoluted way to use their existing devices on the web for the technology to be able to gain traction and other security-related practices (like the public device registry mentioned above) to be viable & needed.
Renamed this bug, as it's confusing people who are looking for the W3C version of this API. Also, can we close this as "won't fix" or is the community going to keep working on this?
Summary: WebUSB → FirefoxOS USB
The implementation of the W3C version have a ticket?
As i can see for the Firefox OS api there is another ticket https://bugzilla.mozilla.org/show_bug.cgi?id=949215
(In reply to Daniele "Mte90" Scasciafratte from comment #94)
> The implementation of the W3C version have a ticket?

No, as we don't (yet) have any plans to implement it. We need to evaluate the privacy/security aspects, but given that we are not doing bluetooth because of those concerns... it seems pretty unlikely that we will do USB.  

> As i can see for the Firefox OS api there is another ticket
> https://bugzilla.mozilla.org/show_bug.cgi?id=949215

Closed as dup of this one.
(In reply to Marcos Caceres [:marcosc] from comment #96)
> (In reply to Daniele "Mte90" Scasciafratte from comment #94)
> > The implementation of the W3C version have a ticket?
> 
> No, as we don't (yet) have any plans to implement it. We need to evaluate
> the privacy/security aspects, but given that we are not doing bluetooth
> because of those concerns... it seems pretty unlikely that we will do USB.  

Marcos, is there a bug or thread about the bluetooth decision? And where will any conversation/decisions about USB will happen? Thanks!
Flags: needinfo?(mcaceres)
(In reply to Dietrich Ayala (:dietrich) from comment #97)
> (In reply to Marcos Caceres [:marcosc] from comment #96)
> > (In reply to Daniele "Mte90" Scasciafratte from comment #94)
> > > The implementation of the W3C version have a ticket?
> > 
> > No, as we don't (yet) have any plans to implement it. We need to evaluate
> > the privacy/security aspects, but given that we are not doing bluetooth
> > because of those concerns... it seems pretty unlikely that we will do USB.  
> 
> Marcos, is there a bug or thread about the bluetooth decision?

No. It's verbal per Johnny Stenback.


>  And where
> will any conversation/decisions about USB will happen? Thanks!
> No. It's verbal per Johnny Stenback.


What ekr said :)
Flags: needinfo?(mcaceres)
Tried to apply attachment 4 [details] [diff] [review] on Gecko 37 but failed. I have 2 questions:
1.Could anyone tell me which branch the patches work on? I can't find the parent commit of attachment 4 [details] [diff] [review] in Gecko 37.
2.attachment 4 [details] [diff] [review] seems updated to include previous patches excpet attachment 3 [details] [diff] [review] and I can ignore attachment 1 [details] [diff] [review] and attachment 2 [details] [diff] [review]. Am I right?
Appreciate for any suggestion.
(In reply to Jeff Chuang from comment #100)
> Tried to apply attachment 4 [details] [diff] [review] on Gecko 37 but
> failed. I have 2 questions:
> 1.Could anyone tell me which branch the patches work on? I can't find the
> parent commit of attachment 4 [details] [diff] [review] in Gecko 37.
It was my mistake and the parent commit could be found on master branch.
Hi Sam Lin,
I have tried the patch, 0001-WebUSB-v1.patch, but I can't found the file "libudev.so.0" which is listed in the patch.
Do you have any comment？？
Flags: needinfo?(ckjboy2003)
The size of libudev.so.0 created by the patch is zero. I found an android repository of libudev and may help on this: https://github.com/chombourger/android-udev
I draw it to represent the relationship of modules in Sam's implementations and I am not sure whether I misunderstood it or not. I can't figure out the following points yet:
1.How the function UsbThreadImpl::Observe() is invoked?
2.What role does nsLayoutModule play in this implementation?
3.Where to find the information/reference codes about how to add the implementation of the function addListener()?
Appreciate for any suggestions!
Closing as WONTFIX since FxOS is no more.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
Chrome WebUSB support will be released in next version (61). Is there a plan to add this into Firefox too? This is really important for Smartcard, scanner etc usb devices:

https://bugs.chromium.org/p/chromium/issues/detail?id=492204&desc=2
No such plan exists. We do plan to implement https://w3c.github.io/webauthn/ which is tracked elsewhere.
Alias: webusb
It'll be really great if Firefox can support web usb too. Our application (HTML5 RDP) need this for Smartcard and other USB devices redirection.

This is also probably the last thing we need to replace the OS with browser.
+1

guys, let's get this rolled out. it'll be a huge help for devs struggling with USB devices and workarounds.
Please reconsider adding WebUSB to Firefox. Not adding it will hurt Firefox adoption rate similarly as hesitating to add U2F support in the past.
I totally agree with Pavol.

Today people workaround this by deploying local modules.
Those modules will never meet the highest security standards and are a huge burden on developers shoulders.

Moreover, I don't see what is the issue (except a resource issue maybe) since chrome adopted the spec.
We are more and more tempted to encourage our user base to use Chrome instead of Firefox because of this.
+1 for this.
WebUSB in Chrome is very convenient, if you have an USB device and you don't want to pack desktop apps for multiple platforms.
+1 for this.

Please reconsider adding WebUSB to Firefox.

WebUSB is now available as default in Chrome and is about to be an essential feature as many traditional platform dependent aplications are urging to enter the realm of Web apps.

Today developers strugle with terrible plugin based hardware interfaces and Chrome's WebUSB is working like a charm. The need of costum plugins to intarface with USB in Firefox is a NO-GO for most applications.
Please add this feature to make WebUSB a reality, if they do the other implementers will join. In Chorme it is already enabled by default.
Per the guidelines in https://bugzilla.mozilla.org/page.cgi?id=restrict_comments_guidelines.html we are restricting comments for this bug to people in the EDITBUGS group.

Multiple "Me Too!" comments in bugs only add noise and will not affect triage decisions regarding the bug. Please see https://bugzilla.mozilla.org/page.cgi?id=etiquette.html.

With respect to WebUSB, see the discussion at https://github.com/mozilla/standards-positions/issues/58

Thank you, 

Emma Humphries, EPM Firefox/Bugmaster
Flags: needinfo?(ckjboy2003)
Restrict Comments: true
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: