Last Comment Bug 836897 - (webmidi) Implement the WebMIDI API
: Implement the WebMIDI API
Status: NEW
: dev-doc-needed
Product: Core
Classification: Components
Component: DOM: Device Interfaces (show other bugs)
: Trunk
: All All
-- enhancement with 43 votes (vote)
: ---
Assigned To: Kyle Machulis [:qdot]
: Andrew Overholt [:overholt]
Depends on: 1201590 1201593 1201596 1201598 1201603 1123516
  Show dependency treegraph
Reported: 2013-01-31 13:29 PST by Chris Wilson
Modified: 2017-02-25 03:58 PST (History)
66 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Description User image Chris Wilson 2013-01-31 13:29:48 PST
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.28 (KHTML, like Gecko) Chrome/26.0.1396.1 Safari/537.28

Expected results:

Firefox should implement the Web MIDI API (

This W3C specification-in-progress enables the web platform to gain access to MIDI input controllers, such as keyboard, drum and DJ controllers - it also enables access to external and internal synthesizers, such as the built-in software synthesizers on Windows and Mac OS.

MIDI connections are extremely common in musical applications, and have become increasingly popular on mobile and tablet devices as well (MIDI is supported by iOS via CoreMIDI, and the iOS App Store has >400 MIDI applications.  This has become so popolar that many MIDI keyboard controllers are available with iPad slots.)

Support for Web MIDI, along with Web Audio, enables many music production scenarios, from notation entry and playback a la to live synthesizers (e.g., as well as numerous iOS synthesizers from names like Roland, Yamaha and Korg), as well as providing programming interfaces to hardware synthesizers (E.g Roland's JP Synth Editor for iPad, used with their Jupiter-50 and Jupiter-80 synthesizers).
Comment 1 User image Robert O'Callahan (:roc) (email my personal email if necessary) 2013-01-31 19:51:10 PST
This is much lower priority than Web Audio (and many other things) IMHO.
Comment 2 User image Ada [:adalucinet] 2013-02-08 00:09:05 PST
Chris, thanks for your suggestion. Marking this as a possible future enhancement.
Comment 3 User image Jussi Kalliokoski 2013-02-26 21:42:15 PST
To me this sounds like a perfect target for maybe an intern to take on?

Either way, I'm obviously in favor of this. I can also see the benefit of this API in things such as low-end mobile games (when you pay for transfer, MIDI starts sounding surprisingly good).
Comment 4 User image :Ehsan Akhgari 2013-02-27 08:55:04 PST
I'd personally prefer the intern to work on Web Audio first.  It is of much higher priority.
Comment 5 User image Idan Beck 2013-02-27 15:48:50 PST
I just wanted to vocalize official support regarding how important this feature is.  The ability to add MIDI capability to web applications would open up massive new avenues for online music creation, collaboration, entertainment and education.  As great as web audio is, the inability to flexibly manipulate it is a big drawback. 

We've already started to build support for our hardware (the gTar) in web midi through the use of the poly-fill / Jazz Soft midi plug in.  The ability to build/deploy music related applications along with the coupling of Web MIDI is extremely exciting. 

If needed we would be willing to provide support however possible and within our means!

Comment 6 User image :Ehsan Akhgari 2013-02-27 17:55:58 PST
(In reply to comment #5)
> If needed we would be willing to provide support however possible and within
> our means!

Are you interested in submitting patches to implement Web MIDI in Gecko?
Comment 7 User image oskar.det 2013-03-01 10:27:26 PST
I'd definitely agree that this API should be implemented in a not too distant future. While it's great that Web Audio is coming along, having this API in parallell would make a world of difference in terms of professional audio on the web. 

Having gone the long route implementing a musical "keyboard" on the computer keyboard, I can honestly say I wish this was around when we made Jam With Chrome, and I'd love to use it in a lot of coming projects too.
Comment 8 User image joe 2013-03-01 11:27:07 PST
This API is really important, as MIDI is the universal standard for real-time I/O controllers of many kinds (not just music performance; also lighting, mixing, and much more). Supporting MIDI is also the obvious complement to Web Audio API support, allowing the browser to support a full spectrum of applications with real-time musical input from the user.
Comment 9 User image pete 2013-03-03 08:46:43 PST
As the software lead at a MIDI controller manufacturer, we have already charted out new ways of reaching customers and customers reaching each other that uses MIDI in the browser. The potential is overwhelming! There is also a great cultural benefit in preserving and distributing non-fixed media artworks, such as algorithmic composition, many (if not most) of which are composed with MIDI. 
There is no reason that the only device that can control a browser is a keyboard, and there are millions of devices out there that speak MIDI. Beyond music applications, the industrial applications for show control would explode. 
Every major OS has seen fit to include MIDI natively - as the browser becomes the default "operating system" for screens around the world, this established and important hardware interface needs to be part of this growth.
Comment 10 User image Boris Zbarsky [:bz] (still a bit busy) 2013-03-03 11:40:38 PST
We all agree this should be implemented.  The question is who is going to do it and whether other things that are _also_ important should be dropped to implement this.

Again, if people are willing to step up and help implement that would probably help a lot in terms of finding people to work on this.
Comment 11 User image Scott Mire 2013-03-07 10:10:53 PST
I wanted to cast my vote of support for getting the MIDI API implemented. Although many seem to not understand the importance of MIDI and it's status as a long-standing standard for real-time I/O control, I personally cannot imagine us achieving the dream of the "browser being the platform" without both low-level audio AND MIDI support. If every OS on the market found the need to add MIDI functionality to their product, so should browser developers. From lighting and show control, to music education, to gaming, to controlling industrial machinery, MIDI has seemingly limitless applications. If you ponder MIDI’s potential in the home automation market alone, it is mind-boggling. 

Having said that, I will now tell you that I work for one of the biggest audio manufacturers in the world. Obviously, MIDI plays a huge part in most things we do...but that is probably not enough to have the API put in a browser. It should be in the browser because of the potential for developers to make applications that reach out of the browser and interact with the "real" world. Again, the applications are almost limitless.

My company is a month away from shipping the follow-up to one of our biggest selling products in our history, with expected initial sales to be thousands and thousands of units a month. This products will ship with a PC/Mac desktop Editor/Librarian app. We have spent a considerable amount of time and resources developing this app. Before the product even ships, we are already beginning the process of porting the app over to the browser, utilizing the poly-fill/Jazz Soft midi plug in. If history repeats itself, that would be potentially 500,000 users of the web app. Strong stuff.    

In closing, I am not excited about this because of who I work for. I am excited about this because I spent 15 years as a web application developer and I am passionate about the potential of the platform. 

Scott Mire
Peavey Electronics
Comment 12 User image :Ehsan Akhgari 2013-03-07 10:54:51 PST
Guys, this is not the correct place to advocate for this.  Nobody has said that we _don't_ want to support Web MIDI.  It's just that there are a lot of more important things for us to work on.
Comment 13 User image Darren DeRidder 2013-11-14 10:23:23 PST
Responding to a feature request by advocating lower priority, and then telling the community that they're out of place for advocating higher priority, is unsupportable. This absolutely is the place to advocate.

+1 for Web MIDI.

It's a higher priority than Web Audio for lots of people. Web MIDI would allow the genesis of a new class of web based applications that leverage the vast ecosystem of MIDI-enabled technologies.

Boris has the correct view that it's a matter of resourcing, and I get what Ehsan is saying. I want to be constructive here, so I'll see if I can drum up support from our OttawaJS meetup. We have a lot of good devs including a couple of Mozilla interns.
Comment 14 User image Kyle Machulis [:qdot] 2014-03-13 14:35:57 PDT
I'll give this a shot, though it won't be a full time initiative (I've still got lots of commitments on FxOS). I've done a bit of MIDI driver work (hi pete!) as well as WebAPI development (RIL and bluetooth for FxOS, gamepad oop still in development), so I'm capable of breaking lots of things.

If there's anyone watching this that's interested in helping out (platform specific MIDI would be great, I've got some experience on mac and linux but haven't worked on MIDI code in windows), please let me know by commenting here. 

First task is probably dividing down into subtasks, getting WebAPI matching the current spec state and fleshing out HAL needs per platform.
Comment 15 User image Darren DeRidder 2014-03-15 21:46:22 PDT
Kyle, I started looking into the rtmidi open source cross platform midi classes. Didn't get as far as writing an xpcom component but I'm still interested and would like to help.
Comment 16 User image Kyle Machulis [:qdot] 2014-03-15 22:16:12 PDT
Things have changed enough that there probably won't be XPCOM involved (See the recent post on dev.platform about no more xpcom for web facing objects). Off the top of my head, I'm guessing we'll do all of the platform work back in the HAL, have an IPDL protocol in front of that for IPC, then the DOM side will be WebIDL backed by whatever language is easiest and fulfills needs (been a while since I've touched DOM bindings so I'm not sure if we can deal with events in js impls yet).

rtmidi looks neat, I'll check it out, though I doubt we'll be able to use it. Chrome's also got a webmidi implementation already, though it needs to be pref'd on. Not sure if that landed in Webkit before the fork or went with blink though. Something to look at though.

Gonna try to get a branch running off my gecko-dev repo ASAP, will update bug once that's done. Will also break out above tasks into blocking bugs for this so it'll be easier to figure out what's needed.
Comment 17 User image Boris Zbarsky [:bz] (still a bit busy) 2014-03-15 22:26:39 PDT
Events in JS impls work fine, both in terms of subclassing Event and subclassing EventTarget.
Comment 18 User image Paul Adenot (:padenot) [on PTO until 2017-03-06] 2014-03-18 01:28:57 PDT
Are we sure we want to implement this in javascript ? I'm all for implementing stuff in javascript, but the purpose of MIDI is kind of having very tight scheduling and super low latency.
Comment 19 User image Chris Wilson 2014-03-18 08:37:44 PDT
You will definitely want the delivery of scheduled outgoing messages and timestamping of incoming messages to not be done in JS, as those need to be tightly scheduled*; but the actual delivery of messages ends up being events or JS method anyway, so not critically bad.

* i.e. the WebMIDIAPIShim polyfill can never be good enough, because it has to use setTimeout to deliver scheduled-in-the-future send() messages.  However, the underlying NPAPI control is delivering messages with system-level timestamps, so the input part is okay in terms of timing.
Comment 20 User image Kyle Machulis [:qdot] 2014-03-18 09:20:46 PDT
Yeah, that's the idea. The way it looks in my head currently is:

OS API <-> HAL (C++) <-> IPDL <-> DOM API (JS)

Everything that has to do with timing and communications to the world will happen in the HAL, as putting that before the JS/IPDL would be questionable in terms of scheduling. Doing the DOM work to get things out to content is mostly to keep things cleaner, though if it for some reasons causes massive issues it can always be rethought.
Comment 21 User image Chris Wilson 2014-03-18 09:26:39 PDT
Right.  The part you'll have to do yourself in that case is scheduling on Windows; the Windows MIDI API doesn't have a "send this message at xxx time" method, or a system-level timestamp on incoming messages.  (CoreMIDI on OSX does; it has timestamps on MIDIPackets.)  That will have to not be blocked on JS execution or GC, so it will need to happen on the left side of IPDL.
Comment 22 User image [:fabrice] Fabrice Desré 2014-03-20 11:24:43 PDT
If we want to ever use that in workers (and I think it's likely) we can't use js. Easy decision ;)
Comment 23 User image Chris Wilson 2014-03-20 11:42:29 PDT
Well, there you go then.  Access to Web MIDI from Workers is fairly high priority in the issues list -
Comment 24 User image Kyle Machulis [:qdot] 2014-03-20 12:30:07 PDT
Ok, not a prob, glad this got hashed out now then. Not like there's much happening up there anyways, was just trying to make the boilerplate simpler. :)
Comment 25 User image Kyle Machulis [:qdot] 2014-06-14 21:08:33 PDT
And so it begins.

Also posted intent to implement on dev-platform and dev-webapi. So we'll see how that goes.

Right now I'm just getting WebIDL and DOM boilerplate in. Then it's on to IPDL (the protocol should be pretty simple), then a HAL template that'll send over test data so I can run mochitests. Once the HAL template is done, we can start creating platform specific versions to actually make things work.

Going to break this out into multiple bugs, one for WebIDL and DOM implementation, and one for each platform for the HAL. Assuming this actually keeps momentum for more than like, today, I'll go ahead and schedule a sec review, though I imagine due to priority we'll sit on the bottom of the pile for a while.
Comment 26 User image Alexandre Folle de Menezes 2015-01-19 09:57:33 PST
A new draft was published last week:
Comment 27 User image Kyle Machulis [:qdot] 2015-02-04 10:43:32 PST
Branch at

Is now up to date and back in motion. Right now, working on all of the DOM/IPC plumbing. Goal is to get that done with some sort of simple shim layer under it for testing/completeness sake, then starting work on the platform specific bits.
Comment 28 User image Kyle Machulis [:qdot] 2015-03-26 22:03:56 PDT
Just finished the permissions dialog implementation for this, filing for sec-review early since this is going to be an API used on desktop that hits hardware.
Comment 29 User image Adam Muntner [:adamm] (use NEEDINFO) 2015-05-29 10:42:09 PDT
Now supported by Google Chrome
Comment 30 User image Kyle Machulis [:qdot] 2015-05-29 11:23:43 PDT
Yup, we've been talking to the Chrome people, super happy to see them finally get it shipped. Things are moving along on this side, bug 1123516 (which is a requirement for the WebIDL needs for WebMIDI) is almost done, at which point hopefully there will be more motion over here.
Comment 31 User image Kyle Machulis [:qdot] 2015-08-21 16:26:31 PDT
So, since I haven't updated this bug in forever...

Assuming anyone really wants to follow code, my dev branch is at now. I've been working on this full time for a while now, and am almost done with the gecko specific parts. Most of the WebAPI is implemented (both in-process and e10s'd), mainly down to filling out unit tests and making sure we comply with the spec. Once that's done, I'll be putting the patch up for review (and apologizing in advance to reviewers, it's gonna be a big one. :/ ), and starting work on the platform specific portions.
Comment 32 User image Chris Wilson 2015-08-25 08:43:07 PDT
Really excited to see this.  Ping me if there's anything I can do to help in testing, etc.
Comment 33 User image Marcos Caceres [:marcosc] 2015-08-25 10:06:52 PDT
(In reply to Chris Wilson from comment #32)
> Really excited to see this.  Ping me if there's anything I can do to help in
> testing, etc.

Chris, are there web platform tests?
Comment 34 User image Ryoya Kawai 2015-08-25 23:28:49 PDT
I am writing platform test. But it is not ready to submit yet....
Can you use that test which is still creating by me??
Comment 35 User image Kyle Machulis [:qdot] 2015-09-03 11:56:28 PDT
So, as some might've seen on twitter, I've got hardware interaction working on OS X. This includes device enumeration, and message sending/receiving. That said, this is still what I'd classify as a pre-alpha state, because there are quite a few hacks that need to be cleaned up before I can get this into reviews.

The current todo list, off the top of my head:
- DOM 
-- Packet scheduler (right now we ignore timestamps and just throw stuff over)
-- Clear() handling to spec
-- Fix port information handling/passing
-- Cleanup and comment fixing
-- Permissions string l10n
- Mac
-- Device notification handling (Having CFRunLoop issues)
-- General cleanup
-- Figuring out licensing (I adapted this off of chromium's midi_manager_mac object)
- Windows
-- Not Started
- Linux
-- Not Started
- Android
-- Not started, nor high priority. If you're an android dev that wants to do this, please feel free to take the bug!

I've turned this bug into a metabug to track all of the MIDI work, so it can land separately and we can get testing happening ASAP.
Comment 36 User image :kip (Kearwood Gilbert) 2015-12-15 13:49:56 PST
This is awesome!  I've been looking forward to this for some time.

I am working on automated WebVR input latency testing (possibly to be used with Talos) and can see how the same is important for WebMIDI.  The concept is to implement a puppet that can be scripted to fire events with precise timing and measure response time.  Perhaps the same system could be used for WebMIDI as well.
Comment 37 User image Chris Wilson 2016-01-29 10:37:32 PST
Any update/target date on WebMIDI support?  Talked to a number of manufacturers at NAMM last week who'd like to see it.  :)
Comment 38 User image Jeff Griffiths (:canuckistani) (:⚡︎) 2016-01-29 12:34:58 PST
(In reply to Chris Wilson from comment #37)
> Any update/target date on WebMIDI support?  Talked to a number of
> manufacturers at NAMM last week who'd like to see it.  :)

So far the project is Kyle's - I think there are a bunch of MIDI nerds in Mozilla ( myself included! ) that are excited about it and Kyle is making progress but it's unlikely to attract additional resources unless through community contribution.
Comment 39 User image Kyle Machulis [:qdot] 2016-01-29 13:39:44 PST
So here's where things are:

- Backend DOM Stuff: Like 97% done. Was gonna try to get the timestamp stuff cleaned up, got stalled on that in November, then got yanked onto some other stuff. I'm going to try to get it rebased, slightly more commented, and into review in the next couple of weeks, barring any major interruptions. I suspect reviews are going to take a while though, it's a big ol' chunk of code.

- OSX Platform Specifics: Basically still working like it always did. With some of the work happening on moving the Gamepad API to our PBackground IPC mechanism, they solved one of the issues I'd run into with device hotplugging, so we need to integrate that.

- Linux/Windows Platform Specifics: Unfortunately no real movement here yet. I've talked to Adam Goode from the Blink team about this a bit, and he said he'd at least possibly be available to answer questions. We'll probably use the chromium implementation for reference, much as we did the OS X backend.
Comment 40 User image Adam Goode 2016-01-29 14:29:01 PST
You give me too much credit! I've not done much for Chromium/Blink outside of Linux WebMIDI. But happy to help as much as I can.
Comment 41 User image Kyle Machulis [:qdot] 2016-05-16 18:12:31 PDT
An update for anyone watching this, the DOM implementation of WebMIDI is now in for review on Bug 1201590. I would suspect this is going to take a few (or more) review rounds, and this doesn't involve anything platform specific, but it's a step in a direction!
Comment 42 User image Daniele "Mte90" Scasciafratte 2017-01-26 01:40:40 PST
Any news about that API?
Comment 43 User image Kyle Machulis [:qdot] 2017-01-26 10:30:15 PST

Right now I'm busy on WebAuthn implementation, which was higher priority. However, it shares some aspects of this bug, in that the hardware access code will be rust, and it has a somewhat similar IPC setup. The architectural questions I have to answer to finish WebAuthn will at least give us a path for parts of WebMIDI implementation.

That said, I still don't know when I'm gonna have time to work on it. :(

Note You need to log in before you can comment on or make changes to this bug.