Created attachment 8763454 [details] PoC, probably will cause crash User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36 Steps to reproduce: Playing around with registerProtocolHandler and looks like it needs a lot of improvement as far as checking the passed variables to it. Check the PoC (note after the alert your firefox will most likely crash or hopefully show you the weird UI behavior) The PoC here doesnt OOM crash me (except if I increase the length) but instead causes a weird UI behavior. This might also be a problem with other sister functions of registerProtocolHandler https://crash-stats.mozilla.com/report/index/bp-aee47d9f-3aec-4dd4-9432-589c92160620 Actual results: You can do things like prepare a big protocol handler and then the prompt will cover the entire page without the add button being visible. More interestingly is that if we pass a big enough variable, firefox acts weird, other than an OOM crash, if the length is set a certain way, the whole firefox UI freaks out and we are left with broken UI (newtab loads with full orange [same color as border]) loading any website will seem to load from the statusbar but nothing else shows up. Tested it on latest nightly and latest stable with same behavior. Expected results: firefox should deal with this better. Like other than limiting the length of the variables passed, I dont think protocol handlers are supposed to be bigger than ~100 characters long. google chrome gives a 'invalid protocol name' error btw
Marking security in case I that weird UI behavior part leads to something worthy of it. Im still testing this area but thought I would report initial results. Will update once I (or others) make sure its not security sensitive.
Created attachment 8763463 [details] Screenshot of the weird UI As you can see the process is not unresponsive or high CPU/memory If you need a dump of this state please let me know
After more testing I did not find anything security sensitive as far as I can tell. Feel free to remove security tag if you agree. Other small bugs involving this: - Control characters are allowed in the protocol name and title fields (if we set protocol name to '\b\b\b' it actually deletes the logs in windbg) -The favicon displayed inside Tools>Options>Applications>Application details does not have a specified width and height, so if we set favicon.ico to a big image, its displayed in full size. - 'moz-action' and 'place' are allowed protocol handlers, even though they are used internally (dont think setting them affects the internal usage) but still might want to blacklist them, as well as other internally used URI schemes.
We need to split this bug up into multiple independent bugs, because most of these issues are unrelated. But I agree that the crash here is not an exploit (it's just out of memory), so I'm going to open up this bug.
Can you help split this up as Benjamin suggests?
(In reply to Andrew Overholt [:overholt] from comment #5) > Can you help split this up as Benjamin suggests? Sure, let me see. -A bug report for limiting the size of the protocol name. -A bug report for cleaning any special characters in the protocol name -A bug report for the possibly reserved internally used protocol names -Finally, a report for the unrealized favicon. Is that what you were thinking?
Correction, unresized* favicon
Yes, that's perfect. Thank you!