Closed Bug 988119 Opened 7 years ago Closed 7 years ago

Plugin whitelist request: gradecam

Categories

(Firefox Graveyard :: Plugin Click-To-Activate Whitelist, defect)

defect
Not set
normal

Tracking

(Not tracked)

VERIFIED FIXED
Firefox 30

People

(Reporter: richard, Assigned: benjamin)

Details

(Whiteboard: application complete - accepted - qa complete)

Plugin name: GCPlugin
Vendor: GradeCam
Point of contact: Richard Bateman
Current version: 1.9.1.7
Download URL: https://downloads.gradecam.com
Sample URL of plugin in use:  http://downloads.gradecam.com

You can print out a sample bubble sheet and use it to test scanning on the downloads page.  In addition, if you want you can sign up easily for a free account of GradeCam Insight at https://insight.gradecam.com

Further test pages available upon request

Plugin details:
---------------

GradeCam's plugin is used by tens of thousands of teachers in the United States with a growing number elsewhere in the world as well.  We have our own system that uses our plugin and technology but in addition it is licensed for use with other partners, such as Houghton Mifflin Harcourt, Illuminate Education, MasteryConnect, and others.

The primary use of the plugin is to collect image data from a webcam or document camera, display live preview video from said camera in the browser, and process every frame of that video looking for bubble answer sheet forms.  Upon recognizing a form our proprietary technology analyzes that form and sends data to the page where it is analyzed and used by the software on the page.  This technology is used by teachers as a replacement for legacy systems such as Scantron.  Additionally the plugin interacts with some other parts of the system in order to facilitate grade transfer into any computerized gradebook.

Particularly since the Chrome team announced their plans to drop support for NPAPI we have been looking for other ways to provide this support without relying on Browser Plugins.  Most recently we tried compiling our image processing code with emscripten, hoping that we could use it to process camera data from HTML5 APIs in the browser, thus removing part of our reliance on plugins.  For our software to be effective we need to process approximately 10 frames per second, which we can easily accomplish on any relatively modern computer using a plugin and highly optimized C++ code.  Despite our efforts at performance tuning, tweaking compiler and optimizer settings, and running it on the fastest computers we own, the emscripten image processor takes 25 times longer to process -- roughly a second per frame.  We are hoping we can improve the performance further but being able to sufficiently increase the speed of this processing seems extremely unlikely at this point.

In short, we don't currently see any possible way to provide the service we are providing in Firefox using any currently known HTML5 or Firefox features.  In Chrome we hope to use Native Client (nacl) to provide the processing, but the gradebook transfer will require Native Messaging.  Needless to say, we're not terribly excited about having to implement these things differently in each browser.

about:plugins output
--------------------

Mac:
    GCPlugin

        File: GCPlugin.plugin
        Path: /Users/richard/Library/Internet Plug-Ins/GCPlugin.plugin
        Version: GCPlugin 1.9.1.7
        State: Enabled
        GCPlugin

    MIME Type	Description	Suffixes
    application/x-gradecam	GCPlugin	

Windows:
    GCPlugin

        File: npGCPlugin_1.9.1.7.dll
        Path: C:\Users\richard\AppData\Roaming\GradeCam Corporation\GCPlugin\npGCPlugin_1.9.1.7.dll
        Version: 1.9.1.7
        State: Enabled
        GCPlugin

    MIME Type	Description	Suffixes
    application/x-gradecam	GCPlugin


Are there any variations in the plugin file name, MIME types, description, or version from one release to the next?
--

On windows we include the version of the plugin in the filename, but otherwise no.


Are there any known security issues in current or older versions of the plugin?
--

There or no known current or past security issues with GradeCam's plugin.  GCPlugin is built using FireBreath, which to date has never had a known security vulnerability in any of the core libraries and makes careful use of standards libraries to eliminate any possibility of buffer overflows and other such common security issues.  Our javascript API is restricted to the minimum needed and very carefully controlled in an attempt to minimize the possibility of issues.



Unless something changes, I can only imagine that we will be reapplying for whitelist every 3 or 4 releases as needed because there simply aren't any other options that I can find that will let us do what we need to do.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: Request for whitelist → Plugin whitelist request: gradecam
Whiteboard: application complete
25x slower than native is about 10x slower than I would expect based on other emscripten benchmarks and applications we've ported. Unless the C++ is extremely multithreaded of SIMD-capable? Even so, that is very surprising.

Can you make a standalone benchmark of the performance-critical part of this codebase, and I'll investigate that? Hopefully there is a small problem here that can be fixed.
There is about a 5-10x performance difference when building natively between debug mode and our release options, and we optimized our code to take advantage of performance boosts from -ftree-vectorize.  While I have enabled that in our emscripten build and I'm using -O3 I suspect it doesn't help nearly as much in JS.  There is a *lot* of processing of byte arrays, both integer and floating point math, etc.

Unfortunately the performance-critical part of the codebase is the most sensitive part of our intellectual property and I don't know that we can make it available to an outside party. We'd love to be able to provide partial functionality without requiring a plugin but we'd need to improve the speed by about 10x in order to use this even for basic scanning. I will talk to the CEO and see if there is any way we could produce similar or stripped down code that would let you play with optimizations without giving out our full code.  I don't know if he would consider some sort of NDA to release the core technology or not, but I strongly suspect that he would not.
Ok, hopefully we can get access to the code to try to help here. Otherwise, please take a look at

https://github.com/kripken/emscripten/wiki/Optimizing-Code

and in particular the last segment on troubleshooting that was just added, that might be helpful.
Yes, I spent several hours looking over that page and trying different optimization options yesterday =] I haven't tried it on a bunch of different browsers yet; in fact my tests were just in node.js, but I somehow suspect it won't be a 10x speed difference.  All the same, I will attempt it and let you know if I find anything surprising there.
If you tested only in node.js, then you may easily see a huge speedup in the browser. First of all in firefox because it has specific asm.js optimizations that node.js does not do, and also because node.js tends to lag behind v8 development, even if you used the very latest node (and especially if not).
So, very interesting results from this conversation. It took me some time to figure out how to do the timing in the browser (never having done the interacting with emscript code from javascript stuff before) but I got it working and firefox is substantially faster.  For my test I embedded an image file and ran the process step on it 100 times.  Here are the breakdowns, for your information:

Native code: ~5 seconds
Node.js: ~100 seconds
Firefox: ~22 seconds
Chrome: ~146 seconds
Safari: ~ ??  (it's still running, but I need to leave. I'll see if it finishes by the time I get back... going on 5 minutes now)

So the firefox run is promising enough that I'm going to try to mock up a proof of concept using it.  If I can make it work at all with getUserMedia we'll try doing a web worker to fix the whole "this is 1/4 second of blocking javascript" thing and see if it's realistic to use.  The grade transfer feature will still require a plugin and we won't be able to support all of the proprietary cameras that we need this way, but it will give us an option to start moving that direction, I suppose.... at least, in Firefox.
Ok good, that is about what I would expect. A 4x slowdown is worse than the usual 2x, but makes sense given you said the code vectorizes well in native builds. Note that work on SIMD is progressing for JS, so eventually that speedup will not be lost.

(I am surprised though that Chrome and Safari are so slow. If possible, it would be good to file a bug on them, but I guess that might be hard to do here.)
I got back and found that Safari had finished.  2587.115 seconds.
So in related and weird news, later on after having restarted Chrome, etc I tried again and this time Chrome took 15 seconds, significantly faster than Firefox.

Very weird, but very nice as well.
Whiteboard: application complete → application complete - accepted
Builds for testing are now available at http://people.mozilla.org/~bsmedberg/plugin-whitelisting-91f6f3380041/

Please do a QA pass using a new Firefox profile to ensure that the plugin activates without a popup and appears as "Always Activate" in the addon manager. Report back in this bug when QA is complete. Please try to complete QA by the end of this week.
Flags: needinfo?(richard)
Fixed for Firefox 30 beta in bug 992995. Richard have you done QA verification?
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
I have, so as long as it hasn't broken since the 15th of last month it looks good.
Flags: needinfo?(richard)
Status: RESOLVED → VERIFIED
Whiteboard: application complete - accepted → application complete - accepted - qa complete
Target Milestone: --- → Firefox 30
Product: Firefox → Firefox Graveyard
You need to log in before you can comment on or make changes to this bug.