664147 - [WebCL] add openCL in gecko

Status: RESOLVED WONTFIX

(Core :: General, enhancement)

Description

[Cellier fabien [:fcellier]]

User-Agent:       Mozilla/5.0 (X11; Linux x86_64; rv:6.0a2) Gecko/20110610 Firefox/6.0a2
Build Identifier: 

Discussion about how we can "integrate" openCL in gecko.

We have 2 solutions:
using wrapper from http://projects.forum.nokia.com/webcl/browser/trunk/wrapper

or use like we have in gfx/thebes for openGL provider 
(http://hg.mozilla.org/mozilla-central/file/e00f1b194440/gfx/thebes (GL|EGL*) or 
https://bitbucket.org/tallion/gecko-webcl/src/e55e84d5eb31/modules/openCL/)

Second question : where did we integrate it? 

Reproducible: Always

Comment 1

[Benoit Jacob [:bjacob] (mostly away)]

Since at this point this is completely internal to the WebCL implementation, do as you want. The OpenGL code (GLContext) in gfx/thebes/ was designed so that it could be reused in diverse parts of the codebase. At this point you don't have this kind of requirement for OpenCL.

There's the question of where to put it; WebGL is in content/canvas/src; I'd say you could put the WebCL implementation in content/webcl, and at this point I'd just put all the OpenCL code in there since it's the only user of it.

Comment 2

[Janne Pietiäinen]

The CLWrapper in Nokia WebCL was originally designed as a portability layer so at least that part of the code could have been re-used in case there's need to support multiple browsers. I don't think there's need for that kind of separation when we're working within the Mozilla tree. CLWrapper also doesn't use dynamic library loading which is obviously essential for Gecko integration.

In my opinion, PRLibrary is the way to go, similar to Fabien's implementation. The library location should be made a preference, however.

I think content/webcl is a good location. We can always reorganize at a later time if needed.

Comment 3

[Adrien Plagnol]

Attachment: openCL library

We've worked together with Fabien on this patch. If everything is fine with it we'll commit it in the following days

Comment 4

[Janne Pietiäinen]

(In reply to comment #3)
> We've worked together with Fabien on this patch. If everything is fine with
> it we'll commit it in the following days

Looks good to me (after a cursory read).

I've been cleaning the IDL files from Nokia WebCL and am prepared to attach a patch tomorrow.

Comment 5

[Janne Pietiäinen]

Attachment: Initial WebCL IDLs, adapted from Nokia WebCL and relicensed to Mozilla tri-license with permission.

Comments?
I've placed the files in dom/interfaces/webcl similar to WebGL. Is this OK?

Comment 6

[Benoit Jacob [:bjacob] (mostly away)]

> Comments?
> I've placed the files in dom/interfaces/webcl similar to WebGL. Is this OK?

Sounds like the right location.

Comment 7

[Adrien Plagnol]

Attachment: openCL library, with factory

I'm updating the openCL lib with a version including a factory class. The lib name has been made a preference and the factory should save the library loading from conflicts

I've also added a simple unit test (which won't build unless LIBXUL_LIBRARY is changed to FORCE_STATIC_LIB in the main makefile)

Comment 8

[Benoit Jacob [:bjacob] (mostly away)]

CC'ing Anant, who is Adrien's GSOC mentor. Suggesting to take this work into account for GSOC evaluation. Adrien, don't hesitate to CC him on all your WebCL bugs!

Comment 9

[Cellier fabien [:fcellier]]

Attachment: add WebCL in DOM (need idl)

hi,
I have added WebCL object and idl bindings in DOM but without security for now.
Perhaps we can add it directly in the getter from nsGlobalWindow -> I think we could do something like geolocation (eg. with one authorisation per site that's why i want to add it in nsGlobalWindow ) 

bjacob, is this kind of security planned for webGL? ( perhaps we can mutualize the code)

pieti
I have some dummy questions

In your patch, in functions like  " [noscript] nsIDOMWebCLEvent enqueueReadBuffer_impl", I don't understand why you use nsIVariant in args and not a simple vector/typdedef or nsTarray<size_t>    

In equeueMap, you said that you use same pointer for WebCL and typedArray. Is this ok with concurent access ?

For the TODO conflict with cl.h, we can use namespace like mozilla::webcl ?

And sometimes you use ISomething instead of nsISomething (WebCLCommand and WebCLContext)(corrected in my patch)

Comment 10

[Benoit Jacob [:bjacob] (mostly away)]

I'm not in favor of security approaches that rely on end-user input (like "Do you allow this site to use WebGL/WebCL") because most users are not technical enough to make this decision.

In the case of WebCL, I think a better approach is to keep it disabled by default for now, so the user who wants it has to go to about:config and set, say, webcl.force-enabled=true.

In the case of WebGL, since it's already enabled by default, the best way forward is to just hurry up addressing the existing security issues (most importantly, the DOS vulnerability).

Comment 11

[Cellier fabien [:fcellier]]

Adrien,
You forgot the modification in build : if you look in my patch, you'll see I added the static lib in build/layout/Makefile.in . Without this, we can't use your lib.

Comment 12

[Adrien Plagnol]

Attachment: openCL library loading fixes, basic security check from preference

I'm patching my patch (is that ok?)
This patch includes : 
- updated makefile for layout
- default preferences in all.js
- simplier preferences reading
- simple security check

For the security check: when a WebCL dom object is created, the webcl.force-enabled preference is read. After that, if the preference is set to false, an error message is displayed in the error console when any method is called.
Is that a correct way to do the job? (I'm really not sure but that's a start)

Comment 13

[Benoit Jacob [:bjacob] (mostly away)]

(In reply to comment #12)
> For the security check: when a WebCL dom object is created, the
> webcl.force-enabled preference is read. After that, if the preference is set
> to false, an error message is displayed in the error console when any method
> is called.
> Is that a correct way to do the job? (I'm really not sure but that's a start)

I would rather have it behave like WebGL: when WebGL context creation fails, for example because it's blacklisted, the getContext() method returns null (and generates an exception), so you can't even call webgl.someFunction since webgl==null.

I don't know what is the equivalent of getContext() for WebCL but I'd do the same there.

Comment 14

[Adrien Plagnol]

Attachment: opencL library, with factory

I'm cleaning up my patch patching mess. This patch contains all the code related to library loading (security patch coming next)

Sorry for the email spam

Comment 15

[Adrien Plagnol]

Attachment: disabling webcl by default

Trivial security check. The pref is read from the dom accessor.

myWebCL = window.webCL; will return NS_ERROR_FAILURE unless webcl.force-enabled is true.

Comment 16

[Cellier fabien [:fcellier]]

Adrien and I have made some progress (http://www.youtube.com/watch?v=OXpHF0TA9fI). We will clean and split patch before submit it (soon).

Comment 18

[Cellier fabien [:fcellier]]

In the meantime, if someone wants to test, you can checkout https://hg.mozilla.org/projects/webcl/ and build webcl branch
nbbody test page is available : http://liris.cnrs.fr/~fcellier/webcl-tests/

Comment 19

[Benoit Jacob [:bjacob] (mostly away)]

Notice that if you want to make test builds, you can push to tryserver: level 1 access is enough. Let me know if you need help with that. Also, if you want we can have automatic building and testing enabled for every changeset on the webcl tree, but I was thinking that at this early stage it wasn't worth the resources.

Comment 20

[Andreas Gal :gal]

Whats the status here?

Comment 21

[Cellier fabien [:fcellier]]

For the patch, I don't know.

The spec from khronos are not released yet and the work continues on hg:
I added async js callback last week and I merge with central.

Compilation works only on macOSX 10.6 + :so I have some headers issues on mac with the try server (<OpenGL/CGLDevice.h>). 
Benoit, I think you had the same kind of issues with webGL in GLContextProviderCGL.mm. 
Any idea how i can fix that?

Comment 22

[Mozilla RelEng Bot]

Try run for 45d923177e73 is complete.
Detailed breakdown of the results available here:
    http://tbpl.mozilla.org/?tree=Try&rev=45d923177e73
Results (out of 6 total builds):
    success: 6
Builds available at http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-45d923177e73

Comment 23

[Benoit Jacob [:bjacob] (mostly away)]

If you want that I try to help with Mac compiler errors, please show me the error messages :-) Somehow the TBPL link doesn't have the usual links.

But really don't bother wbout WebCL on Mac < 10.6. We don't even enable WebGL by default on 10.5 (although it does build and run). The simplest thing might be to just disable WebCL at compile time on old OSes that give you trouble.

Comment 24

[Andreas Gal :gal]

We should definitely not bother with 10.5. Getting 10.6 to run is a bonus. Lets focus on current macosx for new features.

Comment 25

[Andreas Gal :gal]

Hey Fabien, whats the status here? Do you need help with this?

Comment 26

[Cellier fabien [:fcellier]]

Unfortunately, I didn't had much time for webcl last month, sorry.

Nevertheless, we have the skeleton : https://hg.mozilla.org/projects/webcl/
I haven't implemented texture interaction with dom yet but, if someone need it I can do that next week. 

>Do you need help with this?
Feedbacks and opinions are welcome :)

Comment 27

[Chia-hung Tai [:ctai :ctai_mozilla :cht]]

Attachment: WebCL-FXOS.png

I have tried to port Nokia WebCL to Firefox/Firefox OS in webidl. The desktop version of my work is fine (Windows 7 with AMD GPU, Ubuntu with AMD GPU, Ubuntu with nVidia GPU).
Attachment WebCL-FXOS.png is the result running on FX OS. I can get the device info on Nexus 4 in FX OS. But still have some OpenCL driver issue when createContext.It is due to the Nexus 4 based on Android 4.3, and Android remove OpenCL support on 4.3. I just grabbed the libOpenCL.so from 4.2 and push to FX OS. So some functions are not work properly. 
Now I will follow the WebCL working draft to write a Mozilla version of WebCL. Any suggestion is welcome.

Comment 28

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

We have no plans to ship WebCL on any platform, whether desktop or Firefox OS.

For that to change, it would need to be an explicit decision to support it and ship it, and I don't believe the spec is in a state where that can be done yet.  If anyone would like to revisit that decision, please send me an email and we can start that conversation (either in email or in newsgroups).

Comment 29

[Chia-hung Tai [:ctai :ctai_mozilla :cht]]

Hi, Vlad,
It is my personal interest for heterogeneous computing. I took my leisure time to do this. So there is no email about this. I just told my manager for this demo yeterday. I think we can see how the WebCL validtor done by Khronos when they complete it. This is a good experience for me to port a new module to webidl. I really enjoy it. Thanks for your comments.

Comment 30

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

No, there are no plans to support WebCL.  Despite the spec being done, there is little mobile OS support (is there any, by default?), and there's no value in Firefox OS trying to lead here because developers will simply not write code for it.

The future for GPU compute in Firefox is ARB_compute_shader, which is now part of ES 3.1.  Again, happy to discuss this in email or in newsgroups, but not in the bug.