1100203 - (foxeye) [FoxEye][Meta] Project FoxEye: Bring modern computer vision and image processing technology to the Web.

Status: NEW

(Core :: Audio/Video: MediaStreamGraph, defect, P5)

Description

[Chia-hung Tai [:ctai :ctai_mozilla :cht]]

The goal of this project is bringing the power of computer vision and image processing to the Web. By extending the spec of Media Capture and Streams, the web developers can write video processing related applications in better way. The primary idea is to incorporate Worker-based JavaScript video processing with MediaStreamTrack. The user's video processing script can do the real image processing and analysis works frame by frame.

Comment 1

[David Bruant]

Short of a strong signal from other implementors, bug 961689 (which is "only" an optimization, no new API) seems like a more realistic approach than bugs 801176 and 709490 (new APIs available to web workers) if the goal is to bring something to the Web.

Comment 2

[Chia-hung Tai [:ctai :ctai_mozilla :cht]]

The main spirit of this project is not bugs 801176 and bug 709490. The whole blue print is in https://wiki.mozilla.org/Project_FoxEye . Since it is still in early stage, the wiki will be changed frequently.



(In reply to David Bruant from comment #1)
> Short of a strong signal from other implementors, bug 961689 (which is
> "only" an optimization, no new API) seems like a more realistic approach
> than bugs 801176 and 709490 (new APIs available to web workers) if the goal
> is to bring something to the Web.

Comment 3

[Chia-hung Tai [:ctai :ctai_mozilla :cht]]

In current Web technology, we don't have a good(generic) way to do video/camera processing/analysis task. That limited the application developers to show their creative ideas on Web platform with camera and video. Those kind of features are important for mobile OS. There are of lots of popular Apps about camera/video processing area. For example, Amazon firefly, WeVideo, WordLens, Waygo, AR effects are popular apps in Android and iOS. 

Project FoxEye unleash those constraints by providing MediaStreamTrack with worker, OfflineMediaContext and WebImage library.

Comment 4

[Chia-hung Tai [:ctai :ctai_mozilla :cht]]

API draft for

Comment 5

[Chia-hung Tai [:ctai :ctai_mozilla :cht]]

API draft: MediaStream extension for VideoWorker.
http://chiahungtai.github.io/mediacapture-worker/
Feel free to discuss it.

Comment 6

[Bobby Chien]

[Tracking Requested - why for this release]:

Comment 7

[Paul Adenot (:padenot)]

We don't have plans to pursue this experiment for now.

Comment 8

[Xtrempoint]

Through the use of cutting-edge machine learning algorithms and deep neural networks, FoxEye can analyze and interpret images and video in real-time, enabling a range of advanced features such as object recognition, face detection, and scene segmentation. This technology has a wide range of potential applications, from e-commerce and advertising to security and surveillance. Find your more interesting facts there https://visegrad24.info/category/tech/