Closed Bug 839981 Opened 9 years ago Closed 9 years ago

Cannot copy video frames from Camera to Canvas on Unagi

Categories

(Firefox OS Graveyard :: General, defect)

ARM
Gonk (Firefox OS)
defect
Not set
major

Tracking

(firefox20 wontfix, firefox21 wontfix, firefox22 fixed, b2g18+ fixed, b2g18-v1.0.0 wontfix, b2g18-v1.0.1 wontfix)

RESOLVED FIXED
B2G C4 (2jan on)
Tracking Status
firefox20 --- wontfix
firefox21 --- wontfix
firefox22 --- fixed
b2g18 + fixed
b2g18-v1.0.0 --- wontfix
b2g18-v1.0.1 --- wontfix

People

(Reporter: julienw, Unassigned)

References

Details

Attachments

(1 file)

+++ This bug was initially created as a clone of Bug #832745 +++

(patch in Bug 832745 did not fix the bug on Unagi)

Trying to write an application that targets video frames analysis (e.g., QR Code reader, things like this), I have:
 - <video> element, with JS code that sets the source to the camera, .mozSrcObject = stream, which works
 - <canvas> element, to which frames will be copied through Context2D.

The drawImage() call returns an exception:
Exception: [Exception... "Component is not available"  nsresult: "0x80040111 (NS_ERROR_NOT_AVAILABLE)"  location: "JS frame :: app://heart-rate.gaiamobile.org/js/heart-rate.js :: heartrate_computeFrame :: line 70"  data: no]

It does not happens if the <video> targets a local WebM file.

Tracking the exception, I finally found the code path of the issue.

In |nsCanvasRenderingContext2DAzure::DrawImage()|, there is a call on
|CanvasImageCache::Lookup(element, mCanvasElement, &imgSize);| which fails, hence the next call will be on |nsLayoutUtils::SurfaceFromElement(element, sfeFlags);|.

It calls |nsLayoutUtils::SurfaceFromElement(nsHTMLVideoElement* aElement, uint32_t aSurfaceFlags)|, which will then fail on |nsCOMPtr<nsIPrincipal> principal = aElement->GetCurrentPrincipal();| with principal staying null.

Looking in details at GetCurrentPrincipal(), I had the following:
Breakpoint 10, nsHTMLMediaElement::GetCurrentPrincipal (this=0x41983e00) at /home/alex/codaz/B2G/gecko/content/html/content/src/nsHTMLMediaElement.cpp:3406
3406	  if (mSrcStream) {
(gdb) print mSrcStream
$14 = {mRawPtr = 0x4462b700}
(gdb) n
3407	    nsRefPtr<nsIPrincipal> principal = mSrcStream->GetPrincipal();
(gdb) print mSrcStream
$15 = {mRawPtr = 0x4462b700}
(gdb) print principal
$16 = {mRawPtr = 0xbe99b94e}
(gdb) n
3408	    return principal.forget();
(gdb) print principal
$17 = {mRawPtr = 0x0}
(gdb) print mSrcStream
Cannot access memory at address 0x54
(gdb) info breakpoints 
Num     Type           Disp Enb Address    What
9       breakpoint     keep y   0x406b7728 in nsHTMLMediaElement::GetCurrentPrincipal() at /home/alex/codaz/B2G/gecko/content/html/content/src/nsHTMLMediaElement.cpp:3407
	breakpoint already hit 28 times
10      breakpoint     keep y   0x406b7710 in nsHTMLMediaElement::GetCurrentPrincipal() at /home/alex/codaz/B2G/gecko/content/html/content/src/nsHTMLMediaElement.cpp:3406
	breakpoint already hit 16 times
Basically, this bug prevents any application that needs to analyze a video frame.

This should not be difficult to fix (according to Alexandre, we just need to find the correct value to put in an array) but this needs both access to an Unagi and gdb skills that I don't have.
Flags: needinfo?(lissyx+mozillians)
Flags: needinfo?(lissyx+mozillians)
(In reply to Julien Wajsberg [:julienw] from comment #1)
> Basically, this bug prevents any application that needs to analyze a video
> frame.

(think: a "barcode/flashcode scanner" for example)
No longer blocks: 787812
Depends on: 787812
With this patch and patch from bug 832745 I can draw the camera preview onto a canvas.

Test app: https://github.com/mpizza/gaia/tree/master/apps/camera-test
Attachment #723395 - Flags: review?(edwin)
That is awesome news! We did a quick review with julienw when I was in Paris office last wednesday, and we thought we had the correct color format at that time. Turns out we were wrong :)
https://hg.mozilla.org/mozilla-central/rev/ea8197d7d588
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → B2G C4 (2jan on)
You need to log in before you can comment on or make changes to this bug.