1139843 - Trust USB camera (Microdia Triplex i-mini) doesn't work in webrtc on linux (raw Bayer output)

Status: NEW

(Core :: WebRTC, defect, P5)

Description

[Mike de Boer [:mikedeboer]]

This might be a duplicate of bug 1127382.

I have a - rather old - Trust USB camera, which is connected as 

Bus 001 Device 005: ID 0c45:6029 Microdia Triplex i-mini PC Camera

...according to `lsusb`.

I'm running Ubuntu 14.04 64bit, Intel® Core™ i7-3770 CPU @ 3.40GHz × 8.

Comment 1

[Mike de Boer [:mikedeboer]]

It's important to note that other Linux (GTK) apps, like Cheese, detect the camera just fine.

Comment 2

[Mike de Boer [:mikedeboer]]

It's apparently a Trust Spacec@m 150, succeeded by the Trust Mini Webcam WB-1200p, which looks identical.

I found a used model on Amazon UK: http://www.amazon.co.uk/Trust-13405-TRUST-WB-1200P-WEBCAM/dp/B000185AN8/ref=pd_sxp_f_pt/277-4732895-6021127

Comment 3

[Randell Jesup [:jesup] (needinfo me)]

Please attach the output from NSPR_LOG_MODULES=mediamanager:5,getusermedia:5

Comment 4

[Mike de Boer [:mikedeboer]]

1566545856[7f075c122a00]: MediaManager: default prefs: 0x0 @30fps (min 10)
1566545856[7f075c122a00]: New Media thread for gum
1566545856[7f075c122a00]: OnNavigation for 30
1566545856[7f075c122a00]: OnNavigation for 30
512751360[7f07221ea100]:   Capture Device Index 0, Name USB camera
512751360[7f07221ea100]: Number of Capabilities 12
512751360[7f07221ea100]: type=10 width=160 height=120 maxFPS=30
512751360[7f07221ea100]: type=10 width=176 height=144 maxFPS=30
512751360[7f07221ea100]: type=10 width=320 height=240 maxFPS=30
512751360[7f07221ea100]: type=10 width=352 height=288 maxFPS=30
512751360[7f07221ea100]: type=0 width=160 height=120 maxFPS=30
512751360[7f07221ea100]: type=0 width=176 height=144 maxFPS=30
512751360[7f07221ea100]: type=0 width=320 height=240 maxFPS=30
512751360[7f07221ea100]: type=0 width=352 height=288 maxFPS=30
512751360[7f07221ea100]: type=2 width=160 height=120 maxFPS=30
512751360[7f07221ea100]: type=2 width=176 height=144 maxFPS=30
512751360[7f07221ea100]: type=2 width=320 height=240 maxFPS=30
512751360[7f07221ea100]: type=2 width=352 height=288 maxFPS=30
512751360[7f07221ea100]: Init
512751360[7f07221ea100]: Selected video device
512751360[7f07221ea100]:  VoEHardware:GetRecordingDeviceName: Failed 9013
512751360[7f07221ea100]: Selected audio device
512751360[7f07221ea100]: Audio device 0 allocated
512751360[7f07221ea100]: Allocate
512751360[7f07221ea100]: ChooseCapability: prefs: 0x0 @30-10fps
512751360[7f07221ea100]: chose cap 352x288 @30fps codec 7 raw 0
512751360[7f07221ea100]: Video device 4097 allocated
512751360[7f07221ea100]: Audio config: aec: 1, agc: -1, noise: 1
512751360[7f07221ea100]: Start audio for stream 7f0721a8e880
512751360[7f07221ea100]: Audio config: aec: 0, agc: -1, noise: 0
512751360[7f07221ea100]: Start
512751360[7f07221ea100]: Starting video failed , rv=-2147467259
512751360[7f07221ea100]:  VoEHardware:GetRecordingDeviceName: Failed 9013
512751360[7f07221ea100]: Selected audio device
512751360[7f07221ea100]: Audio device 0 allocated shared
512751360[7f07221ea100]: Audio config: aec: 1, agc: -1, noise: 1
512751360[7f07221ea100]: Start audio for stream 7f0721a91600
512751360[7f07221ea100]: started all sources
1566545856[7f075c122a00]: MediaCaptureWindowState: window 31 capturing video audio   
474793728[7f072dfcd480]: Listener removed by DOM Destroy(), mFinished = 1
512751360[7f07221ea100]: Audio device 0 deallocated but still in use
512751360[7f07221ea100]: Stop
1566545856[7f075c122a00]: Returning success for getUserMedia()
512751360[7f07221ea100]: Deallocate
512751360[7f07221ea100]: Video device 4097 deallocated
1566545856[7f075c122a00]: MediaCaptureWindowState: window 31 capturing  audio   
1566545856[7f075c122a00]: OnNavigation for 31
1566545856[7f075c122a00]: Listener removed on purpose, mFinished = 0
512751360[7f07221ea100]: Audio device 0 deallocated
1566545856[7f075c122a00]: Sent recording-window-ended for window 31 (outer 29)
474793728[7f072dfcd480]: Listener removed by DOM Destroy(), mFinished = 0
1566545856[7f075c122a00]: OnNavigation for 20
1566545856[7f075c122a00]: OnNavigation for 11
1566545856[7f075c122a00]: OnNavigation for 26
1566545856[7f075c122a00]: Releasing MediaManager singleton and thread
1566545856[7f075c122a00]: Shutdown

Comment 5

[Randell Jesup [:jesup] (needinfo me)]

512751360[7f07221ea100]: ChooseCapability: prefs: 0x0 @30-10fps
512751360[7f07221ea100]: chose cap 352x288 @30fps codec 7 raw 0
512751360[7f07221ea100]: Video device 4097 allocated
512751360[7f07221ea100]: Start
512751360[7f07221ea100]: Starting video failed , rv=-2147467259
512751360[7f07221ea100]:  VoEHardware:GetRecordingDeviceName: Failed 9013

Ok, this is all reasonable, but it seems the camera simply didn't open.  It also doesn't support a great set of resolutions: i.e. it's a really bad camera.

You can set webrtc_trace:65535 and WEBRTC_TRACE_FILE=whatever and see what lower-level error if any is reported.  Or try forcing resolution to 320x240 in about:config

Comment 6

[Mike de Boer [:mikedeboer]]

Attachment: webrtc.log

I don't know how to force the resolution as I was unable to find the pref which controls that.

Here's the webrtc_trace output, but I didn't see anything mighty interesting in there. Perhaps you do! :)

Comment 7

[Mike de Boer [:mikedeboer]]

(In reply to Randell Jesup [:jesup] from comment #5)
> Ok, this is all reasonable, but it seems the camera simply didn't open.  It
> also doesn't support a great set of resolutions: i.e. it's a really bad
> camera.

Yeah, it's a super old, whacky camera. It still works, though, even though the picture quality is _very_ bad.

But you know, some people just can't throw away their things! ;-)

Comment 8

[Mike de Boer [:mikedeboer]]

So I tried forcing a resolution of 320x240, like this:

media.navigator.video.default_width > 320
media.navigator.video.default_height > 240

But this didn't help - I tried restarting the browser with the same profile.

It seems like the code that attempts to fetch the camera gets stuck on retrieving the device name. That's all I can gather from the logs, so perhaps the issue is as simple as that?

Comment 9

[Randell Jesup [:jesup] (needinfo me)]

Here's the issue:
DEBUGINFO ; (18:21:22:570 |    0) VIDEO CAPTUR:    0  4097;     16139; Video Capture enumerats supported image formats:
DEBUGINFO ; (18:21:22:570 |    0) VIDEO CAPTUR:    0  4097;     16139;   { pixelformat = S910, description = 'S910' }
DEBUGINFO ; (18:21:22:570 |    0) VIDEO CAPTUR:    0  4097;     16139;   { pixelformat = BA81, description = 'BA81' }
ERROR     ; (18:21:22:570 |    0) VIDEO CAPTUR:    0  4097;     16139; no supporting video formats found

Those are: #define V4L2_PIX_FMT_SN9C10X  v4l2_fourcc('S', '9', '1', '0') /* SN9C10x compression */
and #define V4L2_PIX_FMT_SBGGR8  v4l2_fourcc('B', 'A', '8', '1') /*  8  BGBG.. GRGR.. */

We support
        fmts[0] = V4L2_PIX_FMT_MJPEG;
        fmts[1] = V4L2_PIX_FMT_YUV420;
        fmts[2] = V4L2_PIX_FMT_YUYV;
        fmts[3] = V4L2_PIX_FMT_UYVY;
        fmts[4] = V4L2_PIX_FMT_JPEG;

Note that libyuv (which we use for conversion) doesn't support any of the Bayer formats (just variants of YUV and RGB and Motion-JPEG -- see libyuv.h).

Comment 10

[Mike de Boer [:mikedeboer]]

libyuv _does_ seem to support Bayer formats, but require an additional format conversion from Bayer -> I420. (see format_conversion.h).

I'm a complete n00b here, so apologies if I'm mistaken.

Comment 11

[Randell Jesup [:jesup] (needinfo me)]

I meant webrtc_libyuv.h, sorry - the interface code between us and libyuv.

So, ConvertToI420() would in fact support Bayer formats.

I have a patch... can you try it?  also, can you try that camera on WIndows, and if it doesn't work, get the same webrtc_trace logs?

Comment 12

[Randell Jesup [:jesup] (needinfo me)]

Attachment: add Bayer camera format supports to getUserMedia capture

Comment 13

[Mike de Boer [:mikedeboer]]

Hi Randell, thanks for the patch! It doesn't work yet, with the same message in the webrtc_trace log ('no supporting video formats found').

I think you still need to do the `V4L2_PIX_FMT_SBGGR8` dance.

Comment 14

[Randell Jesup [:jesup] (needinfo me)]

Attachment: add Bayer camera format supports to getUserMedia capture

Comment 15

[Mike de Boer [:mikedeboer]]

On Linux I get a finished gUM flow, but a black screen as camera output. Something tells me things went wrong, because the webrtc trace log file is not populated with data.

$ export NSPR_LOG_MODULES=mediamanager:5,getusermedia:5,webrtc_trace:65535
$ export NSPR_LOG_FILE=/tmp/mediamgr.log
$ export WEBRTC_TRACE_FILE=/tmp/webrtc.log

With the above I get nothing in webrtc.log, but the following in mediamgr.log:

2124412864[7f3b7d522a00]: MediaManager: default prefs: 320x240 @30fps (min 10)
2124412864[7f3b7d522a00]: New Media thread for gum
2124412864[7f3b7d522a00]: OnNavigation for 30
2124412864[7f3b7d522a00]: OnNavigation for 30
1050670848[7f3b43edbd80]:   Capture Device Index 0, Name USB camera
1050670848[7f3b43edbd80]: Number of Capabilities 12
1050670848[7f3b43edbd80]: type=10 width=160 height=120 maxFPS=30
1050670848[7f3b43edbd80]: type=10 width=176 height=144 maxFPS=30
1050670848[7f3b43edbd80]: type=10 width=320 height=240 maxFPS=30
1050670848[7f3b43edbd80]: type=10 width=352 height=288 maxFPS=30
1050670848[7f3b43edbd80]: type=0 width=160 height=120 maxFPS=30
1050670848[7f3b43edbd80]: type=0 width=176 height=144 maxFPS=30
1050670848[7f3b43edbd80]: type=0 width=320 height=240 maxFPS=30
1050670848[7f3b43edbd80]: type=0 width=352 height=288 maxFPS=30
1050670848[7f3b43edbd80]: type=2 width=160 height=120 maxFPS=30
1050670848[7f3b43edbd80]: type=2 width=176 height=144 maxFPS=30
1050670848[7f3b43edbd80]: type=2 width=320 height=240 maxFPS=30
1050670848[7f3b43edbd80]: type=2 width=352 height=288 maxFPS=30
1050670848[7f3b43edbd80]: Init
1050670848[7f3b43edbd80]: Selected video device
1050670848[7f3b43edbd80]:  VoEHardware:GetRecordingDeviceName: Failed 9013
1050670848[7f3b43edbd80]: Selected audio device
1050670848[7f3b43edbd80]: Audio device 0 allocated
1050670848[7f3b43edbd80]: Allocate
1050670848[7f3b43edbd80]: ChooseCapability: prefs: 320x240 @30-10fps
1050670848[7f3b43edbd80]: chose cap 320x240 @30fps codec 7 raw 0
1050670848[7f3b43edbd80]: Video device 4097 allocated
1050670848[7f3b43edbd80]: Audio config: aec: 1, agc: -1, noise: 1
1050670848[7f3b43edbd80]: Start audio for stream 7f3b5055a080
1050670848[7f3b43edbd80]: Audio config: aec: 0, agc: -1, noise: 0
1050670848[7f3b43edbd80]: Start
1050670848[7f3b43edbd80]: started all sources
2124412864[7f3b7d522a00]: MediaCaptureWindowState: window 31 capturing video audio   
2124412864[7f3b7d522a00]: Returning success for getUserMedia()
2124412864[7f3b7d522a00]: OnNavigation for 31
2124412864[7f3b7d522a00]: Listener removed on purpose, mFinished = 0
1050670848[7f3b43edbd80]: Audio device 0 deallocated
1050670848[7f3b43edbd80]: Stop
2124412864[7f3b7d522a00]: Sent recording-window-ended for window 31 (outer 29)
1006421760[7f3b4f434f40]: Listener removed by DOM Destroy(), mFinished = 1
1050670848[7f3b43edbd80]: Deallocate
1050670848[7f3b43edbd80]: Video device 4097 deallocated
1050670848[7f3b43edbd80]: Stop
1050670848[7f3b43edbd80]: Deallocate
2124412864[7f3b7d522a00]: OnNavigation for 20
2124412864[7f3b7d522a00]: OnNavigation for 11
2124412864[7f3b7d522a00]: OnNavigation for 26
2124412864[7f3b7d522a00]: Releasing MediaManager singleton and thread
2124412864[7f3b7d522a00]: Shutdown

I did just pull from fx-team and built that... maybe that's the culprit?

Comment 16

[Randell Jesup [:jesup] (needinfo me)]

Well, that was all blind patching, and thus far no one seems to care about raw-Bayer cameras. 

I think that's all I have time to care about for this; if you (or someone) cares/debugs the issues, I'll help review/etc.  (Unless there's some modern-camera usecase.)  Or if it works in Chrome on linux, find where they're handling the bayer input.

Comment 17

[Mike de Boer [:mikedeboer]]

Thanks! I'll try to take it home.

Comment 18

[Randell Jesup [:jesup] (needinfo me)]

Attachment: add Bayer camera format supports to getUserMedia capture

was missing some entries in a table for Bayer