998444 - [Camera][Nexus-4] Video/Picture previews are distorted

Status: RESOLVED DUPLICATE of bug 1036814

(Firefox OS Graveyard :: Gaia::Camera, defect)

Description

[Diego Marcos [:dmarcos]]

There's a big distortion on the picture/video previews (see attached images) It's worse on video mode. I've seen this problem before but disappeared when I moved to the larch gecko branch. Now that I'm back to gecko master the problem is back.

Gaia      375d87bfef8a5d7135f139da8c17f237b990d3f5
Gecko     https://hg.mozilla.org/mozilla-central/rev/7fe3ee0cf8be
BuildID   20140418040202
Version   31.0a1
ro.build.version.incremental=eng.dietrich.20140301.160048
ro.build.date=Sat Mar  1 16:01:04 UTC 2014

Comment 1

[Diego Marcos [:dmarcos]]

Attachment: videoPreviewPortrait.png

Comment 2

[Diego Marcos [:dmarcos]]

Attachment: picturePreviewLandscaoe.png

Comment 3

[Diego Marcos [:dmarcos]]

Attachment: videoPreviewLandscape.png

Comment 4

[Diego Marcos [:dmarcos]]

Attachment: picturePreviewPortrait.png

Comment 5

[Diego Marcos [:dmarcos]]

Mikeh, do you have any insight here?

Comment 6

[Wayne Chang [:wchang]]

Diego,

Does this happen only on master now or is v1.4 branch also impacted?

Comment 7

[Diego Marcos [:dmarcos]]

1.4 is also impacted

Comment 8

[David Flanagan [:djf]]

Diego,

Do you see this on multiple devices or on nexus 4 only?  

You've attached four images. Do you think that all four are distorted?  Or is the portrait mode photo preview okay?

How do the previews compare to the pictures and videos that we capture?  Are those distorted as well, or do we record correctly?

Can you tell whether it is a gecko or a gaia bug?

Do you think it is the same as bugs 976404 and 994641?

I think we need to get on top of these distortion bugs. If they are gaia bugs or if they are gecko bugs that affect devices other than nexus 4, then I think we should block on them and get fixes into 1.4.

Comment 9

[Diego Marcos [:dmarcos]]

- I see this problem only on Nexus 4
- I attached screenshots of the photo preview for comparison purposes but the bug only affects to the video preview.
- I think this comes from gecko/gonk. We just display the preview as given by gecko
- This bug is the same as 976404 and 994641. We should converge on a single bug. The other open bugs were filed before but the scope of this one is more general. David, Should I mark this one as duplicate of one of the others or the older bugs as duplicates of this one?

Comment 10

[Mike Habicher [:mikeh] (high bugzilla latency)]

I'll close bug 976404 and any others.

Comment 13

[David Flanagan [:djf]]

Looks like Mike is on top of the closing as duplicates issue.

Everyone seems to be pretty sure that this is a gecko issue (and if it is gecko + nexus4 only then we don't need to block on it).  But I haven't seen the proof of that yet.

I think it would be interesting to add some debugging console.log statements to print out exactly what video resolution we're requesting, what we're getting back from gecko, and what sort of CSS transform we're applying to it.  That part of gaia is complicated enough that I wouldn't be surprised if there were still bugs in it. Nexus 4 is our device with the most resolution choices, so if a gaia bug was going to manifest anywhere it would manifest on the nexus 4.

Diego or Justin: do one of you have time to find out what resolution we're requesting, what we're getting, and what CSS transform we're applying to it to make it fit the screen?

Comment 14

[Justin D'Arcangelo [:justindarc]]

I'm going to investigate this to determine if this is on our end in Gaia.

Comment 15

[Mike Habicher [:mikeh] (high bugzilla latency)]

So... I just tried to reproduce this on my nexus-4, with:
- gecko: b2g-inbound:66bc10187f1e
- gaia: master:f7f448ab229abb104392e1f1c461fca050708c9d

...and I am unable to do so. The preview looks fine in both video and picture mode and using both front and back cameras.

Comment 16

[Justin D'Arcangelo [:justindarc]]

Attachment: camera-picture-mode.png

Comment 17

[Justin D'Arcangelo [:justindarc]]

Attachment: camera-video-mode.png

Comment 18

[Justin D'Arcangelo [:justindarc]]

Attachment: camera-video-mode-recording.png

I've attached two screenshots of the DOM Inspector in both the picture and video modes and this third screenshot of video mode while recording. You can see from the DOM Inspector what the width/height of the viewfinder is being set to (it is different for both the picture and video modes because it is based on the preview size). In video mode, we are using the 1280x720 preview size. What I have observed leads me to believe that this aspect ratio distortion is *NOT* occurring in Gaia. The aspect ratio appears distorted in the rear camera in video mode on the Nexus 4. However, when you begin recording, the aspect ratio corrects itself and the viewfinder no longer shows a distorted preview. That being said, there is *NO* change in the width/height/transform being applied in the DOM before/after video recording. Since we are not altering the sizing/positioning of the <video> element before/after recording begins, the cause of the aspect ratio distortion must be happening at the driver level.

Comment 19

[Mike Habicher [:mikeh] (high bugzilla latency)]

Further to comment 15, STR:
1. open Camera app
2. tap repeatedly on the video/picture mode switch

...eventually the camera will wind up in video mode with the viewfinder compressed along the narrow axis.

The logcat when this happens shows a consistent 1280x720 viewfinder/recording resolution.

Comment 20

[Mike Habicher [:mikeh] (high bugzilla latency)]

I have confirmed that when we set the video recorder profile to '720p', both the preview and video size parameters are set to 1280x720 in the driver. (I did this by adding a call to getParameters() after setParameters() and logging the values.)

For good measure, I also commented out the call to configureCamera() in GonkCameraSource::init(), and the aspect ratio weirdness still happens.

Comment 21

[Mike Habicher [:mikeh] (high bugzilla latency)]

If I stub out StartRecordingImpl() to make the call to SetupRecording() but return an error before calling mRecorder->start(), the distortion remains.

So whatever is correcting the aspect ratio weirdness is happening as a result of the call to start():

nsGonkCameraControl::StartRecordingImpl()
  SetupRecording()
    GonkRecorderProfile::ConfigureRecorder()
      GonkRecorder::setVideoFrameRate()
      GonkRecorder::setVideoSize()
  GonkRecorder::start()
    startMPEG4Recording()
      setupMPEG4Recording()
        setupMediaSource()
          setupCameraSource()
            GonkCameraSource::Create()
              GonkCameraSource()
                init()
                  configureCamera()
                    setVideoSize()
                    setPreviewSize()
                    setPreviewFrameRate()
                    PushParameters()
      setupMPEG4MetaData()
      GonkCameraSource::start()
        startCameraRecording()
          GonkCameraHardware::StartRecording()

Comment 22

[Justin D'Arcangelo [:justindarc]]

Just noticed this too. The distortion issue goes away if HDR is enabled also. Weird.

Comment 23

[Mike Habicher [:mikeh] (high bugzilla latency)]

Another data point: if I add |return UNKNOWN_ERROR;| to GonkRecorder::startMPEG4Recording(), just after the call to setupMPEG4MetaData() and just before the call to mWriter->start(), the distortion remains.

Comment 24

[Justin D'Arcangelo [:justindarc]]

Some more clues:

I cycled through every possible ISO, focusMode, sceneMode, whiteBalanceMode and almost all of the effects (some crashed the camera such as the 'whiteboard' effect). The *only* setting that corrected the viewfinder's aspect ratio distortion was when HDR is enabled.

Comment 25

[Mike Habicher [:mikeh] (high bugzilla latency)]

Inder, I think someone who knows the driver code needs to take a look at this.

Comment 26

[Inder]

Mike, is this also an issue on QRD? If not then we won't be able to figure out.

Comment 27

[Mike Habicher [:mikeh] (high bugzilla latency)]

(In reply to Inder from comment #26)
>
> Mike, is this also an issue on QRD? If not then we won't be able to figure out.

Inder, I can't reproduce this on the QRD right now because one of your repos is malfunctioning.

Comment 28

[Hema Koka [:hema]]

Mike is unable to confirm on QRD because the build is still broken because of some mis-configuration. 
Inder, any update on this issue?

Comment 29

[Inder]

Hema, MikeH -- I am looking into the build issues mikeh is having but regarding the camera issue mentioned in this bug, we are not seeing any preview distortions on QRD.

Comment 31

[Justin D'Arcangelo [:justindarc]]

Conversation around this issue has moved to Bug 1036814.