808099 - Camera - video recording - recorded video is always rotated 90° CCW

Status: RESOLVED DUPLICATE of bug 810424

(Firefox OS Graveyard :: General, defect, P1)

Description

[Mike Habicher [:mikeh] (high bugzilla latency)]

Recorded video, when played back, appears rotated by 90° counter-clockwise, regardless of the setting of 'video-param-rotation-angle-degrees'.

Comment 1

[Mike Habicher [:mikeh] (high bugzilla latency)]

Some background:
http://code.google.com/p/spydroid-ipcamera/issues/detail?id=63

In normal execution, logcat shows:

V/GonkRecorder(  406): setParameters: video-param-rotation-angle-degrees=90
V/GonkRecorder(  406): setParameter: key (video-param-rotation-angle-degrees) => value (90)
...and...
E/MPEG4Writer(  406): writeCompositionMatrix 0 degrees
E/MPEG4Writer(  406): writeCompositionMatrix 90 degrees
E/MPEG4Writer(  406): writeCompositionMatrix 0 degrees
(written at the end of the recording)

Suspected that maybe the two writeCompositionMatrix(0) calls were overriding the writeCompositionMatrix(mRotation = 90) call; modified the code to produce:

E/MPEG4Writer(  109): writeCompositionMatrix 90 degrees
E/MPEG4Writer(  109): writeCompositionMatrix 90 degrees
E/MPEG4Writer(  109): writeCompositionMatrix 90 degrees

No difference--VLC still plays the video rotated 90-deg CCW.

Comment 2

[Mike Habicher [:mikeh] (high bugzilla latency)]

Interesting discovery: for videos with 'video-param-rotation-angle-degrees' set to 90 degrees, QuickTime 7.7.2 plays the video -PROPERLY-.  What's going on?  It looks like many (most?) media players _ignore_ the composition matrix that MPEG4Writer appends to the end of a recorded file to tell it which was to rotate the recorded video.  QuickTime, however, seems to respect it.

Something to consider for our own media player.

Comment 3

[David Flanagan [:djf]]

Here are my thoughts after discussing this with Mike on IRC:

It seems to me that the best, but least likely solution would be to rotate the frames when recording them.  That means we're not outputting files that aren't portable to existing players.  

The second best would be to bake the rotation into the video element in gecko. Though if the metadata is at the end of the file, that won't work for streaming media. 

The third best will be to detect the metadata in gaia and rotate appropriately when needed.

And the fallback would be to ignore the metadata and just rotate any video we know came from the phone.

If we're going to have to read metadata from the video files, we'll need to know more about the format. Mike, can you link to the MPEG4Writer module?

Comment 4

[Mike Habicher [:mikeh] (high bugzilla latency)]

(In reply to David Flanagan [:djf] from comment #3)
>
> Mike, can you link to the MPEG4Writer module?

Indeed: http://androidxref.com/4.0.4/xref/frameworks/base/media/libstagefright/MPEG4Writer.cpp

The function that actually writes the rotational transform is writeCompositionMatrix().

Comment 5

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: Write the correct rotational meta-data to the video stream

It turns out that in the case of video recording (at least on unagi), 'video-param-rotation-angle-degrees' must be set to the info.rotation output of get_camera_info().  This required exposing that raw information from GonkCameraHardware::GetSensorOrientation().

With that data plugged in, videos appear with the proper orientation in players that respect the composition matrix.  So far, the only player that seems to it QuickTime.  (Windows Media Player 12, VLC 2.0.3, and Ubuntu's Movie Player do not.)

This change at least makes sure our meta-data is correct.

Comment 6

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Comment on attachment 677927 [details] [diff] [review]
Write the correct rotational meta-data to the video stream

This patch looks OK, but how does android get this right?  Videos from those phones play well in all those players.

We should figure that out, perhaps as a followup.

Comment 7

[Mike Habicher [:mikeh] (high bugzilla latency)]

Confirmed that videos recorded on the Nexus S are also rotated 90-deg CCW.

Comment 8

[Mike Habicher [:mikeh] (high bugzilla latency)]

(In reply to Mike Habicher [:mikeh] from comment #7)
>
> Confirmed that videos recorded on the Nexus S are also rotated 90-deg CCW.

(Videos recorded with B2G loaded onto the Nexus S, that is.)

Comment 9

[Mike Habicher [:mikeh] (high bugzilla latency)]

And for those still following at home, I just recorded a video on the Nexus S running ICS (ref. https://developers.google.com/android/nexus/images#sojuaimm76d ) and it too is rotated 90-deg CCW in everything except QuickTime7.

(It's played properly on the phone itself.)

Comment 10

[Ryan VanderMeulen [:RyanVM]]

This doesn't apply very cleanly to inbound. Please rebase. Also, should this have a test?

Comment 11

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: Write the correct rotational meta-data to the video stream

rebased, as requested.

Comment 12

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/5697b0c7d3d5

Comment 13

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/5697b0c7d3d5

Comment 14

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/59271fe8c55f

Comment 15

[David Scravaglieri [:scravag]]

Reopened in todays build:

gecko-aurora: b2dd5f11c4e1
gaia: 3a1320ad953480759edbf43dd94347416e084416

Comment 16

[David Scravaglieri [:scravag]]

gecko-aurora: 149ad5c9f94d
gaia: 8baa7e64178c2fe8e6a9f07490834de7c67e7a85

In todays build, the rotation is 270° ...

Comment 17

[Dale Harvey (:daleharvey)]

We opened a new bug after this video-param-rotation-angle-degrees fix didnt work