[Mac] not able to capture 60 frames per second

NEW
Unassigned

Status

()

defect
P2
normal
Rank:
18
2 years ago
2 years ago

People

(Reporter: florian, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Reporter

Description

2 years ago
When using an HDMI recorder with getUserMedia and sending the stream to a MediaRecorder (with the patch from bug 1409727 applied, and the media.recorder.video.frame_drops pref flipped), I usually get 35 to 50 frames per second in the resulting recordings captured on my macbook.

After changing the setAlwaysDiscardsLateVideoFrames parameter at http://searchfox.org/mozilla-central/rev/40e8eb46609dcb8780764774ec550afff1eed3a5/media/webrtc/trunk/webrtc/modules/video_capture/mac/avfoundation/video_capture_avfoundation_objc.mm#198 to "NO", I consistently get 60fps.

The frame pacing is very irregular though:
3ms
6ms
21ms
3ms
50ms
20ms
8ms
10ms
20ms
6ms
47ms
11ms
14ms
13ms
11ms
4ms
34ms
9ms
22ms
7ms
5ms
15ms
47ms
6ms
19ms
4ms
3ms
16ms
54ms
5ms
20ms
3ms
3ms
17ms
53ms
4ms
16ms
3ms
4ms
24ms
3ms
51ms
17ms
3ms
2ms
26ms
4ms
48ms
23ms
5ms
4ms
21ms
3ms
49ms
17ms
4ms
3ms
26ms
4ms
38ms
4ms
4ms

The alwaysDiscardsLateVideoFrames documentation:
https://developer.apple.com/documentation/avfoundation/avcapturevideodataoutput/1385780-alwaysdiscardslatevideoframes

An Apple Technical Note that seems very relevant:
https://developer.apple.com/library/content/technotes/tn2445/_index.html
I seem to recall pehrsons having a look at this one. Florian, how important is this?
Rank: 18
Flags: needinfo?(florian)
Flags: needinfo?(apehrson)
Priority: -- → P2
Yeah, we investigated florian's frame drops on IRC and it led to this.

An analysis of why we're causing these frames to drop would be good (are we too slow, then why and how slow?), including if we can do anything simple to avoid it. Also how many devices would be affected -- in particular the built-in ones in macbooks.

Then we'd have to figure out what impact flipping this to "NO" has in terms of memory consumption and growth in a worst case.
Flags: needinfo?(apehrson)
Reporter

Comment 3

2 years ago
(In reply to Paul Adenot (:padenot) from comment #1)
> Florian, how important is this?

Not dropping frames is important for the accuracy of my frame recording based performance tests.

There are 2 possible approaches here:
- add an about:config pref so that I can just flip it when running my tests.
- figure out why we are slow, and fix it to possibly make this benefit all Mac users. The lost frames don't seem related to trying to reach 60fps: when my HDMI recorder was set to generate 30fps, we were still losing some.
Flags: needinfo?(florian)
You need to log in before you can comment on or make changes to this bug.