1796069 - Implement a way to measure realtime (webrtc et al) video performance

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, enhancement, P2)

Description

[Andreas Pehrson [:pehrsons]]

To find what video performance bottlenecks to prioritize we need a way to measure them.

I intend to add profiler markers to gecko for all known video copies and conversions. Getting into capture/decode/encode steps would be handy too.

Then in an extension to the profiler frontend I intend to write something to group and analyze the markers.

This bug is for adding a first round of markers.

Comment 1

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Simplify and make more flexible the PerformanceRecorder API. r?alwu

Comment 2

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - To PerformanceRecorder add a CopyVideoStage to record webrtc video copies. r?alwu!

Comment 3

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Mark video copies with PerformanceRecorder<CopyVideoStage> throughout the gecko realtime video pipe. r?padenot!

Comment 4

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Annotate MediaStreamTrackSources with a tracking id and apply it to CopyVideoStages. r?padenot!

This will allow time consuming tasks (copies, capture, etc.) along the pipe to
be grouped per-track-source when visualizing profiler markers.

Comment 5

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Remove unused BasicTrackSource. r?padenot!

Comment 6

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - To PerformanceRecorder add a DecodeStage for tracking (possibly async) decodes. r?alwu!

Comment 7

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Make PerformanceRecorder threadsafe. r?alwu!

For use cases where PerformanceRecorderMulti has Start() called on one thread,
and Record() on another, typically a callback thread from an underlying decoder.

Comment 8

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Record decoded video frames in AOMDecoder. r?alwu!

Comment 9

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Record decoded video frames in DAV1DDecoder. r?alwu!

Comment 10

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Record decoded video frames in TheoraDecoder. r?alwu!

Comment 11

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Record decoded video frames in VPXDecoder. r?alwu!

Comment 12

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Record decoded video frames in GMPVideoDecoder. r?alwu!

Comment 13

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Record decoded video frames in android/RemoteDataDecoder. r?jolin!

Comment 14

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Record decoded video frames in AppleVTDecoder. r?alwu!

Comment 15

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Record decoded video frames in FFmpegVideoDecoder. r?alwu!

Comment 16

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Record decoded video frames in OmxDataDecoder. r?jolin!

Comment 17

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Record decoded video frames in WMFVideoMFTManager. r?alwu!

Comment 18

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Record decoded video frames in WebrtcGmpVideoCodec. r?mjf!

Comment 19

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Record camera/screen capture with tracking id. r?padenot!

Comment 20

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Apply tracking id to DecodeStages. r?alwu!

Comment 21

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Register AppleVTDecoder callback thread with the profiler. r?padenot!

Comment 22

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Set tracking ids with playback decoders to get more markers. r?alwu!

Comment 23

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Move CallbackThreadRegistry definitions to cpp file. r?padenot!

Comment 24

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Implement thread tracking in CallbackThreadRegistry so exiting threads unregister themselves from the profiler. r?padenot!

This patch uses thread-local storage through nspr to register a destructor
that will unregister the current thread from CallbackThreadRegistry, and
by extension, from the profiler and nsThreadManager too.

~CallbackThreadRegistry will block as part of shutdown to wait for threads
to exit, should there be any tracked ones registered.

Comment 25

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Destroy CallbackThreadRegistry on XPCOMShutdownThreads instead of XPCOMShutdownFinal. r?padenot!

In XPCOMShutdownFinal, nsThreadManager has shut down any non-main nsThreads,
so functions such as GetCurrentThread or IsNSThread cannot be relied on.

XPCOMShutdownThreads is a safe time for CallbackThreadRegistry to be
destroyed since any tracked threads are external, and we should get
release any references to such resources in an earlier phase.

Comment 26

[Andreas Pehrson [:pehrsons]]

Attachment: Bug 1796069 - Ignore registrations in CallbackThreadRegistry if refcnt logging is enabled. r?padenot!

Callback threads that need to be registered with CallbackThreadRegistry are by
definition externally managed; otherwise they wouldn't need to be registered in
the first place.

We may not have any control of the lifetime of the externally managed thread,
which, when registered with the profiler has an nsThread wrapper created
around it. If the thread outlives all xpcom shutdown stages, the nsThread
wrapper instance will leak (it implements nsIThread and nsISupports!). If
refcnt logging is enabled when it leaks, this particular instance will be
flagged as a leak, and could mark a run on try as failed.

To work around this potential leak, this patch makes CallbackThreadRegistry
registration and unregistration noops when refcnt logging is enabled.

I will also note, that the lifetime issues found with externally managed
threads were most notable on Mac with Grand Central Dispatch's Dispatch Queues.
Over time a large number of real threads could be seen, and the risk of one of
them outliving xpcom shutdown was close to 100% for a common use case involving
camera capture (though timing dependent).

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3386fd76208c
Simplify and make more flexible the PerformanceRecorder API. r=alwu
https://hg.mozilla.org/integration/autoland/rev/fb8bcc54d864
To PerformanceRecorder add a CopyVideoStage to record webrtc video copies. r=alwu
https://hg.mozilla.org/integration/autoland/rev/2477d101c651
Mark video copies with PerformanceRecorder<CopyVideoStage> throughout the gecko realtime video pipe. r=padenot
https://hg.mozilla.org/integration/autoland/rev/e5a0e684e6a9
Annotate MediaStreamTrackSources with a tracking id and apply it to CopyVideoStages. r=padenot
https://hg.mozilla.org/integration/autoland/rev/132d57446788
Remove unused BasicTrackSource. r=padenot
https://hg.mozilla.org/integration/autoland/rev/a968b1ab1a0c
To PerformanceRecorder add a DecodeStage for tracking (possibly async) decodes. r=alwu
https://hg.mozilla.org/integration/autoland/rev/b70e54dd3ec7
Make PerformanceRecorder threadsafe. r=alwu
https://hg.mozilla.org/integration/autoland/rev/1219fc624920
Record decoded video frames in AOMDecoder. r=alwu
https://hg.mozilla.org/integration/autoland/rev/353c027b2929
Record decoded video frames in DAV1DDecoder. r=alwu
https://hg.mozilla.org/integration/autoland/rev/2304ce742aea
Record decoded video frames in TheoraDecoder. r=alwu
https://hg.mozilla.org/integration/autoland/rev/d89846418f30
Record decoded video frames in VPXDecoder. r=alwu
https://hg.mozilla.org/integration/autoland/rev/db2aa9c89371
Record decoded video frames in GMPVideoDecoder. r=alwu
https://hg.mozilla.org/integration/autoland/rev/7a522a083f8e
Record decoded video frames in android/RemoteDataDecoder. r=jolin,geckoview-reviewers,owlish
https://hg.mozilla.org/integration/autoland/rev/089c68e8f61f
Record decoded video frames in AppleVTDecoder. r=alwu
https://hg.mozilla.org/integration/autoland/rev/c207e5d2bf3c
Record decoded video frames in FFmpegVideoDecoder. r=alwu
https://hg.mozilla.org/integration/autoland/rev/2fa6734258d2
Record decoded video frames in OmxDataDecoder. r=jolin
https://hg.mozilla.org/integration/autoland/rev/d0e0b7bf4303
Record decoded video frames in WMFVideoMFTManager. r=alwu
https://hg.mozilla.org/integration/autoland/rev/ab47ea1bc28a
Record decoded video frames in WebrtcGmpVideoCodec. r=mjf
https://hg.mozilla.org/integration/autoland/rev/461b7234402c
Record camera/screen capture with tracking id. r=padenot
https://hg.mozilla.org/integration/autoland/rev/08691c305d21
Apply tracking id to DecodeStages. r=alwu
https://hg.mozilla.org/integration/autoland/rev/a09f857b4881
Move CallbackThreadRegistry definitions to cpp file. r=padenot
https://hg.mozilla.org/integration/autoland/rev/63105f58a332
Ignore registrations in CallbackThreadRegistry if refcnt logging is enabled. r=padenot
https://hg.mozilla.org/integration/autoland/rev/76e9498e7e20
Register AppleVTDecoder callback thread with the profiler. r=padenot
https://hg.mozilla.org/integration/autoland/rev/edde6d7420b7
Set tracking ids with playback decoders to get more markers. r=alwu

Comment 28

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/3386fd76208c
https://hg.mozilla.org/mozilla-central/rev/fb8bcc54d864
https://hg.mozilla.org/mozilla-central/rev/2477d101c651
https://hg.mozilla.org/mozilla-central/rev/e5a0e684e6a9
https://hg.mozilla.org/mozilla-central/rev/132d57446788
https://hg.mozilla.org/mozilla-central/rev/a968b1ab1a0c
https://hg.mozilla.org/mozilla-central/rev/b70e54dd3ec7
https://hg.mozilla.org/mozilla-central/rev/1219fc624920
https://hg.mozilla.org/mozilla-central/rev/353c027b2929
https://hg.mozilla.org/mozilla-central/rev/2304ce742aea
https://hg.mozilla.org/mozilla-central/rev/d89846418f30
https://hg.mozilla.org/mozilla-central/rev/db2aa9c89371
https://hg.mozilla.org/mozilla-central/rev/7a522a083f8e
https://hg.mozilla.org/mozilla-central/rev/089c68e8f61f
https://hg.mozilla.org/mozilla-central/rev/c207e5d2bf3c
https://hg.mozilla.org/mozilla-central/rev/2fa6734258d2
https://hg.mozilla.org/mozilla-central/rev/d0e0b7bf4303
https://hg.mozilla.org/mozilla-central/rev/ab47ea1bc28a
https://hg.mozilla.org/mozilla-central/rev/461b7234402c
https://hg.mozilla.org/mozilla-central/rev/08691c305d21
https://hg.mozilla.org/mozilla-central/rev/a09f857b4881
https://hg.mozilla.org/mozilla-central/rev/63105f58a332
https://hg.mozilla.org/mozilla-central/rev/76e9498e7e20
https://hg.mozilla.org/mozilla-central/rev/edde6d7420b7