1730020 - Add a mechanism to measure the media performance by recording the passed time on different stages

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, task, P2)

Description

[Alastor Wu [:alwu]]

In order to get one decoded data, we need to go through the following processes

  This represents the different stages that a media data will go through
   within the playback journey.
   
              |---|                      |---|           |------|
              Copy Demuxed            Copy Demuxed        Copy Decoded
              Data                      Data                Video
     |------------- |      |-----------------------------------|
        Request Demux         Request Decode
    |-----------------------------------------------------------|
      Request Data
   
    RequestData : Record the time where MediaDecoderStateMachine(MDSM) starts
    asking for a decoded data to MDSM receives a decoded data.
   
    RequestDemux : Record the time where MediaFormatReader(MFR) starts asking
    a demuxed sample to MFR received a demuxed sample. This stage is a sub-
    stage of RequestData.
   
    CopyDemuxedData : On some situations, we will need to copy the demuxed
    data, which is still not decoded yet so its size is still small. This
    records the time which we spend on copying data. This stage could happen
    multiple times, either being a sub-stage of RequestDemux (in MSE case), or
    being a sub-stage of RequestDecode (when sending data via IPC).
   
    RequestDecode : Record the time where MFR starts asking decoder to return
    a decoded data to MFR receives a decoded data. As the decoder might be
    remote, this stage might include the time spending on IPC trips. This stage
    is a sub-stage of RequestData.
   
    CopyDecodedVideo : If we can't reuse same decoder texture to the
    compositor, then we have to copy video data to to another sharable texture.
    This records the time which we spend on copying data. This stage is a sub-
    stage of RequestDecode.

Therefore, I would like to record the passed time on each stage to know how we can improve the overall performance. Although the critical part would definitely be the decoding part, we still want to know how long it takes. It's also good for us to have recorded data if we want to do the comparasion after we put improvement on the media pipeline.

Comment 1

[Alastor Wu [:alwu]]

Attachment: Bug 1730020 - part1 : implement a new class to record the time which each stage of media pinepline spends on.

Comment 2

[Alastor Wu [:alwu]]

Attachment: Bug 1730020 - part2 : repord the average time for each stage.

Depends on D125140

Comment 3

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

I'm not sure if you're wanting to track samples through decoders, but FWIW I've looked at this a little before and have seen some approaches for libav and windows decoders:

• For libavcodec the output AVFrame has a pkt_dts which should be copied from the input AVPacket to the associated output frame. This can be used to associate inputs to outputs.
• For the Windows decoders, I've seen programs hack the input times so that instead of storing a time, they store an ID. Then on output that ID is used to match inputs to outputs, and also used to lookup the original presentation time for that frame.

Comment 4

[Alastor Wu [:alwu]]

Attachment: Bug 1730020 - part2 : add profiler markers to show additional information.

Depends on D125141

Comment 5

[Alastor Wu [:alwu]]

Attachment: Bug 1730020 - part4 : collect decoding time for certain codec formats.

Depends on D126148

Comment 6

[Alastor Wu [:alwu]]

Attachment: Bug 1730020 - part3 : add gtest.

Depends on D126389

Comment 7

[Alastor Wu [:alwu]]

Attachment: Bug 1730020 - part4 : remove unused variable 'videoDecodeStartTime'.

Depends on D126423

Comment 8

[Sotaro Ikeda [:sotaro]]

CopyDecodedVideo : If we can't reuse same decoder texture to the
compositor, then we have to copy video data to to another sharable texture.
This records the time which we spend on copying data. This stage is a sub-
stage of RequestDecode.

We might need to be careful about this data. This does not directly connected to GPU power consumption. GPU power consumption increases when GPU have more tasks. On windows, software video decoding seemed to consume more power than hardware video decoding from Bug 1733282. Removing video copy with hardware video decoder is connected to use YUV SwapChain as Overlay on Windows.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8799f103c043
part1 : implement a new class to record the time which each stage of media pinepline spends on. r=bryce,padenot
https://hg.mozilla.org/integration/autoland/rev/085fb78aac06
part2 : add profiler markers to show additional information. r=padenot
https://hg.mozilla.org/integration/autoland/rev/b57ce3e0f39f
part3 : add gtest. r=padenot
https://hg.mozilla.org/integration/autoland/rev/580203e4b76a
part4 : remove unused variable 'videoDecodeStartTime'. r=bryce

Comment 10

[Atila Butkovits]

Backed out for causing build bustages.

Backout link: https://hg.mozilla.org/integration/autoland/rev/93c734e36aac9ad7db8f7d779683054f2203a431

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=580203e4b76ad81f6f6bb736de1bfe92584f8c73&selectedTaskRun=Z3FdiLDPSdGNhEkc7crzCg.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=353598293&repo=autoland&lineNumber=14577

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/be3ba5198e9b
part1 : implement a new class to record the time which each stage of media pinepline spends on. r=bryce,padenot
https://hg.mozilla.org/integration/autoland/rev/f61112a4def4
part2 : add profiler markers to show additional information. r=padenot
https://hg.mozilla.org/integration/autoland/rev/a49969885ed8
part3 : add gtest. r=padenot
https://hg.mozilla.org/integration/autoland/rev/78046ce4a00d
part4 : remove unused variable 'videoDecodeStartTime'. r=bryce

Comment 12

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/be3ba5198e9b
https://hg.mozilla.org/mozilla-central/rev/f61112a4def4
https://hg.mozilla.org/mozilla-central/rev/a49969885ed8
https://hg.mozilla.org/mozilla-central/rev/78046ce4a00d