1253974 - Playback metadata may take a lot of memory for some videos

Status: NEW

(Core :: Audio/Video: Playback, defect, P3)

Description

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Spawned from bug 1253471 comment 9.

Some sites have video streams with a lot of metadata.
For example, http://embed.azubu.tv/video188762SVgameshowcsgoen provides a 40MB 'moov', which covers 26 hours at 50Hz. It is believed this metadata is kept around during playback.
Also, after parsing, an index of frames is needed during playback, which takes 214MB (4.7M items, 48 bytes each).

It would be nice to see if some optimizations are possible, to reduce this memory usage while not harming playback.

Some crude ideas:
- Discard source metadata (making sure it is not needed anymore of course).
- Reduce the size of the index items (possibly at the cost of some extra processing during playback -- to be studied).
- Instead of pre-computing the index table, keep a subset of the source metadata, and use it to find indices when needed during playback (cost analysis needed as well).

Feedback & ideas welcome.

Comment 1

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

A note, spawned from bug 1253471 comment 13:
If we manage to reduce the memory taken by the metadata, we may want to revisit the box size limit (128MB when bug 1253471 lands) to allow even-bigger-than-reasonable videos.

Comment 2

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Disregard comment 1; the arbitrary metadata size limit will be removed in bug 1253471.

Comment 3

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Lets fix this in the rust demuxer instead of trying to fix it the C++ code. The best way to do it is probably to break it up into chunks. For example if we have a million samples we could process the first 10,000 and save sufficient state to process the next 10,000. At that point we can discard the first 10,000 samples again.

Comment 4

[Jean-Yves Avenard [:jya]]

the use of memory is in the MP4Demuxer , we've already split the MP4Metadata component so that stagefright can be seemingly replaced with the rust demuxer (which is just parsing the metadata anyway, and not really demuxing).

As such, I don't think rust vs stagefright makes any differences here.