Closed Bug 2082607 Opened 1 month ago Closed 24 days ago

Video decoders created while the GPU process restarts use the previous compositor's settings

Categories

(Core :: Audio/Video, defect)

Product:
Core
Component:
Audio/Video
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
160 Branch
Tracking Flags:
Tracking Status
firefox160 --- fixed

People

(Reporter: aosmond, Assigned: aosmond)

References

(Blocks 1 open bug)

Details

Attachments

(4 external files, 1 obsolete file)

When the GPU process crashes and is recreated, a video decoder created in the GPU or RDD process during the restart can be set up with the TextureFactoryIdentifier of the compositor that just went away.

The content process passes its TextureFactoryIdentifier when creating a remote video decoder. That identifier comes from the caller's KnowsCompositor:

  • for media elements, a KnowsCompositorMediaProxy snapshot taken by MediaDecoder::GetCompositor(), and only refreshed when the owner's activity changes;
  • for WebCodecs, the ImageBridgeChild singleton.

Both can be from the old compositor:

  • A creation that starts before ContentChild::RecvReinitRendering keeps the snapshot it captured. This applies to creations waiting for the GPU process to come back, falling back from one decoder module to another, waiting in AllocationPolicy, or being recreated by MediaChangeMonitor.
  • ImageBridgeChild::ReinitForContent replaces the image bridge, so a reference held from before the restart points at the dead one.

The decoder process uses the identifier to decide the compositor backend, whether D3D11 textures can be shared, whether hardware WebRender is in use, and which sync handle to use. With the old identifier, a decoder can:

  • make hardware or software decisions that don't match the new compositor, e.g. if the restart came with a fallback from acceleration;
  • in the GPU process, open a SyncObjectClient for a compositor that no longer exists.

Before GPU decoder creation waited for the GPU process to be recreated, such creations failed and fell back to the RDD process. That fallback received the same old identifier.

There is a related race in the RDD and utility (for MF CDM) processes. After the GPU process restarts, the parent sends them a new video bridge (RDDChild::OnCompositorUnexpectedShutdown -> CreateRddVideoBridge), but nothing orders this against decoder creation requests from content. A decoder created in that window can't reach the new compositor until the video bridge is recreated.

Steps to reproduce (timing-dependent):

  1. On Windows, with decoding in the GPU process enabled, open a page that plays video.
  2. Crash the GPU process (e.g. load about:crashgpu).
  3. While the GPU process is restarting, start new video playback, or trigger decoder recreation (e.g. seek, or a resolution change).

Expected: decoders created during or after the restart are set up for the new compositor.

Actual: they may be set up with the old compositor's TextureFactoryIdentifier.

Proposed fix: have the decoder process ask the compositor for the identifier used by the content process, over its existing PVideoBridge connection, keyed by ContentParentId. Decoders in the GPU, RDD and utility processes then all use the same path. Decoder creation would also wait for a new video bridge when the old one has gone away. The content process would then only need to say whether the decoded frames should be shared with the compositor, so media elements would no longer need to carry a compositor snapshot.

If a remote video decoder is created after the compositor was lost, but before
its process was given a new video bridge, it cannot share the decoded frames
with the new compositor until the video bridge is recreated. Creation now waits
for the new video bridge.

Video decoders are given the TextureFactoryIdentifier by the content process,
which may be from a compositor that no longer exists, e.g. if the decoder was
created while the GPU process was being recreated. PVideoBridge now allows the
decoder process to ask the compositor for the identifier used by a content
process instead.

The identifier given by the content process may be from a compositor which no
longer exists, e.g. if the decoder was created while the GPU process was being
recreated. Remote video decoders now ask the compositor for the identifier used
by the content process over their video bridge, so the content process only
needs to say whether the decoded frames should be shared with the compositor.
If the compositor cannot provide an identifier, decoders in the GPU process fail
so that another process may be used, and decoders elsewhere copy the decoded
frames to the content process instead. The KnowsCompositor for the decoder is
now created when the decoder is constructed, rather than when the actor is
created.

MediaDecoder gave the reader a KnowsCompositorMediaProxy for the document's
compositor, which kept the TextureFactoryIdentifier and image bridge from when
it was created, and was only updated when the owner's activity changed. The
reader no longer gives a compositor, and PDMFactory instead uses the current
image bridge in the content and parent processes whenever it creates a decoder
that was not given one, as WebCodecs and WebRTC already do, since it is replaced
and given the new identifier when the compositor is reinitialized. This avoids
using a replaced image bridge when the decoder creation parameters are kept
across asynchronous steps, e.g. by MediaChangeMonitor or AllocationPolicy.

This also affects MediaCapabilities, which does not give a compositor when its
document has none, e.g. in workers or tabs that have not been painted yet. Its
decoder probes now use the image bridge, once the content process has a
compositor, and so may now use the GPU process, as media element playback would.

The identifier given by the content process may be from a compositor which no
longer exists, e.g. if the decoder was created while the GPU process was being
recreated. Remote video decoders now ask the compositor for the identifier used
by the content process over their video bridge, so the content process no
longer gives one.

If the compositor cannot provide an identifier, decoders in the RDD and utility
processes copy the decoded frames back to the content process, as they already
did when the content process did not give an identifier. Decoders in the GPU
process fail instead so that another process may be used, as the content
process previously did not create them there without a compositor. The
KnowsCompositor for the decoder is now created when the decoder is constructed,
rather than when the actor is created.

Attachment #9661707 - Attachment is obsolete: true
Blocks: 2082552
Depends on: 2083416
See Also: → 2083416
See Also: 2083416 →
Pushed by aosmond@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/2771daa18f44 https://hg.mozilla.org/integration/autoland/rev/d8070538e5b4 Part 1. Wait for new video bridge before creating remote video decoders while the compositor is recreated. r=media-playback-reviewers,padenot https://github.com/mozilla-firefox/firefox/commit/0c522bc3e7cf https://hg.mozilla.org/integration/autoland/rev/2db49e516efc Part 2. Allow video decoders to get the TextureFactoryIdentifier of a content process from the compositor. r=gfx-reviewers,sotaro https://github.com/mozilla-firefox/firefox/commit/6f3066a651a2 https://hg.mozilla.org/integration/autoland/rev/d71e0e964874 Part 3. Ask the compositor for the TextureFactoryIdentifier when creating remote video decoders. r=media-playback-reviewers,padenot https://github.com/mozilla-firefox/firefox/commit/37f624225e6d https://hg.mozilla.org/integration/autoland/rev/6fa3eb08b69f Part 4. Use the current image bridge for media element video decoders. r=media-playback-reviewers,padenot
Blocks: 2085189
Regressions: 2086107
QA Whiteboard: [qa-triage-done-c161/b160]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: