1756260 - Implement a mechanism to communicate with the remote media engine

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, task, P1)

Description

[Alastor Wu [:alwu]]

In the content process, we would need communicate with IMFMediaEngine which will be implemented in the RDD process.

This bug aims to build up a complete mechanism between two processes.

Comment 1

[Alastor Wu [:alwu]]

Attachment: WIP: Bug 1756260 - part1 : make `LaunchRDDProcessIfNeeded()` public.

In following patch, we will need to explicitly launch the RDD process outside the RemoteDecoderManagerChild, which requires to make LaunchRDDProcessIfNeeded() public.

Comment 2

[Alastor Wu [:alwu]]

Attachment: Bug 1756260 - part3 : implement a mechanism to communicate with the MediaEngine in the remote process.

Depends on D139203

Comment 3

[Alastor Wu [:alwu]]

Attachment: WIP: Bug 1756260 - part3 : use MediaEngineChild in the media pipeline.

Comment 4

[Alastor Wu [:alwu]]

Attachment: WIP: Bug 1756260 - part3 : use MediaEngineChild in the media pipeline.

Depends on D139204

Comment 5

[Alastor Wu [:alwu]]

Attachment: Bug 1756260 - part1 : split some of current state machine methods and variables to another new base class preparing for a new type of state machine.

Comment 6

[Alastor Wu [:alwu]]

Attachment: Bug 1756260 - part2 : implement a new state machine for the external engine.

Depends on D140013

Comment 7

[Alastor Wu [:alwu]]

Attachment: Bug 1756260 - part4 : set the media engine Id to the format reader, which would be used to find correct remote decoders.

Depends on D139204

Comment 8

[Alastor Wu [:alwu]]

Attachment: Bug 1756260 - part7 : use ExternalEngineStateMachine in MediaSourceDecoder.

Now it's just used for testing and development purpose.

In the future, this should only be used for encrypted playback and we would also need a fallback mechanism if the externak engine fails.

Depends on D140153

Comment 9

[Alastor Wu [:alwu]]

Attachment: Bug 1756260 - part8 : use a light-weight state object on ExternalEngineStateMachine.

Comment 10

[Alastor Wu [:alwu]]

Attachment: Bug 1756260 - part6 : create remote decoders based on the engine Id.

By checking the id in the CreateDecoderParams, we can use it to find the correspond MFMediaEngineParent and ask it to create a real decoder, which is implemented by Media Engine API.

The data flow would be like that, the encoded data would be passed to the RDD process by using the remote decoder, and remote decoder would feed the data into media engine decoder, such as MFMediaEngineVideoStream and MFMediaEngineAudioStream.

Depends on D140153

Comment 11

[Alastor Wu [:alwu]]

Attachment: Bug 1756260 - part5 : pass the engine Id via IPC.

We need to padd the media engine Id to the remote process in order to create correct remote decoders.

Depends on D140153

Comment 12

[Alastor Wu [:alwu]]

These patches are ready to be reviewed again. Now with these patches and bug 1758789, audio playback could be done via the Media Engine API.

Comment 13

[Alastor Wu [:alwu]]

Attachment: Bug 1756260 - part9 : only execute engine's methods after it got intialized correctly and not in a shutdown state.

If we call them too early, the IPC connection might not be created yet.

Depends on D143678

Comment 14

[Alastor Wu [:alwu]]

Attachment: Bug 1756260 - part10 : fix macOS build error.

/builds/worker/fetches/MacOSX10.12.sdk/usr/include/MacTypes.h:542:16: error: reference to 'Point' is ambiguous

The point type is conflict with the gfx::Point, so move the media data out of the unify build.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ed9640359f3a
part1 : split some of current state machine methods and variables to another new base class preparing for a new type of state machine. r=jolin
https://hg.mozilla.org/integration/autoland/rev/34b10d427f0f
part2 : implement a new state machine for the external engine. r=jolin
https://hg.mozilla.org/integration/autoland/rev/9fc71b8478d5
part3 : implement a mechanism to communicate with the MediaEngine in the remote process. r=jolin
https://hg.mozilla.org/integration/autoland/rev/b69ee552b64e
part4 : set the media engine Id to the format reader, which would be used to find correct remote decoders. r=jolin
https://hg.mozilla.org/integration/autoland/rev/93245b0e2118
part5 : pass the engine Id via IPC. r=jolin
https://hg.mozilla.org/integration/autoland/rev/46e8a8457044
part6 : create remote decoders based on the engine Id. r=jolin
https://hg.mozilla.org/integration/autoland/rev/d624c80ba67f
part7 : use ExternalEngineStateMachine in MediaSourceDecoder. r=jolin
https://hg.mozilla.org/integration/autoland/rev/1a9aa3f175d0
part8 : use a light-weight state object on ExternalEngineStateMachine. r=jolin
https://hg.mozilla.org/integration/autoland/rev/391201cf338b
part9 : only execute engine's methods after it got intialized correctly and not in a shutdown state. r=jolin
https://hg.mozilla.org/integration/autoland/rev/d2339ec8194e
part10 : fix macOS build error. r=media-playback-reviewers,bryce

Comment 16

[Cristian Tuns]

Backed out for causing build bustages on ExternalEngineStateMachine.obj

• Backout link
• Push with failures
• Failure Log
• Failure Log 2
• Failure line: /builds/worker/checkouts/gecko/dom/media/ExternalEngineStateMachine.cpp:185:11: error: unknown type name 'TimeUnit'; did you mean 'media::TimeUnit'?
• Build bustage log: https://treeherder.mozilla.org/logviewer?job_id=377644158&repo=autoland

Comment 17

[Alastor Wu [:alwu]]

Try for windows looks good https://treeherder.mozilla.org/jobs?repo=try&revision=b1283647a7e19f8b377292c509eb26a24c8a0651

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b6f98bdaa557
part1 : split some of current state machine methods and variables to another new base class preparing for a new type of state machine. r=jolin
https://hg.mozilla.org/integration/autoland/rev/a7190215baac
part2 : implement a new state machine for the external engine. r=jolin
https://hg.mozilla.org/integration/autoland/rev/4d2d5b2974cb
part3 : implement a mechanism to communicate with the MediaEngine in the remote process. r=jolin
https://hg.mozilla.org/integration/autoland/rev/79a967c1d605
part4 : set the media engine Id to the format reader, which would be used to find correct remote decoders. r=jolin
https://hg.mozilla.org/integration/autoland/rev/455935df901d
part5 : pass the engine Id via IPC. r=jolin
https://hg.mozilla.org/integration/autoland/rev/7f518efee9db
part6 : create remote decoders based on the engine Id. r=jolin
https://hg.mozilla.org/integration/autoland/rev/49644876a6a2
part7 : use ExternalEngineStateMachine in MediaSourceDecoder. r=jolin
https://hg.mozilla.org/integration/autoland/rev/9eacbae59c29
part8 : use a light-weight state object on ExternalEngineStateMachine. r=jolin
https://hg.mozilla.org/integration/autoland/rev/5613cede41c3
part9 : only execute engine's methods after it got intialized correctly and not in a shutdown state. r=jolin
https://hg.mozilla.org/integration/autoland/rev/d85b9d46c688
part10 : fix macOS build error. r=media-playback-reviewers,bryce

Comment 19

[bszekely]

https://hg.mozilla.org/mozilla-central/rev/b6f98bdaa557
https://hg.mozilla.org/mozilla-central/rev/a7190215baac
https://hg.mozilla.org/mozilla-central/rev/4d2d5b2974cb
https://hg.mozilla.org/mozilla-central/rev/79a967c1d605
https://hg.mozilla.org/mozilla-central/rev/455935df901d
https://hg.mozilla.org/mozilla-central/rev/7f518efee9db
https://hg.mozilla.org/mozilla-central/rev/49644876a6a2
https://hg.mozilla.org/mozilla-central/rev/9eacbae59c29
https://hg.mozilla.org/mozilla-central/rev/5613cede41c3
https://hg.mozilla.org/mozilla-central/rev/d85b9d46c688