1328231 - Replay WebRendercmds in WebRenderBridgeParent when OMTA is running

Status: RESOLVED INVALID

(Core :: Graphics: WebRender, defect)

Description

[Peter Chang[:pchang]]

To support OMTA, need to replay webrendercmds via scheduling a composition with new sampled animation value.

Comment 1

[Peter Chang[:pchang]]

Replay cmds from WebRenderBridgeParent::RecvDPEnd or WebRenderBridgeParent::RecvDPSyncEnd

Comment 2

[Peter Chang[:pchang]]

My idea is to save the following WebRenderCommand and DestroyOp and replay them if animation is running.

mozilla::ipc::IPCResult                                                                                                                                                                                     
WebRenderBridgeParent::RecvDPEnd(InfallibleTArray<WebRenderCommand>&& aCommands,                                                                                                                            
                                 InfallibleTArray<OpDestroy>&& aToDestroy,                                                                                                                                  
                                 const uint64_t& aFwdTransactionId,                                                                                                                                         
                                 const uint64_t& aTransactionId)                                                                                                                                            
{                                                                                                                                                                                                           
  HandleDPEnd(Move(aCommands), Move(aToDestroy), aFwdTransactionId, aTransactionId);                                                                                                                        
  return IPC_OK();                                                                                                                                                                                          
}                                                                                                                                                                                                           
                                                                                                                                                                                                            
mozilla::ipc::IPCResult                                                                                                                                                                                     
WebRenderBridgeParent::RecvDPSyncEnd(InfallibleTArray<WebRenderCommand>&& aCommands,                                                                                                                        
                                     InfallibleTArray<OpDestroy>&& aToDestroy,                                                                                                                              
                                     const uint64_t& aFwdTransactionId,                                                                                                                                     
                                     const uint64_t& aTransactionId)                                                                                                                                        
{                                                                                                                                                                                                           
  HandleDPEnd(Move(aCommands), Move(aToDestroy), aFwdTransactionId, aTransactionId);                                                                                                                        
  return IPC_OK();                                                                                                                                                                                          
}

Comment 3

[Peter Chang[:pchang]]

Now I'm able to reply the WrCmds but I have to refactor the animation code in gfx/layer/composite first to call SampleAnimation in WebRenderBridgeParent.

Comment 4

[Kartikaya Gupta (email:[e-mail])]

It's not really clear to me how you intend to do the replay. Can you explain more what you're planning? Generally OMTA needs to be driven by vsync. Do we have that hooked up sufficiently at this point to make it work? Well it interfere with the work nical is doing to separate the threads? How do you plan on updating the commands to advance the animation? That part seems very complicated to me since you need to unpack the commands, find the transform that needs updating, and then repack the commands. We also eventually want to move the rust display list building into the content process so RecvDPEnd will disappear entirely. The equivalent will live in the content process, so we can't do OMTA there.

I think we need to hash out this plan much more before going ahead with any implementation.

Comment 5

[Peter Chang[:pchang]]

The following is the logic how to replay WRCmds and it is similar with current gecko flow(except WRCmds). Since I save the whole commands based on last transaction, it won't be hard to update the transform for the related DisplayItem.

+WebRenderBridgeParent::ApplyAnimationIfHas(TimeStamp aCurrentFrame)
+{
+  bool wantNextFrame = false;

+  if (animationexist) {
+    // TODO sample animation for div who has animation
+    // TODO update sample value for that div in WRCmds
+    ProcessWebrenderCommands(mWRCommands);
+    wantNextFrame = true;
+  }
+
+  return wantNextFrame;
+}
 void
 WebRenderBridgeParent::CompositeToTarget(gfx::DrawTarget* aTarget, const gfx::IntRect* aRect)
 {
   if (aTarget) {
     // XXX Add compositing to DrawTarget
     return;
   }
   if (!mWidget) {
@@ -622,17 +651,25 @@ WebRenderBridgeParent::CompositeToTarget(gfx::DrawTarget* aTarget, const gfx::In
     MOZ_ASSERT(mApi);
     // TODO(bug 1328602) With the RenderThread, calling SetRootStackingContext
     // should trigger the composition on the render thread.
     return;
   }
 
   TimeStamp start = TimeStamp::Now();
 
-  mCompositor->SetCompositionTime(TimeStamp::Now());
+  // TODO adding test mode
+  //  TimeStamp time = mIsTesting ? mTestTime :
+  TimeStamp time = mCompositorScheduler->GetLastComposeTime();
+  bool requestNextFrame = ApplyAnimationIfHas(time);
+
+  if (requestNextFrame) {
+    ScheduleComposition();
+  }

Comment 6

[Peter Chang[:pchang]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #4)
> It's not really clear to me how you intend to do the replay. Can you explain
> more what you're planning? Generally OMTA needs to be driven by vsync. Do we
> have that hooked up sufficiently at this point to make it work? Well it
> interfere with the work nical is doing to separate the threads? 
Sure, I will check nical's design to see any changes I need to do.
>How do you
> plan on updating the commands to advance the animation? That part seems very
> complicated to me since you need to unpack the commands, find the transform
> that needs updating, and then repack the commands. We also eventually want
> to move the rust display list building into the content process so RecvDPEnd
> will disappear entirely. The equivalent will live in the content process, so
> we can't do OMTA there.
Even we move the display list building to content process, I think we could still find a way to record or save the display list cmds in content process or WebRender(if it supports). Then we could move the logic in comment 5 to new place. Later I will send out my plan to gfx-dev-tech to have more discussion about OMTA.
> 
> I think we need to hash out this plan much more before going ahead with any
> implementation.

Comment 7

[Peter Chang[:pchang]]

Instead of replaying WRCmds, I'm checking the possibility to have new WebRender API to update the animation value for related div(mapped to a display Item in WebRender). If has, then we don't need to reply whole WRCmds.

Comment 8

[Peter Chang[:pchang]]

Right now we decide to use animatedProperty approach for OMTA. So we don't need this bug.