Closed Bug 1392476 Opened 7 years ago Closed 7 years ago

Add dropframe telemetry for WebVR

Categories

(Core :: WebVR, enhancement)

Product:
Core
Component:
WebVR
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla57
Tracking Flags:
Tracking Status
firefox57 --- fixed

People

(Reporter: daoshengmu, Assigned: daoshengmu)

References

Details

Attachments

(1 file)

Bug 1392476 - Add dropframe telemetry for WebVR;
59 bytes, text/x-review-board-request
kip
: review+
francois
: review+
Details 
We need to detect the dropframe situation when using WebVR. [HMD id, resolution, ms/frame]
Depends on: 1353080
Assignee: nobody → dmu
The current patch will submit telemetry when detecting dropped frames from VR compositors.
Comment on attachment 8901758 [details]
Bug 1392476 - Add dropframe telemetry for WebVR;

Kip, do you think there is other things we need to add?
Attachment #8901758 - Flags: feedback?(kgilbert)
(In reply to Daosheng Mu[:daoshengmu] from comment #4)
> Comment on attachment 8901758 [details]
> Bug 1392476 - Add dropframe telemetry for WebVR;
> 
> Kip, do you think there is other things we need to add?

Using the lower level API's from Oculus and OpenVR is a good idea.

There are a couple of problems with this implementation; however:

- This approach may skew the results towards oversampling users that are dropping more frames as there will be no telemetry for users that didn't drop any frames.

- If the user starts dropping frames, the telemetry might slow down the rendering even further resulting in a "performance vortex" effect.

Perhaps we could instead return a metric like "avg frames dropped / second" and report it once every minute of activity and when the VR sessions are ended?  We should report the same telemetry for users that aren't dropping frames so the results are not skewed.
There are lots of other statistics reported by Oculus and OpenVR that we should track the same way; however, that could be done in a separate bug :-)
Comment on attachment 8901758 [details]
Bug 1392476 - Add dropframe telemetry for WebVR;

https://reviewboard.mozilla.org/r/173196/#review178790

Please see feedback left in bug comment.  This code is good, but we need to report something like "dropped frames per second" and fixed intervals to avoid skewing the results towards lower performance users and to prevent possible "performance vortex" behavior caused by telemetry overhead.
Attachment #8901758 - Flags: review-
(In reply to Daosheng Mu[:daoshengmu] from comment #8)
> Comment on attachment 8901758 [details]
> Bug 1392476 - Add dropframe telemetry for WebVR;
> 
> Review request updated; see interdiff:
> https://reviewboard.mozilla.org/r/173196/diff/1-2/

In the current wip patch, using AppDroppedFrameCount from Oculus Frame timing API would detect the whole APP dropped frames, and it's count is accumulated when the application is running even though I have exited from the present mode. The avg. dropped frame count per sec is around 50 when running it in the debug environment.

For the OpenVR part, using m_nNumDroppedFrames from OpenVR Frame timing API, I always get 0 even though I have blocked the application for debugging. It seems to have something wrong for using this API. Need some assistance about this.
(In reply to Daosheng Mu[:daoshengmu] from comment #9)
> (In reply to Daosheng Mu[:daoshengmu] from comment #8)
> > Comment on attachment 8901758 [details]
> > Bug 1392476 - Add dropframe telemetry for WebVR;
> > 
> > Review request updated; see interdiff:
> > https://reviewboard.mozilla.org/r/173196/diff/1-2/
> 
> In the current wip patch, using AppDroppedFrameCount from Oculus Frame
> timing API would detect the whole APP dropped frames, and it's count is
> accumulated when the application is running even though I have exited from
> the present mode. The avg. dropped frame count per sec is around 50 when
> running it in the debug environment.
> 
> For the OpenVR part, using m_nNumDroppedFrames from OpenVR Frame timing API,
> I always get 0 even though I have blocked the application for debugging. It
> seems to have something wrong for using this API. Need some assistance about
> this.

Perhaps m_nNumDroppedFrames is only incrementing when the asynchronous reprojection fails to submit a frame to the HMD panel, but not when content fails to keep up.

Do you see m_nNumReprojectedFrames incrementing?  That may be a better indication for the purpose of knowing when content is slow.
:kip . You are right. I should use m_nNumReprojectedFrames instead of m_nNumDroppedFrames.

For the OpenVR part, using m_nNumReprojectedFrames, the avg. dropped frame count per sec is around 22 when running it in the debug environment.
Comment on attachment 8901758 [details]
Bug 1392476 - Add dropframe telemetry for WebVR;

https://reviewboard.mozilla.org/r/173196/#review181476

::: gfx/vr/gfxVROculus.cpp:263
(Diff revision 4)
> -                                   mPresentationStart);
> +                          duration.ToMilliseconds());
> +
> +    if (mTelemetry.IsLastDroppedFrameValid()) {
> +      ovrPerfStats perfStats;
> +      if (ovr_GetPerfStats(mSession, &perfStats) == ovrSuccess) {
> +        if (perfStats.FrameStatsCount) {

Perhaps we should also ensure duration.ToSeconds() is non-zero in the if() here to avoid any division-by-zero in the event of a session that is immediately stopped after starting?
Attachment #8901758 - Flags: review?(kgilbert) → review+
Comment on attachment 8901758 [details]
Bug 1392476 - Add dropframe telemetry for WebVR;

https://reviewboard.mozilla.org/r/173196/#review181478

r=me with a sanity check to prevent a division-by-zero.
Attachment #8901758 - Flags: review?(francois)
:francois, could you help me review the telemetry part? Thanks.
Comment on attachment 8901758 [details]
Bug 1392476 - Add dropframe telemetry for WebVR;

https://reviewboard.mozilla.org/r/173196/#review181608

::: toolkit/components/telemetry/Histograms.json:13572
(Diff revision 4)
> +    "expires_in_version": "never",
> +    "kind": "linear",
> +    "high": 200,
> +    "n_buckets": 100,
> +    "releaseChannelCollection": "opt-out",
> +    "description": "The number of dropped frames per sec in Oculus."

When does the measurement take place?

Do you send the number of dropped frames so far every X minutes, or do you send just the total number of dropped when you exit webvr?
Attachment #8901758 - Flags: review?(francois)
Comment on attachment 8901758 [details]
Bug 1392476 - Add dropframe telemetry for WebVR;

https://reviewboard.mozilla.org/r/173196/#review181608

> When does the measurement take place?
> 
> Do you send the number of dropped frames so far every X minutes, or do you send just the total number of dropped when you exit webvr?

I send the measurement result when users exit from WebVR content.
Comment on attachment 8901758 [details]
Bug 1392476 - Add dropframe telemetry for WebVR;

https://reviewboard.mozilla.org/r/173196/#review181934

datareview+ with the expanded description as per my comment below

::: toolkit/components/telemetry/Histograms.json:13572
(Diff revision 6)
> +    "expires_in_version": "never",
> +    "kind": "linear",
> +    "high": 200,
> +    "n_buckets": 100,
> +    "releaseChannelCollection": "opt-out",
> +    "description": "The number of dropped frames per sec in Oculus."

Let's expand the description to include when the probe is sent:

"The average number of dropped frames per sec in Oculus, sent when user exits WebVR content."

I assume you're sending the average since this could vary over a WebVR "session".

::: toolkit/components/telemetry/Histograms.json:13583
(Diff revision 6)
> +    "expires_in_version": "never",
> +    "kind": "linear",
> +    "high": 200,
> +    "n_buckets": 100,
> +    "releaseChannelCollection": "opt-out",
> +    "description": "The number of dropped frames per sec in OpenVR."

Same thing here.
Attachment #8901758 - Flags: review?(francois) → review+
Comment on attachment 8901758 [details]
Bug 1392476 - Add dropframe telemetry for WebVR;

https://reviewboard.mozilla.org/r/173196/#review181934

> Let's expand the description to include when the probe is sent:
> 
> "The average number of dropped frames per sec in Oculus, sent when user exits WebVR content."
> 
> I assume you're sending the average since this could vary over a WebVR "session".

Sounds good! thanks.
Pushed by dmu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d206c52ee4f7
Add dropframe telemetry for WebVR; r=francois,kip
https://hg.mozilla.org/mozilla-central/rev/d206c52ee4f7
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox57: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla57
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: