1206808 - Telemetry for time-to-answer for webrtc

Status: NEW

(Core :: WebRTC, defect, P3)

Description

[Randell Jesup [:jesup] (needinfo me)]

Attachment: Collect telemetry for answer time in WebRTC/Loop

We'd like to collect time-to-answer information about WebRTC use (loop and non-loop).

It's arguable what the best metric for "answer time" is here, since that depends on what the app is doing.  I chose (as a 1st-order approximation for "simple" apps) setRemoteDescription->SetLocalDescription time.  This likely is wrong for cases involving re-invites (and thus we may want to gate it on current call state), or also switch to createAnswer() instead of setLocalDescription.

Comment 1

[Byron Campen [:bwc]]

I'm not at all clear on what we're trying to measure here. What do we mean by "answer time"?

Comment 2

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Byron Campen [:bwc] from comment #1)
> I'm not at all clear on what we're trying to measure here. What do we mean
> by "answer time"?

Well, this was from the doc we did at Whistler (as "answer time").  I'm taking as meaning (roughly) time from asking the user until there's a positive answer to join and we've done that.  Perhaps this *isn't* that useful to us (platform), though maybe it's interesting to Loop and UI or product team people.

That said, there are a number of things we could calculate from the timecard data that would be "interesting".  Measurement from createAnswer -> SetLocalDescription, or createAnswer -> media flowing, etc. Some of these may be useful in terms of recording time from "Answer" to seeing video/hearing audio, or time from Answer to sending audio/video.  Those may require more info to collect...

I have a simple patch for calculating and logging deltas from the timecard data (though we'd need it to be captured when signaling logging isn't on).

Comment 3

[Byron Campen [:bwc]]

(In reply to Randell Jesup [:jesup] from comment #2)
> (In reply to Byron Campen [:bwc] from comment #1)
> > I'm not at all clear on what we're trying to measure here. What do we mean
> > by "answer time"?
> 
> Well, this was from the doc we did at Whistler (as "answer time").  I'm
> taking as meaning (roughly) time from asking the user until there's a
> positive answer to join and we've done that.  Perhaps this *isn't* that
> useful to us (platform), though maybe it's interesting to Loop and UI or
> product team people.

   That's going to be very dependent on the particulars of the webrtc service in question. There is no way to approximate this without that kind of context. Most services don't even have anything like an "answerer"; you just load a link and you're in a room (and generally, the new joiner is the one that sends the offer). If we just want to keep track of how long it takes someone to answer a hello call, I suspect it makes sense to put that telemetry in the hello code, since building assumptions into platform code about how Hello functions seems like a bad idea to me.

> That said, there are a number of things we could calculate from the timecard
> data that would be "interesting".  Measurement from createAnswer ->
> SetLocalDescription, or createAnswer -> media flowing, etc. Some of these
> may be useful in terms of recording time from "Answer" to seeing
> video/hearing audio, or time from Answer to sending audio/video.  Those may
> require more info to collect...
> 
> I have a simple patch for calculating and logging deltas from the timecard
> data (though we'd need it to be captured when signaling logging isn't on).

   Measuring the delay between SLD and SRD is something I've thought about before, but I was unable to think of any way to actually use this data. There are too many things that can increase this, most of them benign.

Comment 4

[Bulk Bug Changes for mreavy's org]

Mass change P2->P3 to align with new Mozilla triage process.