Histogram data for custom metrics sent to server when metric unchanged since last transmission

RESOLVED WONTFIX

Status

--
minor
RESOLVED WONTFIX
3 years ago
a year ago

People

(Reporter: ihsiao, Assigned: thills)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

3 years ago
This bug is opened based on bug 12077992 comment #6 and comment #7.

###Steps###
Here I set Telemetry Report_interval to 4 minutes for the following scenario.

Time        | User Activity              | Expected Result                         
------------+----------------------------+----------------------------------------
15:00:00 PM | An user installed the test | The metrics will be collected and sent
     ~      | app, and is using it.      | to server every 4 minutes.(i.e. 
15:10:00 PM |                            | 15:04:00 PM and 15:08:00 PM)
------------+----------------------------+----------------------------------------
15:10:10 PM | The user deleted the test  |
            | app.                       |
------------+----------------------------+----------------------------------------
15:12:00 PM | N/A                        | (Report_interval arrived) The metrics 
            |                            | will be collected and sent to
            |                            | server.
------------+----------------------------+----------------------------------------
15:16:00 PM | N/A                        | (Report_interval arrived) The metrics 
            |                            | of the test app will NOT send to server anymore.

However the actual result is not the expected. After I deleted test app over 1 hour, I still can see the custom metrics of the test app existed in Telemetry payload, and the metrics' counts are all zero.
For example:
"addonHistograms": {
	"DEVTOOLS_HUD_CUSTOM_{APP_ID}/{Telemetry_name}":{...}
}

###Expected result###
The metrics won't send to server after deleted the test app.

###Actual result###
The metrics keep sending to server until restart device.
(Reporter)

Updated

3 years ago
QA Whiteboard: [COM=Telemetry]
Flags: needinfo?(rnicoletti)
Component: Gaia::System → Metrics
(Reporter)

Comment 1

3 years ago
Correct the typo in the description.
> This bug is opened based on "bug 1207992" comment #6 and comment #7.
See [1] for more information regarding this issue. For the reasons described in [1], IMO this is a low priority issue.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1207992#c7
Severity: normal → minor
Flags: needinfo?(rnicoletti)
Summary: Empty histogram data sent to server when metric not updated → Empty histogram data sent to server when metric unchanged since last transmission
Blocks: 1152000
Further investigation has shown that this issue, unlike what I described here [1], affects only custom metrics. Histogram data for engineering metrics (reflows, jank, startup time, etc.) will be empty, that is, will contain no data -- no zeroes, nothing -- when there hasn't been any data collected since the last time data was sent to the server. Here is an example of 'empty' data sent for engineering metrics and custom metrics [2]. This shows 

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1207992#c7

[2] Console(27640): Content JS LOG: [AdvancedTelemetry]{"type":"advancedtelemetry","id":"a08fa2d6-7b4f-46e7-9be9-2aa64d0049a9","creationDate":"2015-11-17T23:07:03.328Z","version":4,"application":{"architecture":"arm","buildId":"20151112122114","name":"FirefoxOS","version":"45.0a1","vendor":"Mozilla","platformVersion":"45.0a1","xpcomAbi":"arm-gcc3","channel":"default"},"clientId":"f9bee611-24cc-4daf-9757-332293911eee","payload":{"keyedHistograms":{"DEVTOOLS_HUD_REFLOW_DURATION":{"system":{"min":1,"max":1000,"histogram_type":0,"sum":7,"log_sum":3.465735912322998,"log_sum_squares":4.324077129364014,"ranges":[0,1,2,5,12,29,70,170,412,1000],"counts":[0,1,2,0,0,0,0,0,0,0]}},"DEVTOOLS_HUD_REFLOWS":{"system":{"min":1,"max":2,"histogram_type":4,"sum":3,"sum_squares_lo":3,"sum_squares_hi":0,"ranges":[0,1,2],"counts":[3,0,0]}},"DEVTOOLS_HUD_ERRORS":{},"DEVTOOLS_HUD_APP_STARTUP_TIME_NAVIGATIONLOADED":{},"DEVTOOLS_HUD_APP_STARTUP_TIME_NAVIGATIONINTERACTIVE":{},"DEVTOOLS_HUD_APP_MEMORY_NAVIGATIONLOADED_V2":{},"DEVTOOLS_HUD_APP_MEMORY_NAVIGATION_INTERACTIVE_V2":{},"DEVTOOLS_HUD_APP_STARTUP_TIME_VISUALLYLOADED":{},"DEVTOOLS_HUD_APP_STARTUP_TIME_CONTENTINTERACTIVE":{},"DEVTOOLS_HUD_APP_MEMORY_VISUALLYLOADED_V2":{},"DEVTOOLS_HUD_APP_MEMORY_CONTENTINTERACTIVE_V2":{},"DEVTOOLS_HUD_WARNINGS":{},"DEVTOOLS_HUD_APP_STARTUP_TIME_FULLYLOADED":{},"DEVTOOLS_HUD_APP_STARTUP_TIME_SCANEND":{},"DEVTOOLS_HUD_APP_MEMORY_FULLYLOADED_V2":{},"DEVTOOLS_HUD_APP_MEMORY_SCANEND_V2":{},"DEVTOOLS_HUD_APP_STARTUP_TIME_MEDIAENUMERATED":{},"DEVTOOLS_HUD_APP_MEMORY_MEDIAENUMERATED_V2":{}},"addonHistograms":{"DEVTOOLS_HUD_CUSTOM_VIDEO_SHOWPLAYERCOUNT":{"min":1,"max":2,"histogram_type":4,"sum":0,"sum_squares_lo":0,"sum_squares_hi":0,"ranges":[0,1,2],"counts":[0,0,0]}}}}
Summary: Empty histogram data sent to server when metric unchanged since last transmission → Histogram data sent to server when metric unchanged since last transmission
Following up on comment 3, the histogram data in [2] above shows how the histogram data for engineering metrics that have not changed since the last transmission is truly empty while histogram data for custom metrics that haven't changed gets sent as zeroes. I have changed the summary of this bug to reflect the new information.
Summary: Histogram data sent to server when metric unchanged since last transmission → Histogram data for custom metrics sent to server when metric unchanged since last transmission
Iris, can you confirm it is only the custom metrics that have this issue? By that I mean, are you seeing that we're not sending histogram data for engineering metrics that haven't changed (as shown in comment 3)?
Flags: needinfo?(ihsiao)
(Reporter)

Comment 6

3 years ago
Hi Russ,
Yes, what I'm seeing just like the histogram data [2] in comment 3. For engineering metrics, they are empty if that haven't changed since last transmission; but for custom metrics, they are all zeroes.
Flags: needinfo?(ihsiao)
Created attachment 8714810 [details] [review]
[gaia] tamarahills:bugfix/1223269-Remove-Histograms-With-No-Data > mozilla-b2g:master
(Assignee)

Comment 8

3 years ago
Comment on attachment 8714810 [details] [review]
[gaia] tamarahills:bugfix/1223269-Remove-Histograms-With-No-Data > mozilla-b2g:master

Hi Russ,

This removes any extraneous entries so we can save bytes on the payload.

Thanks,
-tamara
Attachment #8714810 - Flags: feedback?(rnicoletti)
(Assignee)

Updated

3 years ago
Assignee: nobody → thills
Status: NEW → ASSIGNED
Comment on attachment 8714810 [details] [review]
[gaia] tamarahills:bugfix/1223269-Remove-Histograms-With-No-Data > mozilla-b2g:master

Looks good to me, a couple of nits -- you can thank Jan :)
Attachment #8714810 - Flags: feedback?(rnicoletti) → feedback+
(Assignee)

Comment 10

3 years ago
Comment on attachment 8714810 [details] [review]
[gaia] tamarahills:bugfix/1223269-Remove-Histograms-With-No-Data > mozilla-b2g:master

Hi Russ,

I changed it to bailout using continue since it's a double loop, but also created a helper function since they are exactly the same anyways.

-tamara
Attachment #8714810 - Flags: feedback+ → feedback?(rnicoletti)
Comment on attachment 8714810 [details] [review]
[gaia] tamarahills:bugfix/1223269-Remove-Histograms-With-No-Data > mozilla-b2g:master

Nice refactoring, I left a couple of nits in the PR.
Attachment #8714810 - Flags: feedback?(rnicoletti) → feedback+

Comment 12

a year ago
Firefox OS is not being worked on
Status: ASSIGNED → RESOLVED
Last Resolved: a year ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.