Investigate "main" ping payload size changes on Nightly

RESOLVED FIXED

Status

()

enhancement
P2
normal
RESOLVED FIXED
Last year
Last year

People

(Reporter: chutten, Assigned: chutten)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

Assignee

Description

Last year
According to the "budget" dashboard[1] we saw size decreases to "main" pings on Nightly:

1) 2018-05-24 a drop of 15kb
2) 2018-06-15 a drop of 120kb
3) 2018-06-16 a drop of about 10kb (though variable)

We should investigate these changes to 1) Explain to our satisfaction why these happened and 2) To ensure we aren't losing data.

[1]: https://metrics.services.mozilla.com/telemetry-budget-dashboard/
Assignee

Comment 1

Last year
3) is probably bug 1468809. If so it should manifest in having decreases in the keyedHistograms portions of the pings.
Assignee

Updated

Last year
Summary: Investigate payload size changes → Investigate "main" ping payload size changes on Nightly
2) could be bug 1458337. Removing `MAIN_THREAD_RUNNABLE_MS` and some other histograms.
(In reply to Chris H-C :chutten from comment #2)
> 3)
> https://hg.mozilla.org/mozilla-central/
> pushloghtml?fromchange=5b2a9b683f20&tochange=9a847d79eb8e

Note that this changelog contains bug 1468809 (dropping expired keyed histograms).
Assignee

Comment 5

Last year
I've taken a look at mean payload segment size for Nightly "main" pings submitted since May 22: https://dbc-caf9527b-e073.cloud.databricks.com/#notebook/18120

The tl;dr is the plot you see as an attachment. It is illustrative, but is using a non-representative sample, so it's not the -best-.

But it pretty clearly shows that the following segments decreased for the events:

1) content, keyedHistograms, histograms, extension
2) content, keyedHistograms, extension, gpu
3) content, keyedHistograms, extension

This supports the theory that 2) is due to bug 1458337 and 3) is due to bug 1468809

1) is still confusing to me, though. It is across keyed and unkeyed histograms, which suggests a broad spectrum change. If it were from another point in time we could ascribe it to the expiry of measures as versions roll on, however the version didn't change anywhere near May 24 (Nightly rolled over to 62 on the 7th of May). There were no changes to Histograms.json that landed on either of the builds that day, so it would have to be a behavioural change.

However, it is not as clearly defined on this graph as I'd like. There's a little up-and-down, and it may have come from the build just before the 24 and this is a continued decline.

Either way, it is far less interesting than whatever happened June 20th to the Extension process. Wow.
Assignee

Comment 6

Last year
I'm going to call this investigation complete. We have solid theories on the biggest changes, and I've now created a method for easy analysis of a population's ping's payload's segments' sizes over time.

(( If we're only interested in payload sizes in total, there's a dedicated dataset for that: telemetry_payload_size_parquet ))

As a final attachment, see here the final plot, optimized for legibility. I dove into the subprocess sections and used a broader palette, taking this as an excuse to remember how to use matplotlib.
Assignee

Updated

Last year
Status: ASSIGNED → RESOLVED
Closed: Last year
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.