1601550 - Enforce disk quota on pending pings

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, enhancement, P1)

Description

[Alessio Placitelli [:Dexter]]

Glean does not currently check how much space is taken by pending pings, but we should, and should also have telemetry about how many times quota is hit. We should probably copy Firefox desktop mechanism as a starting point.

Comment 1

[Beatriz Rizental [:brizental]]

The Firefox mechanism not only has metrics to record the space taken by the pending pings directory, but also the time it took to calculate this space. I don't believe we need this time metric just yet for Glean, what do you think [:Dexter]?

I believe we could have only the metrics for

1. Size of the pending pings directory;
2. How many ping files were deleted due to extrapolating the size.

Now, the current Firefox mechanism records these numbers even if the quota has not been hit. I am not sure if that would be ideal for Glean too or if it should only record when the quota is hit.

Comment 2

[Alessio Placitelli [:Dexter]]

(In reply to Beatriz Rizental from comment #1)

The Firefox mechanism not only has metrics to record the space taken by the pending pings directory, but also the time it took to calculate this space. I don't believe we need this time metric just yet for Glean, what do you think [:Dexter]?

I agree, I don't think we need that time.

I believe we could have only the metrics for

1. Size of the pending pings directory;
2. How many ping files were deleted due to extrapolating the size.

These sounds about right.

Now, the current Firefox mechanism records these numbers even if the quota has not been hit.
I am not sure if that would be ideal for Glean too or if it should only record when the quota is hit.

I believe the same rationale makes sense for Glean. Ideally, most of the scans happen when quota is not hit (i.e. when glean starts).

Comment 3

[Beatriz Rizental [:brizental]]

Attachment: 1601550-data-review.md

Comment 4

[Chris H-C :chutten]

Comment on attachment 9156757 [details]
1601550-data-review.md

An individual must be named as responsible for a permanent data collection.

Also the metric description should contain information about when the data is collected. "The size of the pending pings directory" when will this be measured? In what units will it be reported?

Comment 5

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/glean/pull/1110

Comment 6

[Beatriz Rizental [:brizental]]

Attachment: 1601550-data-review.md

Comment 7

[Chris H-C :chutten]

Comment on attachment 9167640 [details]
1601550-data-review.md

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Bea Rizental is responsible.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---
Result: datareview+