1610813 - `first_run_date` is missing from some metrics pings

Status: RESOLVED WORKSFORME

(Data Platform and Tools :: Glean: SDK, defect, P3)

Description

[Michael Droettboom [:mdroettboom]]

4 out of 26733 ping exhibit this behavior.

For example:

{"client_info":{"android_sdk_version":"27","app_build":"20081832","app_channel":"release","app_display_version":"3.0.2","architecture":"armeabi-v7a","device_manufacturer":"Umx","device_model":"U683CL","os":"Android","os_version":"8.1.0","telemetry_sdk_build":"21.3.0"},"metrics":{"labeled_counter":{"gfx.content.frame_time.reason":{"missed_composite":3}}},"ping_info":{"end_time":"2020-01-22T04:17-08:00","ping_type":"metrics","seq":0,"start_time":"2020-01-22T04:17-08:00"}}

Comment 1

[Jan-Erik Rediger [:janerik]]

I noticed yesterday that some "prototype" pings from the fogotype were rejected due to missing a "first_run_date"

Comment 2

[Alessio Placitelli [:Dexter]]

(In reply to Michael Droettboom [:mdroettboom] from comment #0)

{"client_info":{"android_sdk_version":"27","app_build":"20081832","app_channel":"release","app_display_version":"3.0.2","architecture":"armeabi-v7a","device_manufacturer":"Umx","device_model":"U683CL","os":"Android","os_version":"8.1.0","telemetry_sdk_build":"21.3.0"},"metrics":{"labeled_counter":{"gfx.content.frame_time.reason":{"missed_composite":3}}},"ping_info":{"end_time":"2020-01-22T04:17-08:00","ping_type":"metrics","seq":0,"start_time":"2020-01-22T04:17-08:00"}}

@Mike, Mh, seq: 0 and app_display_version: 3.0.2? This also seems to be missing the client_id (unless you manually redacted it).

(In reply to Jan-Erik Rediger [:janerik] from comment #1)

I noticed yesterday that some "prototype" pings from the fogotype were rejected due to missing a "first_run_date"

@Jan-Erik, can this have to do with the temp directory problem in Firefox?

Comment 3

[Jan-Erik Rediger [:janerik]]

It shouldn't be because of the temp directory.
If all of client_info is missing, it's created, including the first_run_date.
If we disable upload, we clear out everything but first_run_date.

I also haven't looked any closer yet, it was more than 4 for the fogotype, but still such a low number that it was below the threshold (plus it was after only a day or 2 of data, so not yet conclusive anyway).
Keep in mind fogotype data is not yet validated and it might or might not be related.

Comment 4

[Alessio Placitelli [:Dexter]]

Let's come at this after fogotype validation.

Comment 5

[Alessio Placitelli [:Dexter]]

Were pings from comment 0 discarded at ingestion?

Because I can't find any missing field from this query in recent data.

Comment 6

[Jan-Erik Rediger [:janerik]]

first_run_date is a required field, thus if it is missing those will be rejected at ingestion.
I don't see any in the schema rejections across the past 7 days.
I see 1 for the fogotype across the past 7 days.