1424564 - Add more Main Summary fields before next backfill

Status: RESOLVED FIXED

(Data Platform and Tools :: General, enhancement, P1)

Description

[Su-Young Hong]

Please backfill main summary to include profile_reset_date. 

From 20170701 to current, if possible.

Comment 1

["Saptarshi Guha[:joy]"]

Hello,
Could we have backfilled to at least 57 (Nov 14th) or preferably 56 the following:

logical cores
physical cores
total memory: "MEMORY_TOTAL" (anything else?)
a snapshot of the amount of total memory used: (e.g. MEMORY_RESIDENT_FAST,"MEMORY_DISTRIBUTION_AMONG_CONTENT"," "MEMORY_VSIZE","MEMORY_VSIZE_MAX_CONTIGUOUS","MEMORY_HEAP_ALLOCATED","MEMORY_HEAP_COMMITTED_UNUSED")

I'm not sure one aggregates histograms in main_summary but a geometric average might suffice.

Comment 2

[Su-Young Hong]

Also, can we include:

session_id

need to be able to aggregate by session in order to produce any session level metrics (as opposed to subsession level) 

- Su

Comment 3

[Mark Reid [:mreid]]

See Bug 1438927 for the backfill that will populate these fields.

Comment 4

[bugzilla]

A few others which would be useful for events:
- info/sessionStartDate
- info/sessionLength

I took a quick scan through the main ping docs, and here are a few other fields potentially of interest:
- environment/update/*
- environment/sandbox/effectiveContentProcessLevel
- environment/gpuProcess/status
- payload/info/revision

Comment 5

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/telemetry-batch-view/pull/382

Comment 6

[Anthony Miyaguchi [:amiyaguchi]]

This is a list of the fields that were added in this commit.

{'cpu_cores',
 'cpu_count',
 'cpu_family',
 'cpu_l2_cache_kb',
 'cpu_l3_cache_kb',
 'cpu_model',
 'cpu_speed_mhz',
 'cpu_stepping',
 'cpu_vendor',
 'gfx_features_advanced_layers_status',
 'gfx_features_d2d_status',
 'gfx_features_d3d11_status',
 'gfx_features_gpu_process_status',
 'histogram_content_memory_distribution_among_content',
 'histogram_content_memory_heap_allocated',
 'histogram_content_memory_heap_committed_unused',
 'histogram_content_memory_resident_fast',
 'histogram_content_memory_total',
 'histogram_content_memory_unique',
 'histogram_content_memory_vsize',
 'histogram_content_memory_vsize_max_contiguous',
 'histogram_gpu_memory_distribution_among_content',
 'histogram_gpu_memory_heap_allocated',
 'histogram_gpu_memory_heap_committed_unused',
 'histogram_gpu_memory_resident_fast',
 'histogram_gpu_memory_total',
 'histogram_gpu_memory_unique',
 'histogram_gpu_memory_vsize',
 'histogram_gpu_memory_vsize_max_contiguous',
 'histogram_parent_memory_distribution_among_content',
 'histogram_parent_memory_heap_allocated',
 'histogram_parent_memory_heap_committed_unused',
 'histogram_parent_memory_resident_fast',
 'histogram_parent_memory_total',
 'histogram_parent_memory_unique',
 'histogram_parent_memory_vsize',
 'histogram_parent_memory_vsize_max_contiguous',
 'previous_build_id',
 'previous_session_id',
 'previous_subsession_id',
 'sandbox_effective_content_process_level',
 'session_id',
 'session_length',
 'session_start_date',
 'subsession_id',
 'update_auto_download',
 'update_channel',
 'update_enabled'}

Comment 7

[bugzilla]

Great, I think that covers everything mentioned in this bug. Thanks Anthony!

Comment 8

[Anthony Miyaguchi [:amiyaguchi]]

Attachment: Add three important performance histograms. #386