1281795 - Add content processs histograms to the main ping payload

Status: RESOLVED DUPLICATE of bug 1218576

(Toolkit :: Telemetry, defect, P3)

Description

[Georg Fritzsche [:gfritzsche]]

With bug 1218576 in place, we need to put the histogram data from the child processes in the "main" ping payload.

Comment 1

[Georg Fritzsche [:gfritzsche]]

Chris raised the point that we probably have more child processes coming up (compositor per bug 1264543, possibly sandboxed addons, etc.), so that we should probably use a map structure to accommodate that.

E.g. one possible format:

> "payload": {
>  ...
>  "childHistograms": {
>    "content": {
>      ... histograms
>    },
>    "compositor: {
>      ... histograms
>    },
>  },
>  "childKeyedHistograms": {
>    ... similar
>  },
>  "childScalars": { // coming later, not now.
>    ...

Alernatively:
> "payload": {
>   ...
>   "childData": {
>     "histograms": {
>       "content": {
>         ...
>       },
>       ...
>     },
>     "keyedHistograms": {
>       ...
>     },
>     "scalars": { // Later, not now.
>       ...

I'd love to have the "main"/"parent" process histograms in there too, but i think that is not feasible due to the pipeline and analysis code depending on payload.histograms/keyedHistograms.

Comment 2

[Georg Fritzsche [:gfritzsche]]

Opinions on comment 1?

Comment 3

[Mark Reid [:mreid]]

I'd like to suggest a third possibility:
> "payload": {
>   ...
>   "processes": {
>     "content": {
>       "histograms": {...},
>       "keyedHistograms": {...},
>       "scalars": {...}
>     },
>     "compositor": {
>       "histograms": {...},
>       "keyedHistograms": {...},
>       "scalars": {...}
>     },
>     ... eventually, "parent" too? ...,
>     ...
>   }
>   ...
> }

This has a couple of main advantages.

Each section has the same structure, so ideally the processing code can be simple/reusable. 

Also, it allows for the possibility to move standard "parent" measurements into the same structure in the future if we decide it's worthwhile (it doesn't assume "child" for things).

If we wanted to introduce the "parent" section right away, we could even put scalars there for starters.

Comment 4

[Chris H-C :chutten]

I second mreid's suggestion. It's a little bit nested, but it's the clearest in intent.

I wonder if special processes will need special structures beyond the shared hgrams/keyed/scalars triumvirate (threadHangs? slowSQL?)

Comment 5

[Roberto Agostino Vitillo (:rvitillo)]

+1 on Mark's proposal.

Comment 6

[Georg Fritzsche [:gfritzsche]]

Great, lets go with this then.
We should definitely start off with scalars in "parent" right away (per bug 1282091).

Comment 7

[Georg Fritzsche [:gfritzsche]]

The new format approach also opens up possibilities for other use-cases:
E.g. for GoFaster / system addons / etc. we could add additional sections that have the same format but possibly different handling/semantics server-side.

Comment 8

[Georg Fritzsche [:gfritzsche]]

This is happening as part of bug 1218576 now.