903914 - GraphServer should display statistics (min/max/avg/stddev) for view and selection

Status: RESOLVED FIXED

(Webtools Graveyard :: Graph Server, defect)

Description

[Avi Halachmi (:avih)]

Attachment: graphs.m.o-stats.v1.patch

Attached is a client-side only patch for the graph server, which displays simple statistics for the visible/selected datapoints - just above the main plot.

Two or more visible graphs also work.

The values are pretty printed with few significant digits and x1000 units (K/M/G/T) to reduce clutter.

The statistics include:
- Number of points over which the stats is calculated.
- Min/Max/Average/Stddev/Stddev%
- If relevant - number of points which are invisible above/below the view and therefore excluded from the calculation (only for the view. I don't display this data for the selection stats to reduce clutter).

The stats is displayed just above the plot itself - I was lucky to be able to place the stats div inline within #plot-box, and it appears just above #plot, without any special positioning (just some hardcoded left-alignment to align with the plot's left edge).

Issue #1 - The code seems to relate to several y ranges at the plot, but no matter how many graphs I displayed together, the |yaxes| array always had at most one range. So for simplicity I hardcoded it to use the first y-range, and so far I haven't seen it errors.

Note that I haven't set up a local graphs server, but rather downloaded the relevant files which are currently deployed at production, edited them, and overlayed them on a live graphs pages. But after cloning hg.m.o/graphs, it seems that the files I touched are the same at the repo, so I'm optimistic for it to work in first try.

Comment 1

[Avi Halachmi (:avih)]

Attachment: Screenshot of the stats in action

Comment 2

[Avi Halachmi (:avih)]

Comment on attachment 788753 [details] [diff] [review]
graphs.m.o-stats.v1.patch

Review of attachment 788753 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/common.js
@@ +559,5 @@
> +        // At 'runs', the data is runs[i].data[j] (.t/.v)
> +
> +        // Collect all visible (mean) values in range, possibly also out-of-Y-range for Y-report.
> +        var means = [];
> +        var allY = [];

Hmm.. allY should be a plain counter rather than an array since we only use this array's length later. This would improve performance. Though even without this change it's still pretty quick, <200ms for 10k visible points - on a slow system.

Comment 3

[Avi Halachmi (:avih)]

Is this still at the review queue? or was it decided that this change is unnecessary/badly-designed/some-other-reason/etc but the bug was not updated to reflect this decision?

Comment 4

[Robert Helmer [:rhelmer]]

(In reply to Avi Halachmi (:avih) from comment #3)
> Is this still at the review queue? or was it decided that this change is
> unnecessary/badly-designed/some-other-reason/etc but the bug was not updated
> to reflect this decision?

Wasn't set to me so I didn't see it, sorry! I just set it and will get to it soon.

Comment 5

[Robert Helmer [:rhelmer]]

Comment on attachment 788753 [details] [diff] [review]
graphs.m.o-stats.v1.patch

This looks fine to me, thanks!

Note that graphs.m.o is deprecated in favor of datazilla - https://datazilla.mozilla.org/ - it will probably be taken down in the next quarter or two.

Pretty sure datazilla has these features, they'd probably appreciate a patch too if not :)

Comment 6

[Robert Helmer [:rhelmer]]

https://hg.mozilla.org/graphs/rev/0a2940a70894

Comment 7

[Avi Halachmi (:avih)]

Thanks. I know datazilla should take over, but TBH grahserver currently still has use cases where it's more useful than datazilla (overviews, lighter, etc), and also the historic data wasn't imported (yet?) to datazilla.

I might still push a tiny part 2 for comment 2 for slightly improved performance.

Comment 8

[Robert Helmer [:rhelmer]]

(In reply to Avi Halachmi (:avih) from comment #7)
> Thanks. I know datazilla should take over, but TBH grahserver currently
> still has use cases where it's more useful than datazilla (overviews,
> lighter, etc), and also the historic data wasn't imported (yet?) to
> datazilla.

No worries, just wanted to make sure you were aware.

> I might still push a tiny part 2 for comment 2 for slightly improved
> performance.

WFM, thanks!

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

For the record datazilla will not be importing any historical data.  The data models do not match up and the views in datazilla will not be useful.  Please file bugs against Datazilla for features which you would find useful.

Comment 10

[Avi Halachmi (:avih)]

(In reply to Joel Maher (:jmaher) from comment #9)
> Please file bugs against Datazilla for features which you would find useful.

Is there some documentation for datazilla such that I'll try to think what's missing?

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

The docs for datazilla are here: http://datazilla.readthedocs.org/en/latest/.   Bugs can be filed here: https://bugzilla.mozilla.org/enter_bug.cgi?product=Datazilla&component=User%20interface.

Just keep in mind that Datazilla is providing a different view on the data and collecting all the data vs displaying summaries of the data.  So some differences are by design and others will be fixable!