1198137 - Add some summary stats to tool/power/rapl's output

Status: RESOLVED FIXED

(Core :: General, defect)

Description

[Nicholas Nethercote [inactive]]

"mean" is most useful, then "min" and "max". The rest are moderately useful.

You could argue about doing this via post-processing via Python. I promise if
it gets any more complicated after this I will switch to doing that :)

Comment 1

[Nicholas Nethercote [inactive]]

Attachment: Add some summary stats to tool/power/rapl's output

Sample output:

>     total W = _pkg_ (cores + _gpu_ + other) + _ram_ W
> #01  6.38 W =  2.80 ( 0.20 +  0.35 +  2.24) +  3.58 W
> #02  7.13 W =  3.46 ( 0.40 +  0.50 +  2.57) +  3.67 W
> #03 11.23 W =  7.01 ( 1.47 +  1.62 +  3.92) +  4.22 W
> #04 13.34 W =  8.75 ( 1.05 +  1.74 +  5.96) +  4.60 W
> #05 14.92 W = 10.13 ( 0.66 +  1.57 +  7.90) +  4.79 W
> #06  9.73 W =  5.85 ( 1.25 +  0.98 +  3.62) +  3.89 W
> #07 15.60 W = 10.71 ( 2.49 +  2.43 +  5.79) +  4.89 W
> #08 17.60 W = 12.54 ( 2.30 +  2.09 +  8.15) +  5.05 W
> #09  8.85 W =  4.85 ( 0.31 +  0.88 +  3.66) +  4.00 W
> #10 11.58 W =  7.36 ( 1.41 +  1.73 +  4.22) +  4.22 W
> 
> Distribution of 'total' values:
>             mean = 11.64 W
>          std dev =  3.53 W
>   0th percentile =  6.38 W (min)
>   5th percentile =  6.38 W
>  25th percentile =  8.85 W
>  50th percentile = 11.23 W
>  75th percentile = 14.92 W
>  95th percentile = 17.60 W
> 100th percentile = 17.60 W (max)

The moz.build change is a magic trick, courtesy of glandium, that is required
to stop things going haywire on Linux now that the STL is being used.

Comment 2

[Eric Rahm [:erahm]]

Comment on attachment 8652184 [details] [diff] [review]
Add some summary stats to tool/power/rapl's output

Review of attachment 8652184 [details] [diff] [review]:
-----------------------------------------------------------------

r=me. Just a few minor comments; mainly interested in printing the Watt and Time totals at the end.

::: tools/power/rapl.cpp
@@ +646,5 @@
>    char totalStr[kNumStrLen];
> +  double total_J = pkg_J + ram_J;
> +  NormalizeAndPrintAsWatts(totalStr, total_J);
> +
> +  gTotals_W.push_back(total_J / gSampleInterval_sec);

A comment here that W = J/T would be useful (I had to go look that up, it's been a while...), or just simple helper that this and NormalizeAndPrintAsWatts can use: WattsForThisInterval(total_J).

@@ +668,5 @@
> +  // Compute the mean.
> +  double sum = 0;
> +  for (auto iter = gTotals_W.begin(); iter != gTotals_W.end(); ++iter) {
> +    sum += *iter;
> +  }

You can just use |std::accumulate(gTotals_W.begin(), gTotals_W.end(), 0)| here.

@@ +680,5 @@
> +  // population size.
> +  //
> +  // This is different from the *sample* standard deviation, which divides by
> +  // |n - 1|, and would be appropriate if we were using a random sample of a
> +  // larger population.

Thanks for the comment!

@@ +686,5 @@
> +  for (auto iter = gTotals_W.begin(); iter != gTotals_W.end(); ++iter) {
> +    double deviation = (*iter - mean);
> +    sumOfSquaredDeviations += deviation * deviation;
> +  }
> +  double popStdDev = sqrt(sumOfSquaredDeviations / n);

Again you could use STL stuff here, roughly std::transform to get the deltas, then std::inner_product to sum the squares. Really either way is fine.

@@ +693,5 @@
> +  // method of determining percentiles, which is simplest to compute and which
> +  // chooses values from those that appear in the input set.
> +  std::sort(gTotals_W.begin(), gTotals_W.end());
> +
> +  printf("\n");

Printing the total watts and elapsed time would be useful.

Comment 3

[Nicholas Nethercote [inactive]]

> A comment here that W = J/T would be useful (I had to go look that up, it's
> been a while...), or just simple helper that this and
> NormalizeAndPrintAsWatts can use: WattsForThisInterval(total_J).

Good suggestion. It's interesting how people's biases affect their comments. I wrote the big comment about the standard deviation because my stats knowledge is weak, but I guess my physics is stronger because it didn't even occur to me to explain this :)


Thanks for the STL tips. I used accumulate() but didn't use transform()/inner_product() because I felt like that would actually be harder to read than the explicit loop.


> Printing the total watts and elapsed time would be useful.

Elapsed time is a good idea. I'm not sure what you mean by "total watts", though. Given that Watts is the unit for power, which is a rate, that doesn't even make sense. I could print the total energy (in Joules) but that's not useful because people think in Watts about this stuff.

Comment 4

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/c97794b3e3f5

Comment 5

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/66b72003a01d

Comment 6

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/c97794b3e3f5
https://hg.mozilla.org/mozilla-central/rev/66b72003a01d