1109625 - Review analyses

Status: RESOLVED FIXED

(Toolkit :: Telemetry, defect)

Description

[Roberto Agostino Vitillo (:rvitillo)]

We should have a more or less formal process to review telemetry analyses. It's fairly easy to make mistakes or wrong assumptions that can go unnoticed in a final plot and having at least another set of eyes reviewing an analysis can be very beneficial. 

We mostly already do it in a way or the other but we should have a set of best practices in place. For instance, right now a significant fraction of ClientIDs are missing from the submissions which could distort certain analyses. Peer reviews help also spreading that kind of knowledge. 

The first step is to use IPython or Beakle notebooks for our analyses as notebooks can tie code, documentation and plots together. 

I am not sure yet what's the best way to review a notebook, e.g. on Github, Bugzilla or somewhere else where we can add inline comments. Suggestions are welcome.

Comment 1

[Roberto Agostino Vitillo (:rvitillo)]

I setup IPython to work with Spark and Telemetry. Notebooks can be saved as gists and viewed on nbviewer.ipython.org, e.g.: http://nbviewer.ipython.org/gist/vitillo/e1813025e7d26d640c80.

I couldn't find any easy way to leave comments inline in a notebook so for now comments in the gist hosting the analysis will have to do.