559885 - Report on end-to-end time

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P4)

Description

[Chris AtLee [:catlee]]

Similar to how we have wait-time reports, we need end-to-end time reports for our various platforms and branches.

Comment 1

[Anamaria Stoica [:anamarias]]

Attachment: e2e json + other and Average Time per Builder Report

e2e UI:
- All reports main menu
- Sort tables: e2e Times Report by Least when_timestamp, Build Run table by start_time
- JSON for: e2e Times, Build run

e2e Other:
- changed all queries to return build requests with endtime <, and not <=
- replace from sqlalchemy import * with actual imports
- removed whitespaces
- grouped general code in model.util.py

Average Time per Builder Report:
- Main report for all builders. Display: buildername, build type, job type, avg duration, no build request, % of successful/warnings/failed, min duration, max duration
- Individual report for each builder: summary with everything above, plus all build requests that contributed to the result
- running and pending jobs were excluded
- the report is made for a timeframe, selectable through the date picker
- JSON for: Agv time/builder, Builder details

Comment 2

[Chris AtLee [:catlee]]

Comment on attachment 474739 [details] [diff] [review]
e2e json + other and Average Time per Builder Report

>+class BuilderTypeReport(object):
>+
>+    def __init__(self, buildername, starttime=None, endtime=None):
>+        self.starttime = starttime
>+        self.endtime = endtime
>+        self.buildername = buildername
>+
>+        self.platform = get_platform(buildername)
>+        self.build_type = get_build_type(buildername) # opt / debug
>+        self.job_type = get_job_type(buildername)     # build / unittest / talos
>+
>+        self.build_requests = []
>+
>+        self._d_min = NaN
>+        self._d_max = 0
>+        self._d_sum = 0
>+        self._total_br_results = {
>+            BuildRequest.no_result: 0,
>+            BuildRequest.success: 0,
>+            BuildRequest.warnings: 0,
>+            BuildRequest.failure: 0,
>+            BuildRequest.skipped: 0,
>+            BuildRequest.exception: 0,
>+            BuildRequest.retry: 0,
>+        }

This stuff is a bit weird.  Maybe we can move these up into buildapi/model/util.py as top-level variables?
(NO_RESULT, SUCCESS, ...) = range(-1, 6)

> class BuildRequest(object):
>-    st_pending = 0
>-    st_running = 1
>-    st_complete = 2
>-    st_cancelled = 3
>-    st_interrupted = 4
>-    st_misc = 5
>-    
>+    st_pending, st_running, st_complete, st_cancelled, st_interrupted, st_misc = range(6)

Same here.

>--- a/buildapi/model/util.py
>+++ b/buildapi/model/util.py
>@@ -1,9 +1,161 @@
>-import time
>+import time, re
>+
>+NaN = -1

NaN should be really not a number.  None is handy to use.

Comment 3

[Anamaria Stoica [:anamarias]]

Attachment: e2e json + other and Average Time per Builder Report + Comment 2

previous patch updated to reflect Comment 2

Comment 4

[Chris AtLee [:catlee]]

Anamaria finished this a while ago.