Publish Build Faster metrics

RESOLVED FIXED

Status

RESOLVED FIXED
7 years ago
5 years ago

People

(Reporter: catlee, Assigned: catlee)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [buildfaster:p1])

(Assignee)

Description

7 years ago
We need some simple metrics for how long builds / tests / waits are taking.

http://arewebuildingfastyet.com ?

Comment 1

7 years ago
Chris,
We are pulling data out of orange factor right now on specific slow tests in mochitest-plain.  I have :samliu and :harth working on building a dashboard for it.  So if you let :samliu know how to grab the data out of your database, he can include the specific buildbot setup/teardown times and include those in the dashboard.

So let him know how to get to your data and some thoughts on how you want it displayed and he can get started.

Comment 2

7 years ago
Would there be anyway of getting this information along with the buildslave?  I would love to be able to identify a Tegra that is starting to act-up.
(Assignee)

Comment 3

7 years ago
(In reply to comment #1)
> Chris,
> We are pulling data out of orange factor right now on specific slow tests in
> mochitest-plain.  I have :samliu and :harth working on building a dashboard
> for it.  So if you let :samliu know how to grab the data out of your
> database, he can include the specific buildbot setup/teardown times and
> include those in the dashboard.
> 
> So let him know how to get to your data and some thoughts on how you want it
> displayed and he can get started.

We store per-step information in our statusdb, I have a snapshot here:
http://people.mozilla.org/~catlee/statusdb-2011-02-01.sql.bz2

It can be a bit challenging to get at exactly the data you want though :)

Comment 4

7 years ago
Hi Chris,

A few questions:
1. How often are these snapshots made / is it automated / where do they end up?
2. I was thinking about how to extract the data, do you happen to have ideas on how I can do that efficiently? Last I was informed I would be getting everything in json but I don't think this is the case?

Thanks for your help!
(Assignee)

Comment 5

7 years ago
(In reply to comment #4)
> Hi Chris,
> 
> A few questions:
> 1. How often are these snapshots made / is it automated / where do they end
> up?

They're made manually right now. Depending on what data you need, I'm sure we can get you r/o access to the mysql database.

> 2. I was thinking about how to extract the data, do you happen to have ideas
> on how I can do that efficiently? Last I was informed I would be getting
> everything in json but I don't think this is the case?

The detail on how long individual steps take aren't available from our build json. Maybe they're in the ES database? They are in this statusdb snapshot.

Comment 6

7 years ago
Thanks Chris,

Hm so if I want to get data out of this snapshot do you have any recommendations on how that can be accomplished? My thought was importing into a local mysql and extracting from there, but that seems inefficient and I was wondering if there might be a better way?
(Assignee)

Comment 7

7 years ago
(In reply to comment #6)
> Thanks Chris,
> 
> Hm so if I want to get data out of this snapshot do you have any
> recommendations on how that can be accomplished? My thought was importing
> into a local mysql and extracting from there, but that seems inefficient and
> I was wondering if there might be a better way?

It really depends on what you're looking for...

Maybe if I exported data for setup/teardown, and wait times that would be helpful?

Comment 8

7 years ago
FTR
I copy/pasted all the info from this report:
https://build.mozilla.org/buildapi/reports/testruns

Comment 9

7 years ago
Nevermind, I loaded the database locally and will handle getting data. Thanks! :)
(Assignee)

Comment 10

7 years ago
Would a report like this help?

submitted_at,revision,os,jobtype,uid,results,wait_time,start_time,finish_time,elapsed
2011-07-07T03:08:40,b2622d5c857ab7e1cda6340e51227e1ab9efbb59,android,opt build,63e86b391675467091b52e02419b1973,0,0:00:00,2011-07-07T03:08:40,2011-07-07T04:49:27,1:40:47
2011-07-07T03:08:40,b2622d5c857ab7e1cda6340e51227e1ab9efbb59,linux32,opt build,63e86b391675467091b52e02419b1973,0,0:00:01,2011-07-07T03:08:41,2011-07-07T04:40:36,1:31:55
2011-07-07T03:08:40,b2622d5c857ab7e1cda6340e51227e1ab9efbb59,linux64,opt build,63e86b391675467091b52e02419b1973,0,0:00:01,2011-07-07T03:08:41,2011-07-07T04:46:44,1:38:03
2011-07-07T03:08:40,b2622d5c857ab7e1cda6340e51227e1ab9efbb59,osx10.6,opt build,63e86b391675467091b52e02419b1973,0,0:00:04,2011-07-07T03:08:44,2011-07-07T06:19:10,3:10:26
2011-07-07T03:08:40,b2622d5c857ab7e1cda6340e51227e1ab9efbb59,win32,opt build,63e86b391675467091b52e02419b1973,0,0:00:10,2011-07-07T03:08:50,2011-07-07T06:25:58,3:17:08
2011-07-07T03:08:55,3f9e716164ef3c1e26285ce1915d23475bbfb04c,linux32,debug build,8adc5b8486f141139e74ef5fc3dc98f3,0,0:00:02,2011-07-07T03:08:57,2011-07-07T03:55:18,0:46:21
2011-07-07T03:08:55,3f9e716164ef3c1e26285ce1915d23475bbfb04c,osx10.5,debug build,8adc5b8486f141139e74ef5fc3dc98f3,0,0:00:02,2011-07-07T03:08:57,2011-07-07T04:06:44,0:57:47
2011-07-07T03:08:55,3f9e716164ef3c1e26285ce1915d23475bbfb04c,linux32,opt build,8adc5b8486f141139e74ef5fc3dc98f3,0,0:00:03,2011-07-07T03:08:58,2011-07-07T04:23:12,1:14:14
2011-07-07T03:08:55,3f9e716164ef3c1e26285ce1915d23475bbfb04c,linux32,opt build,8adc5b8486f141139e74ef5fc3dc98f3,1,0:00:03,2011-07-07T03:08:58,2011-07-07T04:16:25,1:07:27
2011-07-07T03:08:55,3f9e716164ef3c1e26285ce1915d23475bbfb04c,linux64,opt build,8adc5b8486f141139e74ef5fc3dc98f3,0,0:00:03,2011-07-07T03:08:58,2011-07-07T07:27:16,4:18:18
2011-07-07T03:08:55,3f9e716164ef3c1e26285ce1915d23475bbfb04c,android,opt build,8adc5b8486f141139e74ef5fc3dc98f3,0,0:00:03,2011-07-07T03:08:58,2011-07-07T04:30:09,1:21:11
2011-07-07T03:08:55,3f9e716164ef3c1e26285ce1915d23475bbfb04c,linux64,debug build,8adc5b8486f141139e74ef5fc3dc98f3,0,0:00:03,2011-07-07T03:08:58,2011-07-07T05:06:16,1:57:18
2011-07-07T03:08:55,3f9e716164ef3c1e26285ce1915d23475bbfb04c,osx10.6,opt build,8adc5b8486f141139e74ef5fc3dc98f3,0,0:00:03,2011-07-07T03:08:58,2011-07-07T06:02:14,2:53:16
2011-07-07T03:08:55,3f9e716164ef3c1e26285ce1915d23475bbfb04c,win32,opt build,8adc5b8486f141139e74ef5fc3dc98f3,0,0:00:03,2011-07-07T03:08:58,2011-07-07T06:06:13,2:57:15
2011-07-07T03:08:55,3f9e716164ef3c1e26285ce1915d23475bbfb04c,android,opt build,8adc5b8486f141139e74ef5fc3dc98f3,0,0:00:03,2011-07-07T03:08:58,2011-07-07T04:38:41,1:29:43
2011-07-07T03:08:55,3f9e716164ef3c1e26285ce1915d23475bbfb04c,win32,debug build,8adc5b8486f141139e74ef5fc3dc98f3,0,0:00:03,2011-07-07T03:08:58,2011-07-07T04:43:02,1:34:04
2011-07-07T03:08:55,3f9e716164ef3c1e26285ce1915d23475bbfb04c,osx10.6,debug build,8adc5b8486f141139e74ef5fc3dc98f3,0,0:00:04,2011-07-07T03:08:59,2011-07-07T04:49:40,1:40:41
2011-07-07T03:40:07,3f9e716164ef3c1e26285ce1915d23475bbfb04c,linux32,debug test,8adc5b8486f141139e74ef5fc3dc98f3,0,0:00:16,2011-07-07T03:40:23,2011-07-07T04:08:28,0:28:05

Comment 11

7 years ago
Oh yeah actually that would help, I was wondering in that statusdb snapshot, is it intentional that the request tables are all empty? I've been asked to find these metrics and haven't had much success:

    Bw - time waiting for builder (avg over all OSes)
    Ba - time active building (avg over all OSes)
    Tw - time waiting for test machine (avg over all OSes)
    S1 - setup time
    Ta - time running tests
    S2 - time teardown & reboot

Also, when this dashboard goes into production is it intended to query the statusdb directly?

The only thing I've managed to do with regards to this is getting elapsed times for builds in the steps table.
(Assignee)

Comment 12

7 years ago
Is the data here useful?

http://people.mozilla.org/~catlee/buildfaster.csv.gz

Comment 13

7 years ago
Yes :) So is there a web resource I can request from that will give me this reliably?

Thanks
Sam
(Assignee)

Comment 14

7 years ago
Let me know how the data looks, and then I can automate a daily dump of it.
(Assignee)

Comment 15

7 years ago
This will be uploaded to http://build.mozilla.org/builds/buildfaster.csv.gz every day. The data will include the past 60 days of data.
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.