669137 - builds-4hr.js.gz skips builds that take over 4 hours; should group by endtime, not starttime

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[Markus Stange [:mstange]]

I think build-4hr.js.gz only contains builds that have *started* during the last 4 hours, not those that have actually *finished* in that period. As a result, builds that take over 4 hours to finish never show up in that file.

Symptoms:
The new buildbot-based TBPL, currently running at http://tbpl.swatinem.de/?usebuildbot=1 , gets build data by importing build-4hr.js.gz every 5 minutes.
However, these three builds didn't make it into the database at tbpl.swatinem.de, but they're present in http://build.mozilla.org/builds/builds-2011-07-03.js.gz :

        {
            "builder_id": 34239, 
            "buildnumber": 52, 
            "endtime": 1309704378, 
            "id": 5377928, 
            "master_id": 34, 
            "properties": {
                "appVersion": "7.0a1", 
                "basedir": "/builds/slave/m-cen-lnx64", 
                "branch": "mozilla-central", 
                "builddir": "m-cen-lnx64", 
                "buildername": "Linux x86-64 mozilla-central build", 
                "buildid": "20110703025806", 

                "revision": "b7f03b37cf0c4453473f4b13a4d5858916688c35", 

            }, 
            "reason": "scheduler", 
            "request_ids": [
                4374461
            ], 
            "requesttime": 1309687087, 
            "result": 0, 
            "slave_id": 241, 
            "starttime": 1309687109
        },

        {
            "builder_id": 34452, 
            "buildnumber": 35, 
            "endtime": 1309686019, 
            "id": 5375083, 
            "master_id": 33, 
            "properties": {
                "appVersion": "7.0a1", 
                "basedir": "/builds/slave/tm-osx64", 
                "branch": "tracemonkey", 
                "builddir": "tm-osx64", 
                "buildername": "OS X 10.6.2 tracemonkey build", 
                "buildid": "20110702223516", 

                "revision": "d8e967b8afc86f86b26dbfea6596ae1c0a13d58e", 

            }, 
            "reason": "scheduler", 
            "request_ids": [
                4373200
            ], 
            "requesttime": 1309671317, 
            "result": 0, 
            "slave_id": 633, 
            "starttime": 1309671342
        }, 

        {
            "builder_id": 36174, 
            "buildnumber": 8, 
            "endtime": 1309703420, 
            "id": 5377809, 
            "master_id": 34, 
            "properties": {
                "appVersion": "7.0a1", 
                "basedir": "/builds/slave/tm-osx64-ntly", 
                "branch": "tracemonkey", 
                "builddir": "tm-osx64-ntly", 
                "buildername": "OS X 10.6.2 tracemonkey nightly", 
                "buildid": "20110703033538", 

                "revision": "d8e967b8afc86f86b26dbfea6596ae1c0a13d58e", 

            }, 
            "reason": "The Nightly scheduler named 'tracemonkey nightly' triggered this build", 
            "request_ids": [
                4374687
            ], 
            "requesttime": 1309689338, 
            "result": 0, 
            "slave_id": 646, 
            "starttime": 1309689339
        }, 

Their durations (endtime - starttime) are 4:47:49, 4:04:37 and 3:54:41, respectively.

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

Yes, the query is
    q = session.query(Build).filter(
            and_(Build.starttime >= starttime, Build.starttime <= endtime))

IIRC TBPL is using build-4hr.js.gz for completed builds only, so could we switch that to
    q = session.query(Build).filter(
            and_(Build.endtime >= starttime, Build.endtime <= endtime))
.. ?

Comment 2

[Markus Stange [:mstange]]

Sure, that should work. TBPL gets running builds from builds-running.js.

Where is the file you've quoted this from? I tried to look for the bug myself, but I didn't even know which repo to check :)

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

http://hg.mozilla.org/users/catlee_mozilla.com/bb_db/file/734c96b3a032/reporter.py, although Chris has some local changes where this is running which rearrange build_report() a lot.

Comment 4

[Chris AtLee [:catlee]]

(In reply to comment #1)
> Yes, the query is
>     q = session.query(Build).filter(
>             and_(Build.starttime >= starttime, Build.starttime <= endtime))
> 
> IIRC TBPL is using build-4hr.js.gz for completed builds only, so could we
> switch that to
>     q = session.query(Build).filter(
>             and_(Build.endtime >= starttime, Build.endtime <= endtime))
> .. ?

This should be fine. My initial thought was that it would miss in-progress builds, but those rarely show up in these reports.

Comment 5

[Markus Stange [:mstange]]

(In reply to comment #4)
> My initial thought was that it would miss in-progress
> builds, but those rarely show up in these reports.

In what cases would they show up at all?

Comment 6

[Chris AtLee [:catlee]]

They show up if we do a 'reconfig' on one of the masters. In this case they have a result of null, and no end time (or an end time of jan 1 1970). This actually resulted in duplicate information in the db for the same build, which should be fixed soon (bug 668425).