619729 - [x86-64] "graph server post results complete failed" caused by "No machine_name called '...' can be found"

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P1)

Description

[Serge Gautherie (:sgautherie)]

Before (some buildbot updates), these boxes didn't run this command.
Now they do, but fail.

http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1292514884.1292517285.18390.gz&fulltext=1
Linux x86-64 comm-central-trunk build on 2010/12/16 07:54:44
{
graph server post results complete failed

No machine_name called 'Linux_x86-64_comm-central-trunk' can be found
}

http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1292514556.1292517834.20492.gz&fulltext=1
OS X 10.6 comm-central-trunk build on 2010/12/16 07:49:16
{
graph server post results complete failed

No machine_name called 'OS_X_10.6_comm-central-trunk' can be found
}

NB: I think Firefox had an issue like this one, when they recently +/- shortened build names...

Comment 1

[Robert Kaiser]

We also shortened directory names, where's their bug and solution?

Comment 2

[Serge Gautherie (:sgautherie)]

(In reply to comment #1)
> We also shortened directory names, where's their bug and solution?

I was thinking about bug 614254.

Comment 3

[Robert Kaiser]

(In reply to comment #2)
> (In reply to comment #1)
> > We also shortened directory names, where's their bug and solution?
> 
> I was thinking about bug 614254.

I don't see any mention of graph server problems there. And yes, this has been applied to our side by their original patch.

Comment 4

[Serge Gautherie (:sgautherie)]

(In reply to comment #3)
> I don't see any mention of graph server problems there. And yes, this has been
> applied to our side by their original patch.

That was the only hint I had :-|

{
Executing: ['python', '/builds/slave/comm-cen-trunk-lnx64/tools/buildfarm/utils/graph_server_post.py', '--server', 'graphs.mozilla.org', '--selector', '/server/collect.cgi', '--branch', 'SeaMonkey', '--buildid', '20101217053344', '--sourcestamp', 'c6c5290dc2ae', '--resultsname', 'Linux_x86-64_comm-central-trunk', '--properties-file', 'properties.json', '--timestamp', '1292593836']
}
I was assuming this bug was somehow related to the renaming:
'comm-cen-trunk-lnx64' and 'Linux_x86-64_comm-central-trunk'.

Ftr,
http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1292572586.1292576055.22301.gz&fulltext=1
Linux comm-central-trunk build on 2010/12/16 23:56:26
{
graph server post results complete

Executing: ['python', '/builds/slave/comm-cen-trunk-lnx/tools/buildfarm/utils/graph_server_post.py', '--server', 'graphs.mozilla.org', '--selector', '/server/collect.cgi', '--branch', 'SeaMonkey', '--buildid', '20101216235626', '--sourcestamp', '85faf50deb16', '--resultsname', 'Linux_comm-central-trunk', '--properties-file', 'properties.json', '--timestamp', '1292572586']
}
'comm-cen-trunk-lnx' and 'Linux_comm-central-trunk'.

***

(In reply to comment #0)
> Before (some buildbot updates), these boxes didn't run this command.

Or could it simply be that our '*x86-64*' entries don't exist yet on the graph server?

NB: In any case, you should better ask graph server people than me.

Comment 5

[Robert Kaiser]

As you somewhat mentioned, I suspect that there are just no entries (yet) in the graph server for the linux64 and mac64 boxes, we need to get those created, will need to look up the SQL needed for this in an old bug we had about posting to the graph server.
I guess this failed silently with buildbot 0.7, while with 0.8 we now see the actual errors as purple builds.

Comment 6

[Robert Kaiser]

Alice (or anyone else who can do this), we'd need the following SQL additions to graph server:

insert into machines values (NULL,17,0,NULL,"OS_X_10.6_comm-central-trunk",1,unix_timestamp());
insert into machines values (NULL,17,0,NULL,"OS_X_10.6_comm-central-trunk_leak_test",1,unix_timestamp());
insert into machines values (NULL,18,0,NULL,"Linux_x86-64_comm-1.9.1",1,unix_timestamp());
insert into machines values (NULL,18,0,NULL,"Linux_x86-64_comm-central-trunk",1,unix_timestamp());
insert into machines values (NULL,18,0,NULL,"Linux_x86-64_comm-central-trunk_leak_test",1,unix_timestamp());


(If you wonder, we don't run debug/leaktest for Linux64 yet and don't have plans to on 1.9.1, but we have plans to turn that on for trunk once we have the machines to do so.)


BTW, I didn't find the other comm-* entries (from bug 492406) in http://hg.mozilla.org/graphs/file/tip/sql/data.sql though the SeaMonkey and SeaMonkey2.0 tree entries are there, is there any specific reason for that?
We'll have some more additions to do once we open a 2.0 branch, so I'd like to make sure we find out the best way for you to do so before we actually do.

Comment 7

[alice nodelman [:alice] [:anode]]

To get the inserts run first I'll need a patch for data.sql so that it can remain current.  Once that gets r+ed then we file an IT bug to get them to run the updates against the production graph server db.

Comment 8

[Chris Cooper [:coop] (he/him)]

(In reply to comment #7)
> To get the inserts run first I'll need a patch for data.sql so that it can
> remain current.  Once that gets r+ed then we file an IT bug to get them to run
> the updates against the production graph server db.

Who is on the hook for this? KaiRo? Serge?

Comment 9

[Robert Kaiser]

(In reply to comment #8)
> Who is on the hook for this? KaiRo? Serge?

Should be me (but not sure how much time I'll have this week). I guess the patch for data.sql should include all those machines rows we had added to the server in bug 492406 but which never made that file?

Comment 10

[Serge Gautherie (:sgautherie)]

(In reply to comment #6)

Ftr,

The 3 ones that fail currently:
*Linux_x86-64_comm-1.9.1
*Linux_x86-64_comm-central-trunk
*OS_X_10.6_comm-central-trunk

The one that is expected to fail, if only it didn't break earlier on bug 563643.
*OS_X_10.6_comm-central-trunk_leak_test

The one that is future:
*Linux_x86-64_comm-central-trunk_leak_test

> BTW, I didn't find the other comm-* entries (from bug 492406) in

Indeed, unexpected as this feature is working for them.
But let's sort that out in another bug.
(Yet, thanks for mentioning it!)

Comment 11

[Serge Gautherie (:sgautherie)]

Attachment: (Av1) Insert x86-64 comm-* builders

Comment 12

[Robert Kaiser]

Comment on attachment 499250 [details] [diff] [review]
(Av1) Insert x86-64 comm-* builders

It's easy to fix all right here, so please let's do that unless we hear otherwise. And the 1.9.2 crap should please stay out of this, we'll never have those for SeaMonkey and we don't need placeholder comments.

Comment 13

[Robert Kaiser]

Attachment: add all SeaMonkey builders

Here's a patch to data.sql to add all the SeaMonkey builder, I hope it's correct this way.

Comment 14

[alice nodelman [:alice] [:anode]]

Comment on attachment 499290 [details] [diff] [review]
add all SeaMonkey builders

Looks good to me.  Be sure that it lands in the 1.0 graph server branch.

Comment 15

[Robert Kaiser]

Thanks, pushed as http://hg.mozilla.org/graphs/rev/7dc58c855700 - filing the bug in a moment to get the new machines actually added.

Comment 16

[Robert Kaiser]

If anything, that bug might depend on the initial bug that added graph server support for us, but FOR SURE not the other way round. Even the hacky and ugly "regression" argument for reverse dependencies doesn't apply here as there's no regression anywhere in sight. Fixing something now that we missed then but didn't in any way disturb how things actually work is no reason for this dependency either and far from a regression or something like that.

Comment 17

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Anything left to do here?

Comment 18

[Aki Sasaki (not active)]

This looks done; kick me if I'm wrong.