Closed Bug 492406 Opened 12 years ago Closed 11 years ago

Figure out how to get SeaMonkey data on graphs server

Categories

(Release Engineering :: General, defect, P3)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: kairo, Unassigned)

References

Details

In bug 492221, we're doing a short-term fix to just disable SeaMonkey posting data to the graphs server, but alice and me agreed on IRC that we want to look into actually getting data and graphs up on the new server after the transition.

We need to configure the server to the the machines from the cb network post to it, and do eventual configuration on the SeaMonkey buildbots and the server to report the graphs accurately.

Currently, it's Codesighs (Linux, Mac) and the Trace-Malloc (Linux) numbers that we are generating now, basically as a side effect of using NightlyBuildFactory.
KaiRo, should this be assigned to you?
Ben, it probably ought to be assigned to alice and dependent on finishing the whole graphs server transition. I can't do anything on this before the server side isn't taken care of.
I'm going to put this into future for now. Once the graph server side is taken care off I guess we can come back to this.

After that, I think the biggest thing here is poking a hole in the firewall - I could be wrong though.
Component: Release Engineering → Release Engineering: Future
Depends on: 487329
Duplicate of this bug: 456279
Alice, do we have a chance to get this done now?
I don't think there's a firewall involved -- graphs.m.o is on the internet is it not?

There is, however, an ACL whitelist in apache for the cgi used to submit data.  Server Ops is able to update this whitelist... they'll need the list of IPs that need access.
Does it need to be specific IPs (would be all SeaMonkey build slaves, I guess) or can access for the whole community network be granted?
Assuming that's a subnet, I bet that's possible. IT would know better.
To get this going we're going to need:

- figure out how to securely allow access by these machines to the graph server reporting system
- list of the names of all the tests are going to be reporting
- list of all the machines names that will be reporting (so that they can be added to the graph server db)

We could start by attempting to get things reporting to graphs-stage, I have access there and can do the necessary db updates.

Look at http://hg.mozilla.org/graphs/file/bbb11e47c50d/sql/data.sql for how the machines/tests/os/etc are to be added to the graph server db so that I can be provided with working sql insert statements.
I'm trying my best to figure out the SQL, let's see if I get it right, as documentation is not overwhelming in this file...

insert into branches values (NULL,"SeaMonkey");
insert into branches values (NULL,"SeaMonkey2.0");

insert into machines values (NULL,6,0,NULL,"Linux_comm-1.9.1",1,unix_timestamp());
insert into machines values (NULL,6,0,NULL,"Linux_comm-1.9.1_leak_test",1,unix_timestamp());
insert into machines values (NULL,6,0,NULL,"Linux_comm-central-trunk",1,unix_timestamp());
insert into machines values (NULL,6,0,NULL,"Linux_comm-central-trunk_leak_test_leak_test",1,unix_timestamp());
insert into machines values (NULL,7,0,NULL,"OS_X_10.5_comm-1.9.1",1,unix_timestamp());
insert into machines values (NULL,7,0,NULL,"OS_X_10.5_comm-1.9.1_leak_test",1,unix_timestamp());
insert into machines values (NULL,7,0,NULL,"OS_X_10.5_comm-central-trunk",1,unix_timestamp());
insert into machines values (NULL,7,0,NULL,"OS_X_10.5_comm-central-trunk_leak_test",1,unix_timestamp());
insert into machines values (NULL,8,0,NULL,"WINNT_5.2_comm-1.9.1_leak_test",1,unix_timestamp());
insert into machines values (NULL,8,0,NULL,"WINNT_5.2_comm-central-trunk_leak_test",1,unix_timestamp());


It's just codesighs and leak tests, i.e. everything the build machines run. We don't have talos, but I think it would be nice to have graphs of the other things to be able to track and compare them.
I've applied the changes to the graphs-stage db.  So, now you just have to attempt to send data there and then we can shake out the issues.

If all goes well there then we can move things over to production graphs.
When I set the graph server to graphs-stage.mozilla.org, I now get the following:
"Warning: error when trying to post codesighs, trying again in 5 seconds"
(repeated up to at least 320 seconds, same with the leak test stuff).

Might the community subnet not have access to push data there?
You may need an IT bug to ensure that seamonkey builders have permission to post to graph server.  

Is there any more information in the logs?
Depends on: 544131
Filed bug 544131 - unfortunately those "error when trying to post xxx" warnings are all we get.
Yay, bug 544131 helped, it looks like a first data set has been posted to graphs-stage!
I'm still seeing one error, but that is really on the graph server side:

results not added, response: 
No machine_name called 'Linux_comm-central-trunk_leak_test' can be found
  File "/var/www/html/graphs2/server/pyfomatic/collect.py", line 255, in handleRequest
    metadata = MetaDataFromTalos(databaseCursor, databaseModule, inputStream)
  File "/var/www/html/graphs2/server/pyfomatic/collect.py", line 59, in __init__
    self.doDatabaseThings(databaseCursor)
  File "/var/www/html/graphs2/server/pyfomatic/collect.py", line 88, in doDatabaseThings
    raise DatabaseException("No machine_name called '%s' can be found" % self.machine_name)
(In reply to comment #10)
> insert into machines values
> (NULL,6,0,NULL,"Linux_comm-central-trunk_leak_test_leak_test",1,unix_timestamp());

Oops. that should be:
insert into machines values (NULL,6,0,NULL,"Linux_comm-central-trunk_leak_test",1,unix_timestamp());
OK, with that change, looks like all machines that currently run can post to graphs-stage, all trees are green. :)

Let's leave them running this way over the weekend, so that we get a bit of data up there and can see if things look right. If so, I'll re-post the SQL for the production server.
OK, things look good on graphs-stage, let' go for pushing this to the real graphs server!

Here's the correct SQL statements:

insert into branches values (NULL,"SeaMonkey");
insert into branches values (NULL,"SeaMonkey2.0");

insert into machines values
(NULL,6,0,NULL,"Linux_comm-1.9.1",1,unix_timestamp());
insert into machines values
(NULL,6,0,NULL,"Linux_comm-1.9.1_leak_test",1,unix_timestamp());
insert into machines values
(NULL,6,0,NULL,"Linux_comm-central-trunk",1,unix_timestamp());
insert into machines values
(NULL,6,0,NULL,"Linux_comm-central-trunk_leak_test",1,unix_timestamp());
insert into machines values
(NULL,7,0,NULL,"OS_X_10.5_comm-1.9.1",1,unix_timestamp());
insert into machines values
(NULL,7,0,NULL,"OS_X_10.5_comm-1.9.1_leak_test",1,unix_timestamp());
insert into machines values
(NULL,7,0,NULL,"OS_X_10.5_comm-central-trunk",1,unix_timestamp());
insert into machines values
(NULL,7,0,NULL,"OS_X_10.5_comm-central-trunk_leak_test",1,unix_timestamp());
insert into machines values
(NULL,8,0,NULL,"WINNT_5.2_comm-1.9.1_leak_test",1,unix_timestamp());
insert into machines values
(NULL,8,0,NULL,"WINNT_5.2_comm-central-trunk_leak_test",1,unix_timestamp());
Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.
Component: Release Engineering: Future → Release Engineering
Priority: -- → P3
Depends on: 547099
Switched SeaMonkey configuration to graphs.m.o server in http://hg.mozilla.org/build/buildbot-configs/rev/c75a480422d9 - this should be fixed now.

Thanks to Alice for your help here!
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Depends on: 619729
No longer depends on: 619729
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.