745903 - add new test definitions to graph server database

Status: RESOLVED FIXED

(Data & BI Services Team :: DB: MySQL, task)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

we have new graph server sql definitions we need to add to graphs.mozilla.org and graphs.allizom.org from bug 745500.  This is the SQL to execute:
insert into tests values (NULL,"tprovider","Robocop Database Benchmark",1,1,NULL);
insert into tests values (NULL,"tcheck2","Robocop Checkerboarding Real User Benchmark",1,1,NULL);

Comment 1

[Armen [:armenzg]]

Can this please be done in the next hour or two?

Comment 2

[Sheeri Cabral [:sheeri]]

doing this now.

Comment 3

[Sheeri Cabral [:sheeri]]

done on production:

mysql> insert into tests values (NULL,"tprovider","Robocop Database Benchmark",1,1,NULL);
Query OK, 1 row affected (0.00 sec)

mysql> insert into tests values (NULL,"tcheck2","Robocop Checkerboarding Real User Benchmark",1,1,NULL);
Query OK, 1 row affected (0.00 sec)

Comment 4

[Sheeri Cabral [:sheeri]]

done on staging:

mysql> use graphs_stage2
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> insert into tests values (NULL,"tprovider","Robocop Database Benchmark",1,1,NULL);
Query OK, 1 row affected (0.00 sec)

mysql> insert into tests values (NULL,"tcheck2","Robocop Checkerboarding Real User Benchmark",1,1,NULL);
Query OK, 1 row affected (0.00 sec)

Comment 5

[Armen [:armenzg]]

Thanks a lot sheeri :)

Comment 6

[Armen [:armenzg]]

Can you please verify this for graphs.allizom.org? (current staging).

FTR we don't use graphs-stage.mozilla.org anymore (old staging)

Comment 7

[Armen [:armenzg]]

This works:
http://graphs.mozilla.org/graph.html#tests=[[200,11,20],[201,11,20]]&sel=none&displayrange=7&datatype=running

but this does not:
http://graphs.allizom.org/graph.html#tests=[[200,11,20],[201,11,20]]&sel=none&displayrange=7&datatype=running

Comment 8

[Sheeri Cabral [:sheeri]]

This is a problem that I was fixing - replication was broken. replication is currently catching up on the slave, I'll let you know when it's up to date.

Comment 9

[Sheeri Cabral [:sheeri]]

slave is up to date, but both URLs are broken now. :(

Comment 10

[Sheeri Cabral [:sheeri]]

Passing back to armenzg - I have no clue how to verify the fix.

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

the second link in comment 7 is broken.  I don't believe this is implemented.

Comment 12

[Sheeri Cabral [:sheeri]]

Well, I just double-checked and the data is in the database, and replication is fine. The error I'm seeing is:

Error

Could not download test run data from server

Exception: undefined

---------
Is it possible there's a connection error or something?

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

:rhelmer, any ideas on why graphs.allizom.org gives us an error on this link:
http://graphs.allizom.org/graph.html#tests=[[200,11,20],[201,11,20]]&sel=none&displayrange=7&datatype=running

Comment 14

[Robert Helmer [:rhelmer]]

From looking at http://graphs.allizom.org I would think that the nightly auto-snapshot from prod is not working, probably since the datacenter move.

(In reply to Sheeri Cabral [:sheeri] from comment #12)
> Well, I just double-checked and the data is in the database, and replication
> is fine. The error I'm seeing is:

Sheeri, which DB did you check? The allizom.org URL is staging, and looks like it's not importing from prod nightly anymore.

Comment 15

[Sheeri Cabral [:sheeri]]

:rhelmer, I am checking the graphs_stage2 db on stage1.db.scl3.mozilla.com.

The original scope of this bug was doing some inserts, and those were what I confirmed.

I'm totally OK with broadening the scope of this bug to "graphs staging is no longer importing from prod nightly" but this is the first I've heard of it, and if there are details on how that's all done that'd be great, so I can work on debugging....so feel free to change the title of this bug and give more info, or make a new bug and assign ownership to me.....

(I will also check crontabs and such).

Comment 16

[Sheeri Cabral [:sheeri]]

Looks like there's an import script that imports the latest input_mozilla_org from:
 /data/backup-drop/a01/mysql/input_mozilla_com/

And that stopped 4/9. 

The import script for sumo stopped working 2/22/2012, as per the files here: /data/backup-drop/sumo01/mysql/support_mozilla_com

and for the graphs data, 
/data/backup-drop/b01/mysql/graphs_mozilla_org_new

which also stopped 4/9. 

I will work on fixing these, it's just where the backups can/are copying to.

Would it help to manually copy over a graphs db so you'll have fresh info, or at this point does it not matter if it takes a day to catch up?

Comment 17

[Armen [:armenzg]]

What is the status of this? thanks for looking into this.

Comment 18

[Sheeri Cabral [:sheeri]]

Armen - I'm still working on fixing this copy - but I did offer to copy over a graphs db so you'll have fresh info to import.

Comment 19

[Sheeri Cabral [:sheeri]]

The files are there, I'm waiting for the netflow to be opened up.

Comment 20

[Sheeri Cabral [:sheeri]]

I just made the copy script and config for it:

[[e-mail]1 bin]# ./push_db_files.sh  stage1.db.scl3.mozilla.com

And I made bug 758051 for the netflow.

Comment 21

[Sheeri Cabral [:sheeri]]

netflow is open, but backups aren't happening nightly, even though they seem to work via commandline. Will debug.

Comment 22

[Sheeri Cabral [:sheeri]]

resolving, backups seem to be working.