455628 - AMO stats db failing

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Justin Scott [:fligtar]]

This was in the cron mail yesterday too, but I assumed it was related to CDNetworks changes happening at the time.

[amo-developers] Cron <root@dm-stats01>	/usr/local/bin/addons_stats_downloads.sh

PHP Warning:  mysql_connect(): Can't connect to MySQL server on '10.2.70.104' (4) in /mnt/netapp/nfs/stats/amo_stats/bin/database.class.php on line 64
Could not connect: PHP Warning:  mysql_connect(): Can't connect to MySQL server on '10.2.70.104' (4) in /mnt/netapp/nfs/stats/amo_stats/bin/database.class.php on line 64
Could not connect: PHP Warning:  mysql_connect(): Can't connect to MySQL server on '10.2.70.104' (4) in /mnt/netapp/nfs/stats/amo_stats/bin/database.class.php on line 64
Could not connect:

Comment 1

[Justin Scott [:fligtar]]

Please manually run the script when it is fixed and send the output to amo-developers at mozilla.org.

The download counter can be run at any time as it parses every log and knows what logs it has and hasn't parsed. Need it to make up for yesterday and today.

Comment 2

[Mark Smith [:xb95]]

Script needs to use the new slaves, that IP is no longer valid.  Your new target is tm-amo01-slave02 or tm-amo01-slave03, either works.

Can I go ahead and update the script or is this something you need to do?

Comment 3

[Justin Scott [:fligtar]]

You'll have to because I can't :)

The db settings are in site/app/config/config.php, which is probably

/mnt/netapp/nfs/stats/amo_stats/site/app/config/config.php

Comment 4

[Mark Smith [:xb95]]

It's running now, seems to be doing just fine now on the new databases.  I'll send out the output when it finishes.

Comment 5

[Michael Morgan [:morgamic]]

Thanks guys.

Comment 6

[Mark Smith [:xb95]]

It's still going, but it's hitting the databases pretty hard.  Replication is falling behind on the slaves because of:

    UPDATE download_counts SET count=count+1 WHERE addon_id=? AND date=?

Suggestion:

    UPDATE download_counts SET count=count+100 WHERE addon_id=? AND date=?

At least it finally explains the graphs!  :)  I always wondered why AMO has these weird spikes in the evenings (and you can see no spike last night because it didn't run):

    http://10.2.10.5/cacti/graph.php?action=zoom&local_graph_id=1750&rra_id=2

Comment 7

[Mark Smith [:xb95]]

This was done, results mailed out.