362814 - nagios build files check - follow app version bumps automatically

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[Nick Thomas [:nthomas] (UTC+12)]

This is a follow up to bug 342055, where the lists of monitored files where moved to CVS. That's been a great help, but having to manually bump the version in those config files each time the app version changes shouldn't be necessary. We should just pull that information from CVS, perhaps along the lines of bug 342055 comment #9 (and later).

Comment 1

[Justin Fitzhugh]

Assigning to Rob for monitoring script changes.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

aravind/justdave: could you please attach the current nagios scripts here, to help avoid regressions. Thanks!

Comment 3

[Dave Miller [:justdave]]

grabbing this so I don't lose it, I'll reassign back to rhelmer once the files are posted.

Comment 4

[Dave Miller [:justdave]]

The current nagios scripts are in cvs as of a month or so ago, so nothing on me anymore...

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Taking, as I'm tinkering with this problem.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Not actively working on this, putting it back in the pool. If anyone wants to pick it up then I have some code which might be of use.

Comment 7

[Chris Cooper [:coop] (he/him)]

Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Everything for monitoring the freshness of nightly builds is in CVS at 
  mozilla/tools/tinderbox-configs/monitoring

Example control file - Firefox_mozilla-central.txt
On stage, nagios runs 'check_nightly_builds2 Firefox_mozilla-central'

Everytime we do a release the application version for nightlies gets bumped, eg from 3.5.7pre to 3.5.8pre, and we have to update the control files too. So we need a script that can look up the version from cvs/hg at the time the last nightly ran, and use that when checking the latest-$branch directory.

Comment 9

[Rail Aliiev [:rail]]

Nick,

Thanks for your tips.

After my first review I can see 2 ways to implement this feature:

1. Rewrite check_nightly_builds so that it can handle URL templates, similar to yours in bug 342055.

2. Write another script which parses templates and generates *.txt files with hard coded URLs, and commit it to CVS if there is any change.

Second one doesn't require any change to the current scripts, but should be run as a cronjob and have CVS commit access.

From my POV first one would be more elegant.

Comments are welcome.

Comment 10

[Rail Aliiev [:rail]]

Attachment: Proposed patch

Nick, could you review the following patch, please?

Some comments.

1. Instead of playing around CVS/HG and their parameters I use mxr and hg-web to get the latest versions.

2. Seamonkey_trunk.txt shouldn't mix versions in one file and should be split into 2 files. TBD.

3. Thunderbird_trunk.txt is outdated IMO, since 1.9.1 is not trunk AFAIK. TBD.

4. In normal situations wget doesn't time out, to be tested.

5. Old style files with hard coded version can be used as well.

6. I don't have dos2unix (RH specific) on my machine, replaced with "tr -d '\r'"

Comment 11

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 429704 [details] [diff] [review]
Proposed patch

r- for a few things we can improve but this is a great start.

>Index: Firefox_electrolysis.txt
>+# VERSION_URL http://hg.mozilla.org/projects/electrolysis/raw-file/tip/browser/config/version.txt

Use 'default' instead of 'tip' in all these hg urls. tip can end up on the relbranch if that's the latest change, while nightlies always come from default. Lets also take this opportunity to update these configs for files we're missing, so for Electrolysis we'd want to add firefox-@VER@.en-US.linux-x86_64.tar.bz2. We can ignore the xpi, tests archives and the new .txt files.

I rotted your patch by updating this file and a couple of others, sorry about that.

>Index: Firefox_mozilla1.9.0.txt
>===================================================================
>+# VERSION_URL http://mxr.mozilla.org/mozilla/source/browser/config/version.txt?raw=1

The seamonkey 'module' is slightly more specific to Fx3.0 than mozilla (yes really!) and is updated more frequently. Not super awesome to be depending on mxr but good enough for branches that are going to die.

>Index: XULRunner_mozilla1.9.0.txt

This whole file can go away. The nagios check is being removed in bug 544678.

>Index: check_nightly_builds2
>+    VER=$(wget -q --timeout=7 -O- "$VERSION_URL" | grep -v ^# | grep -v ^$ | tail -n 1)

grep -P '^\d\.\d.*' might be a more direct way of testing for what we want ?

>+    if [ -z $VER ]; then
>+        echo "Cannot parse version information"
>+        exit 0
>+    fi

Perhaps we should exit 1 to be a warning state in this case. Would make tweaking the timeout easier when this is first set up. What do you think ?

Comment 12

[Rail Aliiev [:rail]]

Attachment: Proposed fix

Nick,

Thanks for your review and comments.

Take 2.

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 430242 [details] [diff] [review]
Proposed fix

Looks good, checked in to CVS:
http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=HEAD&branchtype=match&sortby=Date&date=explicit&mindate=2010-03-04+17%3A43&maxdate=2010-03-04+17%3A45&cvsroot=%2Fcvsroot