637594 - No new test results reported

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

The last push to receive any results was

http://hg.mozilla.org/mozilla-central/rev/410519307e63

Since then there has been three pushes

http://hg.mozilla.org/mozilla-central/rev/a7b9ace8b3a3
http://hg.mozilla.org/mozilla-central/rev/372eca6704c2
http://hg.mozilla.org/mozilla-central/rev/ec8a059bea58

Neither of which is showing any results. Only pending and running jobs are being shown, but it appears that as they complete they are dropping off the tbpl page rather than reporting a result.

I've closed the tree since we're currently not receiving any test results.

Comment 1

[Reed Loden [:reed]]

I think it's fair to say that the tinderbox upgrade did not go very well... I doubt it's the patch itself... Maybe sendmail didn't restart correctly? Maybe one of the symlinks isn't right? Maybe some permission issue?

Anyway, this is an IT issue.

Comment 2

[Shyam Mani [:fox2mike]]

Bah. Looking.

Comment 3

[Shyam Mani [:fox2mike]]

Reverted the changes :

[root@dm-webtools02 tinderbox-src]# cvs up -dP -D "27 Feb 2011 00:00"
? cvs-update.log
cvs update: Updating .
M Makefile
P admintree.cgi
M doadmin.cgi
M header.pl
cvs update: Updating examples
[root@dm-webtools02 tinderbox-src]# make
/sbin/service sendmail stop
Shutting down sm-client:                                   [  OK  ]
Shutting down sendmail:                                    [  OK  ]
mkdir -p /var/www/iscsi/webtools/tinderbox/examples
set -e ; \
        for I in Backwards.pm cvsquery.pl ep_mac.pl ep_unix.pl ep_unittest.pl ep_windows.pl get_line.pl header.pl showbuilds.pl tbglobals.pl Empty.html faq.html index.html examples/buildit.config examples/mozilla-unix.pl examples/mozilla-windows.pl  addnote.cgi admintree.cgi buildwho.pl clean.pl checksetup.pl doadmin.cgi scrape.pl showbuilds.cgi showlog.cgi warnings.pl  handlemail.pl processbuild.pl ; do \
          echo Installing $I && \
          sed -e s#/usr/bin/perl#/usr/bin/perl#g \
              -e s#@CVSROOT@#/cvsroot#g \
              -e s#@DATA_DIR@#/var/www/iscsi/webtools/tinderbox/data#g \
              -e s#@TINDERBOX_DIR@#/var/www/iscsi/webtools/tinderbox#g \
              -e s#@SETUID_PATH@#'/usr/bin:/bin'#g \
              $I > /var/www/iscsi/webtools/tinderbox/$I && \
           chmod 644 /var/www/iscsi/webtools/tinderbox/$I; \
        done
Installing Backwards.pm
Installing cvsquery.pl
Installing ep_mac.pl
Installing ep_unix.pl
Installing ep_unittest.pl
Installing ep_windows.pl
Installing get_line.pl
Installing header.pl
Installing showbuilds.pl
Installing tbglobals.pl
Installing Empty.html
Installing faq.html
Installing index.html
Installing examples/buildit.config
Installing examples/mozilla-unix.pl
Installing examples/mozilla-windows.pl
Installing addnote.cgi
Installing admintree.cgi
Installing buildwho.pl
Installing clean.pl
Installing checksetup.pl
Installing doadmin.cgi
Installing scrape.pl
Installing showbuilds.cgi
Installing showlog.cgi
Installing warnings.pl
Installing handlemail.pl
Installing processbuild.pl
set -e ; \
        for I in 1afi003r.gif star.gif ; do \
          echo Installing $I && \
          cp $I /var/www/iscsi/webtools/tinderbox && \
          chmod 644 /var/www/iscsi/webtools/tinderbox/$I; \
        done
Installing 1afi003r.gif
Installing star.gif
mkdir -p /var/www/iscsi/webtools/tinderbox/data/bad
#chown -R tinderbox:tinderbox /var/www/iscsi/webtools/tinderbox
#chgrp -R tinderbox /var/www/iscsi/webtools/tinderbox
chmod 755 /var/www/iscsi/webtools/tinderbox
chmod 770 /var/www/iscsi/webtools/tinderbox/examples
mkdir -p /var/www/iscsi/webtools/tinderbox/data/bad

Not replacing existing queryconfig.pl
Check queryconfig.pl in build directory for new features
#chown -R tinderbox:tinderbox /var/www/iscsi/webtools/tinderbox/data
#chgrp -R tinderbox /var/www/iscsi/webtools/tinderbox/data
chmod 770 /var/www/iscsi/webtools/tinderbox/data
chmod 770 /var/www/iscsi/webtools/tinderbox/data/bad
chmod 755 /var/www/iscsi/webtools/tinderbox/addnote.cgi /var/www/iscsi/webtools/tinderbox/admintree.cgi /var/www/iscsi/webtools/tinderbox/buildwho.pl /var/www/iscsi/webtools/tinderbox/clean.pl /var/www/iscsi/webtools/tinderbox/checksetup.pl /var/www/iscsi/webtools/tinderbox/doadmin.cgi /var/www/iscsi/webtools/tinderbox/scrape.pl /var/www/iscsi/webtools/tinderbox/showbuilds.cgi /var/www/iscsi/webtools/tinderbox/showlog.cgi /var/www/iscsi/webtools/tinderbox/warnings.pl
chmod 4755 /var/www/iscsi/webtools/tinderbox/handlemail.pl /var/www/iscsi/webtools/tinderbox/processbuild.pl
/sbin/service sendmail start
Starting sendmail:                                         [  OK  ]
Starting sm-client:                                        [  OK  ]

Comment 4

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Could you reopen the tree (and close this bug) if you think things should have been returned to normal?

Comment 5

[Shyam Mani [:fox2mike]]

Reed made some more changes, tinderbox is running again now.

Comment 6

[Reed Loden [:reed]]

For the record, the problem was because my bustage fix to the patch in bug 545825 was never landed and apparently was only locally-applied on stage, which is why stage worked fine, but prod fell-over when fox2mike did a cvs up. I just landed attachment 503917 [details] [diff] [review], fox2mike updated, and that fixed the issue.