537840 - Setup a machine in the lab to run daily Mozmill and software update tests

Status: RESOLVED FIXED

(Testing Graveyard :: Mozmill, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As discussed in my Software Update session during our last All Hands we wanna have a machine in the lab which can perform daily Mozmill and Software Update runs on all platforms.

What we wanna have:
* Host should be running OS X 10.5 or 10.6
* 1st VMWare guest: Ubuntu 9.10
* 2nd VMWare guest: Windows XP
* 3rd VMWare guest: Windows Vista
* 4th VMWare guest: Windows 7

For the daily tests the Linux and one of the Windows machines (I would prefer XP for now) should be constantly run.

I will setup this machine once it has been delivered. Al, can you please file a bug for IT and mark it blocking this bug?

Comment 1

[Marcia Knous [:marcia]]

When these are in the lab and setup can we please add them to https://wiki.mozilla.org/MozillaQualityAssurance:QA_Machines#QA_Machines_Inventory?

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The machine has been setup by Al last week. We have 10.5 and 10.6 available on separate partitions. Further we have VM's for Ubuntu 9.10 and Windows (2000, XP, Vista, and Win7) which I have already prepared with everything we need to run Mozmill tests.

Working items which have to be done:
* Create shell scripts for cron tab execution
* Report to brasstacks (needs fixes to our design views first)

(In reply to comment #1)
> When these are in the lab and setup can we please add them to
> https://wiki.mozilla.org/MozillaQualityAssurance:QA_Machines#QA_Machines_Inventory?

Sure. I have added this machine now.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The shell script has been created and works for OS X. It needs some tweaks for Windows and eventually Linux too. But sending reports is supported. The first public results are located here:

http://brasstacks.mozilla.com/couchdb/mozmill/_design/reports/_list/summary/summary

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Python script for test-runs

Just to make sure everything is ok, I'd like to have a review from Clint.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

All machines have been setup to run daily tests. I will call it fixed. Follow-up work will happen on demand. At least we should improve the handling of Mozmill tests which hang.

Comment 6

[cmtalbert]

Comment on attachment 423799 [details]
Python script for test-runs


># For MozillaBuild we have to manually set the path to the Mercurial module
>if sys.platform == 'win32':
>    # There is a bug in Mercurial which blocks importing pyd files from an archive
>    #sys.path.append(r'c:\mozilla-build\hg\library.zip')
>    sys.path.append(r'c:\mozilla-build-libs\hg')
Are you sure this is working ^? In a standard mozilla-build you don't get a directory called mozilla-build-libs but I'm not familiar with your operating environment here.


>        # To run the script via crontab on Linux we have to set the display
>        if self.options.display:
>            os.putenv('DISPLAY', self.options.display)
Since this is only needed for linux, it'd probably make sense to put an "if (linux): around it so that we don't randomly set environment variables on platforms we don't intend to set them on.

Other than those two nits, everything seems fine.

Nice work, Henrik.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to comment #6)
> >    #sys.path.append(r'c:\mozilla-build\hg\library.zip')
> >    sys.path.append(r'c:\mozilla-build-libs\hg')
> Are you sure this is working ^? In a standard mozilla-build you don't get a
> directory called mozilla-build-libs but I'm not familiar with your operating
> environment here.

There is a bug in MozillaBuild which prevents us from using the modules under c:\mozilla-build\hg\library.zip. I investigate this problem with some Mercurial guys. Eventually Mozilla Build uses the wrong version of Mercurial. I revert that once the bug has been fixed.

> Since this is only needed for linux, it'd probably make sense to put an "if
> (linux): around it so that we don't randomly set environment variables on
> platforms we don't intend to set them on.

The display variable is only set when you fold in the --display option to that test. It will only be used on Linux. So it's already working like an if statement.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Python script v2

We were able to figure out the issue with Mercurial. What we missed was the Python specific Mercurial build which can be found at http://bitbucket.org/tortoisehg/thg-winbuild/downloads/

Means, we do not have to differentiate between the different OS anymore.