Closed Bug 541412 Opened 10 years ago Closed 10 years ago

add runtestsremote.py to mozilla-central for remote mochitests

Categories

(Testing :: General, defect)

ARM
Windows Mobile 6 Professional
defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jmaher, Assigned: jmaher)

References

Details

Attachments

(1 file, 5 obsolete files)

In order to run mochitests on a remote device using the SUT agent, we need to change the way we do file i/o, process handling, and system commands such that some are on the local system and others are on the remote system.

runtestsremote.py will work like runtests.py except that it will have additional command line parameters to specify the device we are testing.  These changes in code will be done by creating a subclass of automation.py and runtests.py.
Assignee: nobody → jmaher
Blocks: 538522
this is a subclass of mochitest, mochitestoptions and automation, with the intention of running tests remotely.  This assume devicemanger.py is installed and there is a webserver already running.
Attachment #425071 - Flags: review?(ctalbert)
Comment on attachment 425071 [details] [diff] [review]
subclass of mochitest and related classes

Did a quick review on this yesterday, then tried to use it.  The review found a couple of nits and the using it found a problem.  Here they are:

Big problem.  When trying to use this, I can't get the remoteWebServer variable to be set from the commandline --remote-webserver parameter.

And two nits in the patch.
runtestsremote.patch
patch line 160 - defaulting to windows specific slashes (should use / and let SUTagent sort it out).

comments are more readable if you do the format: # <blah> instead of #blah
Attachment #425071 - Flags: review?(ctalbert) → review-
updated with nits and working example of everything else.  For this to be 100% functional, this needs bug 544097 and bug 512319.  Since this is a standalone file and is not related to anything else, we can start the process of getting it in if the there is an r+
Attachment #428488 - Flags: review?(ctalbert)
Comment on attachment 428488 [details] [diff] [review]
subclass of mochitest and related classes (2)

This looks good.  I ran it on the phone and it works as long as we put in the     auto.setServerPort(options.webServer, options.httpPort, options.sslPort)

right before the last line in main (before the sys.exit(mochitest.runTests(options)

r+ with that.  And I can make that change and check it in.
Attachment #428488 - Flags: review?(ctalbert) → review+
Ok and a little more testing and I realized a couple of other things that we should be aware of, but that I don't think are bugs.

1. The --remoteLogFile parameter MUST have os specific slashes for its path.  I think defaulting to '/' slashes in the runtestsremote.py file is the way to go because percentage wise for our remote platforms more of them will be *nix based and use '/'.  However, this means that for windows mobile, the code that calls runtestsremote MUST specify the --remoteLogFile variable and set it to a path containing '\' style slashes.  This is entirely do-able from the buildbot automation side.  I wonder if we should just make the runtestsremote do a sys.exit if it determines that you did not specify a remote-log file.  Since you can't watch the output through the stdout anyway, I can't really see a use-case for not specifying remoteLogFile.

2. If you are running from a mac, then mochitest will add a -foreground to your generated commandline.  That commandline does not work for fennec and breaks the tests.  While this is not an issue for automation, this is likely something we want to fix for developers who try to run this because it is entirely unobvious what is happening when the failure occurs.  

I think we might want to solve both of these issues before check in.  So I'm going to create a patch and put it up for review rather than check in the code right now.
Here is a patch to fix the issue I mentioned in my review with setServerPort, the remoteLogFile issue and the -foreground issue.
Attachment #425071 - Attachment is obsolete: true
Attachment #428488 - Attachment is obsolete: true
Attachment #428857 - Flags: review?(jmaher)
Attached patch Fixing one more nit (obsolete) — Splinter Review
Joel noticed that the hardcoded .exe endings somehow crept back into these latest set of patches.  Fixed that once and for all this time here. Includes the other changes.
Attachment #428857 - Attachment is obsolete: true
Attachment #428862 - Flags: review?(jmaher)
Attachment #428857 - Flags: review?(jmaher)
Attached patch run mochitests remotely (obsolete) — Splinter Review
add a try clause around the args.remove.  Also cleaned up some of the defaults so we don't have a lot of hardcoded names and .exe's!
Attachment #428862 - Attachment is obsolete: true
Attachment #428930 - Flags: review?(ctalbert)
Attachment #428862 - Flags: review?(jmaher)
forgot to hg add the file since this is a new patch on my system.
Attachment #428930 - Attachment is obsolete: true
Attachment #428934 - Flags: review?(ctalbert)
Attachment #428930 - Flags: review?(ctalbert)
Comment on attachment 428934 [details] [diff] [review]
run mochitests remotely (2)

I think we have it now.  Tested on the phone, ran great.
Attachment #428934 - Flags: review?(ctalbert) → review+
Pushed as changeset d38a004893a5
--> Fixed
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Component: New Frameworks → General
You need to log in before you can comment on or make changes to this bug.