1014803 - ImportError and traceback attempting to run load from Mac

Status: VERIFIED FIXED

(Hello (Loop) :: Server, defect)

Description

[James Bonacci [:jbonacci]]

So, this is how I invoked the simplest of load tests:
./bin/loads-runner --server-url=https://loop.stage.mozaws.net -u 1 --hits=1 --agents=1 --user-id=jbonacci --ssh=[e-mail] loadtest.TestLoop.test_all

I actually did hit the Loads Cluster: 
https://loads.services.mozilla.com/run/b3fb817a-9218-4368-9517-049a4c37db43
Note that the dashboard entries match my load test request

And I am assuming I hit the Loop-Server stage environment.

But this is what I get back on the localhost:
[                                                                                                                                              ]   0%
Duration: 6.89 seconds
Hits: 0
Started: 2014-05-22 19:26:11.288822
Approximate Average RPS: 0
Average request time: 0.00s
Opened web sockets: 0
Bytes received via web sockets : 0

Success: 0
Errors: 1
Failures: 0

1 occurrences of: 
    <type 'exceptions.ImportError'>: 'module' object has no attribute 'TestLoop'  File "/home/ubuntu/loads/loads/runners/local.py", line 167, in execute    
    self._execute()    
  File "/home/ubuntu/loads/loads/runners/local.py", line 247, in _execute    
    self._run_python_tests()    
  File "/home/ubuntu/loads/loads/runners/local.py", line 252, in _run_python_tests    
    self._resolve_name()    
  File "/home/ubuntu/loads/loads/runners/local.py", line 89, in _resolve_name    
    self.test = resolve_name(self.fqn)    
  File "/home/ubuntu/loads/loads/util.py", line 168, in resolve_name    
    raise ImportError(exc)

Comment 1

[James Bonacci [:jbonacci]]

On the other hand, the quickie remote test of Stage works!
./bin/loads-runner loadtest.TestLoop.test_all --server-url http://loop.dev.mozaws.net/

Now, to debug...

Comment 2

[James Bonacci [:jbonacci]]

It's getting more interesting:
So, now I am doing this:
./bin/loads-runner --server-url=https://loop.stage.mozaws.net -u 1 --hits=1 --agents=1 --user-id=jbonacci --ssh=[e-mail] loadtest.TestLoop.test_all

in 3 runs, I saw 2 failures of this type:
Success: 0
Errors: 1
Failures: 0

1 occurrences of: 
    <type 'exceptions.KeyError'>: 'uuid'  File "/usr/lib/python2.7/unittest/case.py", line 332, in run    
    testMethod()    
  File "/home/ubuntu/loads/loadtest.py", line 14, in test_all    
    uuid, session, caller_token, api_key = self.initiate_call(token)    
  File "/home/ubuntu/loads/loadtest.py", line 38, in initiate_call    
    return (res['uuid'], res['sessionId'], res['sessionToken'],


and one generic error/failure:
Success: 0
Errors: 0
Failures: 0

From the dashboard, I see this again:
 1 occurrences:

'module' object has no attribute 'TestLoop'
  File "/home/ubuntu/loads/loads/runners/local.py", line 167, in execute
    self._execute()
  File "/home/ubuntu/loads/loads/runners/local.py", line 247, in _execute
    self._run_python_tests()
  File "/home/ubuntu/loads/loads/runners/local.py", line 252, in _run_python_tests
    self._resolve_name()
  File "/home/ubuntu/loads/loads/runners/local.py", line 89, in _resolve_name
    self.test = resolve_name(self.fqn)
  File "/home/ubuntu/loads/loads/util.py", line 168, in resolve_name
    raise ImportError(exc)

Comment 3

[James Bonacci [:jbonacci]]

So, this works:
./bin/loads-runner --server-url=http://loop.dev.mozaws.net -u 1 --user-id=jbonacci --ssh=[e-mail] loadtest.TestLoop.test_all

This does not:
./bin/loads-runner --server-url=https://loop.stage.mozaws.net -u 1 --hits=1 --agents=1 --user-id=jbonacci --ssh=[e-mail] loadtest.TestLoop.test_all

The only apparent differences being the use of --hits and --agents...

AND, also, important: I see no load test run on the dashboard if the first form is used.
I see a test run on the dashboard if the second form is used!...

Comment 4

[James Bonacci [:jbonacci]]

Adding just --hits
./bin/loads-runner --server-url=https://loop.stage.mozaws.net -u 1 --hits=1 --user-id=jbonacci --ssh=[e-mail] loadtest.TestLoop.test_all

Gives me this error:
Success: 0
Errors: 1
Failures: 0

1 occurrences of: 
    KeyError: 'apiKey'    Traceback: 
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 331, in run
    testMethod()
  File "loadtest.py", line 14, in test_all
    call_id, session, caller_token, api_key = self.initiate_call(token)
  File "loadtest.py", line 39, in initiate_call
    res['apiKey'])

And no update to the dashboard...

Comment 5

[James Bonacci [:jbonacci]]

Adding just --agents
./bin/loads-runner --server-url=https://loop.stage.mozaws.net -u 1 --agents=1 --user-id=jbonacci --ssh=[e-mail] loadtest.TestLoop.test_all

Gives me this error:
Success: 0
Errors: 1
Failures: 0

1 occurrences of: 
    <type 'exceptions.KeyError'>: 'uuid'  File "/usr/lib/python2.7/unittest/case.py", line 332, in run    
    testMethod()    
  File "/home/ubuntu/loads/loadtest.py", line 14, in test_all    
    uuid, session, caller_token, api_key = self.initiate_call(token)    
  File "/home/ubuntu/loads/loadtest.py", line 38, in initiate_call    
    return (res['uuid'], res['sessionId'], res['sessionToken'],   


That is all the permutations I have.
Will be interesting to get some feedback on all this, but also to see how the following bug fix addresses some of these failures: bug 1014271

Comment 6

[James Bonacci [:jbonacci]]

My previous comment was dead on - sorry for all the bug updates.
I realized after deconstructing the megabench load test for Tokenserver, that I was missing two key options that are in the config/megabench.ini file:
--include-file
--python-dep

so, for example, this works for Tokenserver:
../bin/loads-runner -u 1 -d 60 -a 1 --include-file=./loadtest.py --python-dep=PyBrowserID --ssh=[e-mail] --user-id=jbonacci --server-url=https://token.stage.mozaws.net loadtest.NodeAssignmentTest.test_realistic

Over on the Loop-Server side, this does not work:
./bin/loads-runner --server-url=https://loop.stage.mozaws.net -u 1 --agents=1 --user-id=jbonacci --ssh=[e-mail] loadtest.TestLoop.test_all

Stealing from bug 1014271, this should work:
./bin/loads-runner --server-url=https://loop.stage.mozaws.net -u 1 -d 60 -a 1 --include-file=./loadtest.py --user-id=jbonacci --ssh=[e-mail] loadtest.TestLoop.test_all
(no --python is needed in this case, nor is the --hits option)

And I finally get a load test to run!
Fails miserably, but it runs!