1083903 - Running load test with current Master branch causes AssertionFailure

Status: VERIFIED FIXED

(Hello (Loop) :: Server, defect, P1)

Description

[James Bonacci [:jbonacci]]

This is a potential blocker to 0.12.5 load testing...


This is what I see after installing lastest Master from the loop-server repo

Clone, install, cd loadtest, make install, 

$ make test
./venv/bin/loads-runner --config=./config/test.ini --server-url=https://loop.stage.mozaws.net:443 loadtest.TestLoop.test_all
[==============================================================================================================================================] 100%
Duration: 3.18 seconds
Hits: 6
Started: 2014-10-16 16:44:16.622811
Approximate Average RPS: 1
Average request time: 0.26s
Opened web sockets: 2
Bytes received via web sockets : 352

Success: 0
Errors: 0
Failures: 1

1 occurrences of: 
    AssertionError: Registration failed: {"code":400,"errno":108,"error":"Missing: simplePushURL"}    Traceback: 
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 331, in run
    testMethod()
  File "loadtest/__init__.py", line 31, in test_all
    self.setupRoom()
  File "loadtest/rooms.py", line 16, in setupRoom
    "rooms": "http://httpbin.org/deny"
  File "loadtest/calls.py", line 21, in register
    "Registration failed: %s" % resp.content)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 515, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 508, in _baseAssertEqual
    raise self.failureException(msg)


Slowest URL: https://loop.stage.mozaws.net:443/ 	Average Request Time: 0.859505

Stats by URLs:
- https://loop.stage.mozaws.net:443/                 	Average request time: 0.859505	Hits success rate: 1.0
- https://loop.stage.mozaws.net:443/calls/Tkbuvu9HqSk	Average request time: 0.305408	Hits success rate: 1.0
- https://loop.stage.mozaws.net:443/registration     	Average request time: 0.101764	Hits success rate: 0.5
- https://loop.stage.mozaws.net:443/calls?version=200	Average request time: 0.097673	Hits success rate: 1.0
- https://loop.stage.mozaws.net:443/call-url         	Average request time: 0.096407	Hits success rate: 1.0

Custom metrics:
- register : 1
- list-pending-calls : 1
- generate-call-url : 1
- initiate-call : 1
make: *** [test] Error 1

Comment 1

[James Bonacci [:jbonacci]]

Switched to the 0.12.x branch and rebuilt everything.

I see the same error.

1 occurrences of: 
    AssertionError: Registration failed: {"code":400,"errno":108,"error":"Missing: simplePushURL"}

Comment 2

[James Bonacci [:jbonacci]]

Trying Remy's command...

So I cleaned the loadtest directory
I cleaned the loop-server directory
I ran this: 'git checkout -b 0.12.5 0.12.5'
which gave me this:
Switched to a new branch '0.12.5'

make install
cd loadtests
make build
make test
and I get this:
1 occurrences of: 
    AssertionError: Registration failed: {"code":400,"errno":108,"error":"Missing: simplePushURL"}    Traceback: 
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 331, in run
    testMethod()
  File "loadtest/__init__.py", line 31, in test_all
  File "loadtest/rooms.py", line 16, in setupRoom
  File "loadtest/calls.py", line 21, in register
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 515, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 508, in _baseAssertEqual
    raise self.failureException(msg)


$ git branch
* 0.12.5
  master


Ideas?

Comment 3

[James Bonacci [:jbonacci]]

List of available branches:
* 0.12.5
  master
  remotes/origin/0.12.x
  remotes/origin/1070889-heartbeat-write-redis
  remotes/origin/1072184-delete-hawk-session
  remotes/origin/1074468-log-callerId-calleeId
  remotes/origin/HEAD -> origin/master
  remotes/origin/master

I have tried master, remotes/origin/0.12.x, and 0.12.5 directly using Remy's command

I end up with the loadtest subfolder in all cases.


How do we get back to https://github.com/mozilla-services/loop-server/tree/0.12.x/loadtests
in order to run a loadtest without Rooms support?

Comment 4

[Rémy Hubscher (:natim)]

It is because there is some .pyc files inside so git kept it. Just remove the folder to keep going.

Comment 5

[James Bonacci [:jbonacci]]

Ok, starting over.

git/cloned loop-server
changed the branch to remotes/origin/0.12.x
$ make install
$cd loadtests
$ ls
Makefile		config			loadtest.py		requirements.txt

so far, so good

$ make build
$ ls
Makefile		config			loadtest.py		requirements.txt	venv

That should do it.

$ make test
Success: 1


OK. That was messy. But it works now...

I hate git