Closed Bug 1036726 Opened 10 years ago Closed 10 years ago

Loop-Server "make megabench" no longer works w/o a Loads error

Categories

(Hello (Loop) :: Server, defect, P1)

Product:
Hello (Loop)
Component:
Server
Type:
defect

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: jbonacci, Unassigned)

Details

(Whiteboard: [qa+]) 

Something must have changed here?
https://github.com/mozilla-services/loads

So, with a current deploy of loop-server on Mac, if I attempt a "make megabench" from the loadtests directory, this is what I see:

$ make megabench SERVER_URL=https://loop.stage.mozaws.net
./venv/bin/loads-runner --config=./config/megabench.ini --user-id=jbonacci --server-url=https://loop.stage.mozaws.net loadtest.TestLoop.test_all
usage: loads-runner [-h] [--config CONFIG] [-u USERS] [--test-dir TEST_DIR]
                    [--python-dep PYTHON_DEP] [--include-file INCLUDE_FILE]
                    [--ssh SSH] [--hits HITS | -d DURATION] [--version]
                    [--test-runner TEST_RUNNER] [--server-url SERVER_URL]
                    [--observer {irc,email}]
                    [--observer-irc-server OBSERVER_IRC_SERVER]
                    [--observer-irc-channel OBSERVER_IRC_CHANNEL]
                    [--observer-irc-port OBSERVER_IRC_PORT]
                    [--observer-irc-ssl OBSERVER_IRC_SSL]
                    [--observer-irc-nickname OBSERVER_IRC_NICKNAME]
                    [--observer-email-sender OBSERVER_EMAIL_SENDER]
                    [--observer-email-recipient OBSERVER_EMAIL_RECIPIENT]
                    [--observer-email-host OBSERVER_EMAIL_HOST]
                    [--observer-email-port OBSERVER_EMAIL_PORT]
                    [--observer-email-user OBSERVER_EMAIL_USER]
                    [--observer-email-password OBSERVER_EMAIL_PASSWORD]
                    [--observer-email-subject OBSERVER_EMAIL_SUBJECT]
                    [--no-patching] [--project-name PROJECT_NAME] [-a AGENTS]
                    [--zmq-receiver ZMQ_RECEIVER]
                    [--zmq-publisher ZMQ_PUBLISHER] [--ping-broker]
                    [--check-cluster] [--purge-broker] [-b BROKER]
                    [--user-id USER_ID] [--batched] [--quiet]
                    [--output {file,funkload,null,stdout}] [--attach]
                    [--detach] [--no-dns-resolve]
                    [--external-process-timeout EXTERNAL_PROCESS_TIMEOUT]
                    [--output-funkload-filename OUTPUT_FUNKLOAD_FILENAME]
                    [--output-file-filename OUTPUT_FILE_FILENAME]
                    [fqn]
loads-runner: error: unrecognized arguments: 1
make: *** [megabench] Error 2


I discovered if I remove or change --config=./config/megabench.ini
it works OK (of course I can't get my options)

Ideas?
Priority: -- → P1
Whiteboard: [qa+]
Makefile looks like this:
megabench:
	./venv/bin/loads-runner --config=./config/megabench.ini --user-id=jbonacci --server-url=$(SERVER_URL) loadtest.TestLoop.test_all


FYI this works!
"make purge"
./venv/bin/loads-runner --config=./config/megabench.ini --purge-broker

This does not:
./venv/bin/loads-runner --config=./config/megabench.ini  --user-id=jbonacci --server-url=https://loop.stage.mozaws.net loadtest.TestLoop.test_all

nor this:
./venv/bin/loads-runner --config=./config/megabench.ini --server-url=https://loop.stage.mozaws.net loadtest.TestLoop.test_all

This does:
./venv/bin/loads-runner --user-id=jbonacci --server-url=https://loop.stage.mozaws.net loadtest.TestLoop.test_all

I am so confused
Oh, that's probably because the version of load you have isn't up to date anymore. You'll need to reinstall it.

> make clean
> make install

And then start the tests again :)
Nope. That did not fix it.
This works:
./venv/bin/loads-runner --config=./config/test.ini --server-url=https://loop.stage.mozaws.net:443 loadtest.TestLoop.test_all

This works:
./venv/bin/loads-runner --user-id=jbonacci --server-url=https://loop.stage.mozaws.net:443 loadtest.TestLoop.test_all

This still does not work:
./venv/bin/loads-runner --config=./config/megabench.ini --user-id=jbonacci --server-url=https://loop.stage.mozaws.net:443 loadtest.TestLoop.test_all

This still does not work
./venv/bin/loads-runner --config=./config/megabench.ini --server-url=https://loop.stage.mozaws.net:443 loadtest.TestLoop.test_all

And now, this does not work
./venv/bin/loads-runner --config=./config/bench.ini --server-url=https://loop.stage.mozaws.net:443 loadtest.TestLoop.test_all

All of these return:
"loads-runner: error: unrecognized arguments: 1"
Status: NEW → ASSIGNED
Other examples:

Tokenserver works:
../local/bin/loads-runner --config=./config/megabench.ini --user-id=James --server-url=https://token.stage.mozaws.net loadtest.NodeAssignmentTest.test_realistic

Browserid-Verifier works:
./venv/bin/loads-runner --config=./config/megabench.ini --user-id=James --server-url=https://verifier.stage.mozaws.net loadtest.VerifierLoadTest.test_verifier

FxA-Auth-Server works:
./bin/loads-runner --config=./config/megabench.ini --user-id=James --server-url=https://api-accounts.stage.mozaws.net loadtests.LoadTest.test_auth_server

Server-Syncstorage works:
./bin/loads-runner --config=./config/megabench.ini --user-id=James --server-url=https://token.stage.mozaws.net stress.StressTest.test_storage_session
So, no, I can no longer get this to work:
./venv/bin/loads-runner --config=./config/megabench.ini --user-id=James --server-url=https://loop.stage.mozaws.net:443 loadtest.TestLoop.test_all
usage: blah blah blah
loads-runner: error: unrecognized arguments: 1
make: *** [megabench] Error 2

Tried this on Mac, 3 AWS instances, 2 Linux vms.

There is something unusual about the loads install within this venv.
I think this is related to https://github.com/mozilla-services/loads/issues/272 that is used here https://github.com/mozilla-services/loop-server/blob/master/loadtests/config/megabench.ini#L10

And loads cluster didn't get updated with the last version yet.
Flags: needinfo?(tarek)
Yes. You are absolutely right. How did I miss that option in the config file?
Obvious workaround for now is to comment out the "--no-dns-cache" line in config/megabench.ini.

Not sure how commenting out will impact the changes requested for the load test, but it's a hack to keep my going until the loads cluster is updated

FYI: https://github.com/mozilla-services/loads/issues/274
Yeah sorry I forgot to update the cluster. Done.

Let me know if that fixes your issue
Flags: needinfo?(tarek)
I verified the update on Loads Master/Slaves, but I am still getting the same error.
Right now, the only way around this is to comment out that line in the config/megabench.ini file.
Please update your loop-server repo and try again
Flags: needinfo?(jbonacci)
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Flags: needinfo?(jbonacci)
Resolution: --- → FIXED
Ah, so it's the "1" causing all the trouble.
no-dns-resolve = 1 causes the error.
no-dns-resolve = true works great.

The repo has been updated.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.