Closed Bug 1029751 Opened 11 years ago Closed 11 years ago

Some Loop-Server unit tests are failing on Ubuntu and RHEL Linux

Categories

(Hello (Loop) :: Server, defect)

Product:
Hello (Loop)
Component:
Server
Platform:
All
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: jbonacci, Unassigned)

Details

(Whiteboard: [qa+])

14.04 seems to be the new 64-bit default for AWS instances. I installed loop-server and redis on my host. Running 'make test', I see the following: 177 passing (3s) 3 pending 4 failing 1) index.js #shutdown should call exit(0) on the process object: Uncaught AssertionError: expected 'missing audience parameter' to deeply equal 'Invalid audience' + expected - actual +"Invalid audience" -"missing audience parameter" at /home/ubuntu/loop-server/test/fxa_test.js:192:26 at Request._callback (/home/ubuntu/loop-server/loop/fxa.js:9:2208) at Request.self.callback (/home/ubuntu/loop-server/node_modules/request/request.js:121:22) at Request.EventEmitter.emit (events.js:98:17) at Request.<anonymous> (/home/ubuntu/loop-server/node_modules/request/request.js:985:14) at Request.EventEmitter.emit (events.js:117:20) at IncomingMessage.<anonymous> (/home/ubuntu/loop-server/node_modules/request/request.js:936:12) at IncomingMessage.EventEmitter.emit (events.js:117:20) at _stream_readable.js:920:16 at process._tickCallback (node.js:415:13) 2) index.js authentication middleware BrowserID should accept assertions and return hawk credentials: Uncaught AssertionError: expected undefined not to be undefined 3) index.js authentication middleware BrowserID shouldn't accept invalid assertions: Error: expected 401 "Unauthorized", got 500 "Internal Server Error" at Test.assert (/home/ubuntu/loop-server/node_modules/supertest/lib/test.js:197:15) at /home/ubuntu/loop-server/node_modules/supertest/lib/test.js:126:10 at Test.Request.callback (/home/ubuntu/loop-server/node_modules/supertest/node_modules/superagent/lib/node/index.js:660:30) at Test.<anonymous> (/home/ubuntu/loop-server/node_modules/supertest/node_modules/superagent/lib/node/index.js:132:10) at Test.EventEmitter.emit (events.js:95:17) at IncomingMessage.<anonymous> (/home/ubuntu/loop-server/node_modules/supertest/node_modules/superagent/lib/node/index.js:818:12) at IncomingMessage.EventEmitter.emit (events.js:117:20) at _stream_readable.js:920:16 at process._tickCallback (node.js:415:13) 4) index.js authentication middleware Hawk should update session expiration time on auth: TypeError: Cannot call method 'spy' of undefined at Context.<anonymous> (/home/ubuntu/loop-server/test/index_test.js:331:17) at Test.Runnable.run (/home/ubuntu/loop-server/node_modules/mocha/lib/runnable.js:196:15) at Runner.runTest (/home/ubuntu/loop-server/node_modules/mocha/lib/runner.js:373:10) at /home/ubuntu/loop-server/node_modules/mocha/lib/runner.js:451:12 at next (/home/ubuntu/loop-server/node_modules/mocha/lib/runner.js:298:14) at /home/ubuntu/loop-server/node_modules/mocha/lib/runner.js:308:7 at next (/home/ubuntu/loop-server/node_modules/mocha/lib/runner.js:246:23) at /home/ubuntu/loop-server/node_modules/mocha/lib/runner.js:270:7 at done (/home/ubuntu/loop-server/node_modules/mocha/lib/runnable.js:185:5) at /home/ubuntu/loop-server/node_modules/mocha/lib/runnable.js:199:9 at try_callback (/home/ubuntu/loop-server/node_modules/redis/index.js:573:9) at RedisClient.return_reply (/home/ubuntu/loop-server/node_modules/redis/index.js:661:13) at HiredisReplyParser.<anonymous> (/home/ubuntu/loop-server/node_modules/redis/index.js:309:14) at HiredisReplyParser.EventEmitter.emit (events.js:95:17) at HiredisReplyParser.execute (/home/ubuntu/loop-server/node_modules/redis/lib/parser/hiredis.js:43:18) at RedisClient.on_data (/home/ubuntu/loop-server/node_modules/redis/index.js:534:27) at Socket.<anonymous> (/home/ubuntu/loop-server/node_modules/redis/index.js:91:14) at Socket.EventEmitter.emit (events.js:95:17) at Socket.<anonymous> (_stream_readable.js:746:14) at Socket.EventEmitter.emit (events.js:92:17) at emitReadable_ (_stream_readable.js:408:10) at emitReadable (_stream_readable.js:404:5) at readableAddChunk (_stream_readable.js:165:9) at Socket.Readable.push (_stream_readable.js:127:10) at TCP.onread (net.js:526:21) ============================================================================= Writing coverage object [/home/ubuntu/loop-server/coverage/coverage.json] Writing coverage reports at [/home/ubuntu/loop-server/coverage] ============================================================================= =============================== Coverage summary =============================== Statements : 89.43% ( 592/662 ) Branches : 78.74% ( 200/254 ) Functions : 94.93% ( 131/138 ) Lines : 89.97% ( 583/648 ) ================================================================================ make: *** [cover-mocha] Error 4
Whiteboard: [qa+]
Same errors when running against RHEL 6.5. There are slightly different errors for SciLinux 6.4: 172 passing (2s) 3 pending 5 failing 1) index.js #validateSimplePushURL should work with a valid simple push url: Uncaught AssertionError: expected 'missing audience parameter' to deeply equal 'Invalid audience' + expected - actual +"Invalid audience" -"missing audience parameter" at /home/ec2-user/loop-server/test/fxa_test.js:192:26 at Request._callback (/home/ec2-user/loop-server/loop/fxa.js:9:2208) at Request.self.callback (/home/ec2-user/loop-server/node_modules/request/request.js:121:22) at Request.EventEmitter.emit (events.js:98:17) at Request.<anonymous> (/home/ec2-user/loop-server/node_modules/request/request.js:985:14) at Request.EventEmitter.emit (events.js:117:20) at IncomingMessage.<anonymous> (/home/ec2-user/loop-server/node_modules/request/request.js:936:12) at IncomingMessage.EventEmitter.emit (events.js:117:20) at _stream_readable.js:920:16 at process._tickCallback (node.js:415:13) 2) index.js authentication middleware Hawk should accept valid hawk sessions: Error: expected 200 "OK", got 401 "Unauthorized" at Test.assert (/home/ec2-user/loop-server/node_modules/supertest/lib/test.js:197:15) at /home/ec2-user/loop-server/node_modules/supertest/lib/test.js:126:10 at Test.Request.callback (/home/ec2-user/loop-server/node_modules/supertest/node_modules/superagent/lib/node/index.js:660:30) at Test.<anonymous> (/home/ec2-user/loop-server/node_modules/supertest/node_modules/superagent/lib/node/index.js:132:10) at Test.EventEmitter.emit (events.js:95:17) at IncomingMessage.<anonymous> (/home/ec2-user/loop-server/node_modules/supertest/node_modules/superagent/lib/node/index.js:818:12) at IncomingMessage.EventEmitter.emit (events.js:117:20) at _stream_readable.js:920:16 at process._tickCallback (node.js:415:13) 3) index.js authentication middleware Hawk should accept valid hawk sessions: Error: expected 200 "OK", got 401 "Unauthorized" at Test.assert (/home/ec2-user/loop-server/node_modules/supertest/lib/test.js:197:15) at /home/ec2-user/loop-server/node_modules/supertest/lib/test.js:126:10 at Test.Request.callback (/home/ec2-user/loop-server/node_modules/supertest/node_modules/superagent/lib/node/index.js:660:30) at Test.<anonymous> (/home/ec2-user/loop-server/node_modules/supertest/node_modules/superagent/lib/node/index.js:132:10) at Test.EventEmitter.emit (events.js:95:17) at IncomingMessage.<anonymous> (/home/ec2-user/loop-server/node_modules/supertest/node_modules/superagent/lib/node/index.js:818:12) at IncomingMessage.EventEmitter.emit (events.js:117:20) at _stream_readable.js:920:16 at process._tickCallback (node.js:415:13) 4) index.js authentication middleware Hawk shouldn't accept invalid hawk credentials: TypeError: Cannot set property 'id' of undefined at Context.<anonymous> (/home/ec2-user/loop-server/test/index_test.js:321:30) at Test.Runnable.run (/home/ec2-user/loop-server/node_modules/mocha/lib/runnable.js:196:15) at Runner.runTest (/home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:373:10) at /home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:451:12 at next (/home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:298:14) at /home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:308:7 at next (/home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:246:23) at Object._onImmediate (/home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:275:5) at processImmediate [as _immediateCallback] (timers.js:330:15) 5) index.js authentication middleware Hawk should update session expiration time on auth: TypeError: Attempted to wrap touchHawkSession which is already spied on at Object.wrapMethod (/home/ec2-user/loop-server/node_modules/sinon/lib/sinon.js:84:25) at Object.spy (/home/ec2-user/loop-server/node_modules/sinon/lib/sinon/spy.js:41:22) at Object.spy (/home/ec2-user/loop-server/node_modules/sinon/lib/sinon/collection.js:89:39) at Context.<anonymous> (/home/ec2-user/loop-server/test/index_test.js:331:17) at Test.Runnable.run (/home/ec2-user/loop-server/node_modules/mocha/lib/runnable.js:196:15) at Runner.runTest (/home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:373:10) at /home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:451:12 at next (/home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:298:14) at /home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:308:7 at next (/home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:246:23) at Object._onImmediate (/home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:275:5) at processImmediate [as _immediateCallback] (timers.js:330:15) -------------- Error: Stack Trace for original at Object.wrapMethod (/home/ec2-user/loop-server/node_modules/sinon/lib/sinon.js:101:30) at Object.spy (/home/ec2-user/loop-server/node_modules/sinon/lib/sinon/spy.js:41:22) at Object.spy (/home/ec2-user/loop-server/node_modules/sinon/lib/sinon/collection.js:89:39) at Context.<anonymous> (/home/ec2-user/loop-server/test/index_test.js:331:17) at Test.Runnable.run (/home/ec2-user/loop-server/node_modules/mocha/lib/runnable.js:196:15) at Runner.runTest (/home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:373:10) at /home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:451:12 at next (/home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:298:14) at /home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:308:7 at next (/home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:246:23) at Object._onImmediate (/home/ec2-user/loop-server/node_modules/mocha/lib/runner.js:275:5) at processImmediate [as _immediateCallback] (timers.js:330:15) ============================================================================= Writing coverage object [/home/ec2-user/loop-server/coverage/coverage.json] Writing coverage reports at [/home/ec2-user/loop-server/coverage] ============================================================================= =============================== Coverage summary =============================== Statements : 91.09% ( 603/662 ) Branches : 80.31% ( 204/254 ) Functions : 96.38% ( 133/138 ) Lines : 91.98% ( 596/648 ) ================================================================================ make: *** [cover-mocha] Error 5
Summary: Some unit tests are failing on Ubuntu 14.04 → Some unit tests are failing on Ubuntu 14.04 and RHEL 6.5 and SciLinux 6.4
This is related to your configuration parameters. Make sure you have "fxaAudiences": ["http://localhost:5000"], in your config/test.json
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Yep. I am using the default config/test.json in all cases. The all have this line: "fxaAudiences": ["http://localhost:5000"], I am wondering if this is related to the use of AWS instances. I will bring up a Linux VM (on my Mac) and see what happens...
So on SciLinux 6.4 (running in a VM), I get two errors, different from the above runs on AWS instances: 179 passing (886ms) 3 pending 2 failing 1) FakeTokBox #getSessionTokens should expose a apiKey property.: Uncaught AssertionError: expected 'missing audience parameter' to deeply equal 'Invalid audience' + expected - actual +"Invalid audience" -"missing audience parameter" at /home/mozilla/loop-server/test/fxa_test.js:192:26 at Request._callback (/home/mozilla/loop-server/loop/fxa.js:9:2208) at Request.self.callback (/home/mozilla/loop-server/node_modules/request/request.js:121:22) at Request.EventEmitter.emit (events.js:98:17) at Request.<anonymous> (/home/mozilla/loop-server/node_modules/request/request.js:985:14) at Request.EventEmitter.emit (events.js:117:20) at IncomingMessage.<anonymous> (/home/mozilla/loop-server/node_modules/request/request.js:936:12) at IncomingMessage.EventEmitter.emit (events.js:117:20) at _stream_readable.js:920:16 at process._tickCallback (node.js:415:13) 2) FakeTokBox #getSessionTokens "before each" hook: TypeError: Attempted to wrap get which is already spied on at Object.wrapMethod (/home/mozilla/loop-server/node_modules/sinon/lib/sinon.js:84:25) at Object.stub (/home/mozilla/loop-server/node_modules/sinon/lib/sinon/stub.js:56:22) at Object.stub (/home/mozilla/loop-server/node_modules/sinon/lib/sinon/collection.js:122:40) at Context.<anonymous> (/home/mozilla/loop-server/test/tokbox_test.js:126:15) at callFn (/home/mozilla/loop-server/node_modules/mocha/lib/runnable.js:223:21) at Hook.Runnable.run (/home/mozilla/loop-server/node_modules/mocha/lib/runnable.js:216:7) at next (/home/mozilla/loop-server/node_modules/mocha/lib/runner.js:258:10) at Object._onImmediate (/home/mozilla/loop-server/node_modules/mocha/lib/runner.js:275:5) at processImmediate [as _immediateCallback] (timers.js:330:15) -------------- Error: Stack Trace for original at Object.wrapMethod (/home/mozilla/loop-server/node_modules/sinon/lib/sinon.js:101:30) at Object.stub (/home/mozilla/loop-server/node_modules/sinon/lib/sinon/stub.js:56:22) at Object.stub (/home/mozilla/loop-server/node_modules/sinon/lib/sinon/collection.js:122:40) at Context.<anonymous> (/home/mozilla/loop-server/test/tokbox_test.js:126:15) at callFn (/home/mozilla/loop-server/node_modules/mocha/lib/runnable.js:223:21) at Hook.Runnable.run (/home/mozilla/loop-server/node_modules/mocha/lib/runnable.js:216:7) at next (/home/mozilla/loop-server/node_modules/mocha/lib/runner.js:258:10) at Object._onImmediate (/home/mozilla/loop-server/node_modules/mocha/lib/runner.js:275:5) at processImmediate [as _immediateCallback] (timers.js:330:15) ============================================================================= Writing coverage object [/home/mozilla/loop-server/coverage/coverage.json] Writing coverage reports at [/home/mozilla/loop-server/coverage] ============================================================================= =============================== Coverage summary =============================== Statements : 69.25% ( 671/969 ) Branches : 57.22% ( 226/395 ) Functions : 79.31% ( 161/203 ) Lines : 70.1% ( 661/943 ) ================================================================================ make: *** [cover-mocha] Error 2 Bouncing so we can walk through all of these and make sure test are running (or failing) as expected)...
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
:jbonnacci could you please update the status of this bugs with the last version of Loop-server? Could you also try to run then with a hight timeout value? NODE_ENV=test ./node_modules/.bin/istanbul cover \ ./node_modules/.bin/_mocha -- --reporter spec -t 5000 test/*
Flags: needinfo?(jbonacci)
First runs with latest Master on Ubuntu, RHEL 6.4, RHEL 6.5 Ubuntu results - 4 errors: https://jbonacci.pastebin.mozilla.org/5534371 RHEL 6.5 results - no serious errors: https://jbonacci.pastebin.mozilla.org/5534372 RHEL 6.4 results - 1 error: https://jbonacci.pastebin.mozilla.org/5534373
Flags: needinfo?(jbonacci)
Next I used this command: NODE_ENV=test ./node_modules/.bin/istanbul cover ./node_modules/.bin/_mocha -- --reporter spec -t 5000 test/* Ubuntu results: https://jbonacci.pastebin.mozilla.org/5534405 RHEL 6.5 results: https://jbonacci.pastebin.mozilla.org/5534414 RHEL 6.4 results: https://jbonacci.pastebin.mozilla.org/5534416
Ok then I will fix the Makefile to use a higher timeout because of async that may take more time on some computers.
https://github.com/mozilla-services/loop-server/commit/8f76c4eddad25c4d5cd638f91adbdf076cfdf339
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
On Ubuntu, similar errors as before for 'make test': 220 passing (10s) 5 failing I get a cleaner run for 'make travis'. On rhel 6.5, similar errors as before for 'make test': 197 passing (4s) 2 failing For 'make travis': 131 passing (3s) 2 failing On rhel 6.4: 80 passing (2s) 1 failing That seems like to few tests... For 'make travis': 153 passing (3s) 2 failing And just for kicks: NODE_ENV=test ./node_modules/.bin/istanbul cover ./node_modules/.bin/_mocha -- --reporter spec -t 5000 test/* 209 passing (4s) 10 failing so, basically, I continue to see varying numbers and types of errors running the unit tests on Linux. Not sure what to make of this... For comparison, 'make test' on my Mac: https://jbonacci.pastebin.mozilla.org/5538449 but, look at 'make travis' on my Mac: https://jbonacci.pastebin.mozilla.org/5538452 What is Dev's expectations for unit tests on Linux? QA expects 100% pass, near 100% coverage (we are not too bad with that)... Right now, I am mostly concerned with clean runs of the non-websocket tests, which we do not have...
James, can you provide me with an access to a server so I can test there? public key: http://alexis.notmyidea.org/id_rsa.pub
:alexis - yes I will do this tomorrow morning (Friday), so you can hop on the boxes Monday...
OK, I have added you to me 3 AWS instances. I am forwarding on their names in a separate email.
Summary: Some unit tests are failing on Ubuntu 14.04 and RHEL 6.5 and SciLinux 6.4 → Some Loop-Server unit tests are failing on Ubuntu and RHEL Linux
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
:alexis can you have another look at this?
^
Flags: needinfo?(alexis+bugs)
I cannot find the mail with machine names. James can you forward it to me or put the names here? Thanks.
Just sent via email.
:bump:
jbonacci can you check again please?
Flags: needinfo?(jbonacci)
Yes, I will very soon. I have not forgotten...
Flags: needinfo?(jbonacci)
Closing for now. Feel free to reopen if you have any problem.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Resolution: FIXED → WORKSFORME
This looks good for Ubuntu 14.04. This looks good for rhel 6.5. This also looks good for rhel 6.4 (SciLinux 6.4)
Status: RESOLVED → VERIFIED
Resolution: WORKSFORME → FIXED
You need to log in before you can comment on or make changes to this bug.