830920 - Reduce running time of test_websocket_basic.html

Status: RESOLVED FIXED

(Core :: Networking: WebSockets, defect)

Description

[Jason Duell]

Attachment: reduce test count iteration from 100 -> 10.

smaug:

Apparently test_websocket_basic (which sends 100 WS msgs, and calls forcegc() after each one) is now taking so long to run (300 sec) that it gets timed out.

I'm not sure of why this takes longer than it used to (apparently billm says they added more debug assertions in the GC--the timeout happens only for debug builds), nor why we felt the need to make the loop run 100x, which seems like a lot.  This path just drops it down to 10x.  Does that sound ok?

This is blocking 818023.

Comment 1

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 702451 [details] [diff] [review]
reduce test count iteration from 100 -> 10.

Ok

Comment 2

[Jason Duell]

https://hg.mozilla.org/integration/mozilla-inbound/rev/2c96e14d0557

Landed with DONTBUILD after asking edmorley, since there's no real reason to burn a whole bot run on this trivial change.

Let me know if this needs to land on aurora/beta.

Comment 3

[Jason Duell]

Comment on attachment 702451 [details] [diff] [review]
reduce test count iteration from 100 -> 10.

Apparently this test is not quite hitting mochitest timeout (300 seconds) on aurora/beta, but this does seem like low-hanging fruit for reducing test server load.  

[Approval Request Comment]
User impact if declined:  bot cycles burned
Risk to taking this patch (and alternatives if risky): none, test-change only.

Comment 4

[Nicolas B. Pierron [:nbp]]

FYI, I collected all the timestamps from the recent logs of tbpl, and I noticed that beta is still at 180s, and latest revisions of inbound are above 300s (not higher due to timeouts).

http://people.mozilla.com/~npierron/find-websocket-regression/

Can someone find precisely what might be the sources of these slow down ?

Each file contains numbers which are directly coming from tbpl's ftp logs:
ftp://ftp.mozilla.org/pub/firefox/tinderbox-builds/mozilla-inbound-win32-debug/

Comment 5

[Jason Duell]

The slowdown is most likely a change in GC, so hopefully Ollie knows or can CC people who do.  The behavior here (send a single message over and over and forceGC each time) is not realistic behavior, but I'd really love to be assured that we know why the timing is going up and that it's OK.

Comment 6

[Olli Pettay [:smaug][[e-mail]]]

I don't know about changes to GC. (I hack CC only)

Comment 7

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 702451 [details] [diff] [review]
reduce test count iteration from 100 -> 10.

test-only, approving for branches

Comment 8

[Andrew McCreight [:mccr8]]

billm added a bunch of checking to debug GCs, so they are a lot slower now. It doesn't matter much in general, but if you are running it 100 times it could add up.

Comment 9

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/2c96e14d0557

Comment 10

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g18/rev/11818a1d9dfc

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/b6ed291b504f

Comment 12

[Ryan VanderMeulen [:RyanVM]]

(Also, this is in my queue to land on Aurora once it reopens)

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/2fc1b0408462

Comment 14

[Manuela Muntean [Away]]

Marking this [qa-]. Please remove the tag if there is anything the QA can help with.