1255620 - Stop running mochitest in one tab

Status: NEW

Description

[Nils Ohlmeier [:drno]]

The fact that we are running our mochitest in tab is not very realistic:
- we don't have any signaling path
- ICE candidates take a different path then the SDP (and therefore can be faster)
- we are accessing the second PeerConnection sometimes directly
- ...

What can we do to make our PeerConnection mochitest's more realistic and close the gap to the more realistic steeplechase setup (with two Firefox running on two separate machines)?
The motivation to change this is that I hope that it will improve our quality by getting our tests closer to how our APIs are used in the field.

Options I see so far:
1) We could put each PeerConnection into its own tab.
  - This will prevent direct access to the other PeerConnection
  - This would require a signaling path through...
    - localStorage?
    - WebWorkers?
2) We could start two instances of Firefox
  - This is the most realistic scenario
  - Steeplecachase allows to start two Firefox instances on a local host
  - We would still require a signaling path, which probably needs to be a separate server
  - I'm concerned that starting two Firefox instances will bring down all test machines 

Might worth pointing out that we don't necessarily have to move all of the PeerConnection tests over to this new schema (if we find one), but leave certain tests running in the one tab mode (for convenience reasons, or too much effort to port things over, or...).

Comment 1

[Ted Mielczarek [:ted] [:ted.mielczarek]]

I think we could tweak Steeplechase a bit to allow it to run two local Gecko processes, maybe just using Marionette would be fine? We could also rewrite the signalling server it uses in Python (It's all of 21 lines of JS right now: https://github.com/luser/simplesignalling/blob/master/server.js ) and just launch it in the same process as the Steeplechase test harness. I'm actually not sure why I didn't implement things that way in the first place.

Comment 2

[Bulk Bug Changes for mreavy's org]

Mass change P2->P3 to align with new Mozilla triage process.