662180 - Investigate SSL support in fakeserver

Status: NEW

(MailNews Core :: Testing Infrastructure, defect)

Description

[Joshua Cranmer [:jcranmer]]

Let's face it: most people use (or ought to be using) SSL connections for their mail. As long as we don't support SSL in fakeserver, we are testing distinct configurations from what our users use.

Notes: mochitest uses ssltunnel to test SSL connections, which is basically a proxy which forwards from its listening port to another listening port.

Another option which should be tested is STARTTLS, which ssltunnel probably cannot support without us getting crafty (which I may do anyways ^_^). If we do not go with using the vanilla ssltunnel binary, we must either use C++ or jsctypes to implement part of the code, since we do not expose enough NSS APIs in JS. Most annoyingly, we need the raw fd's to use with NSS, and I don't think there is any XPCOM API (even [noscript] !) that allows us to grab the fd for a socket set up via XPCOM socket stuff.

Comment 1

[Joshua Cranmer [:jcranmer]]

So here is the basic crazy idea I thought up last night: when we want to do SSL, we proxy the data that nsMailServer sends/receives to another connection that uses ssltunnel to do encryption/decryption. We should be able to achieve starttls by delaying the start of this ssltunnel proxy until when we initiate the SSL handshake.

Comment 2

[David :Bienvenu]

(In reply to comment #1)
> So here is the basic crazy idea I thought up last night: when we want to do
> SSL, we proxy the data that nsMailServer sends/receives to another
> connection that uses ssltunnel to do encryption/decryption. We should be
> able to achieve starttls by delaying the start of this ssltunnel proxy until
> when we initiate the SSL handshake.

Can we count on all the machines that run tests having ssltunnel in a place that we can use it from xpcshell?

Comment 3

[Joshua Cranmer [:jcranmer]]

ssltunnel is built for mochitests, and I do see it in my objdir, but I don't run packaged tests so I don't know what gets dropped where. It seems the more annoying part is the certs database which appears to be necessary, but I can't seem to find where m-c sticks their database.

If worst comes to worst, it should be possible to implement most of the ssltunnel functionality we need via jsctypes; the main sticking point is that we need PRFileDesc* objects for the streams instead of nsIInputStream and friends. But I don't want to go with that approach if ssltunnel can be made to work.

Comment 4

[David :Bienvenu]

Maybe Clint knows aout the cert db w/ mochitests.

Comment 5

[cmtalbert]

The certs db is built during any build with --enable-tests (which should be the default). It may be dependent on mochitests, so perhaps you have to build them to get it.  If you do make package-tests in your objdir, you should get the certs built into your dist/test-package-stage directory.  it lives in the certs directory. 

If you just need a database to play with for now, just download a packaged tests zip from the FTP directories and use that. (the certs wind up in the certs folder).

Does that answer the question?

If you have more questions w.r.t. ssltunnel itself, I know that Jonathan worked on it a bit when he was working on getting websockets support coded in.  So, I've cc'd him in case he can answer some of the explicit ssltunnel questions.

Comment 6

[David :Bienvenu]

Thx very much for the info, Clint! jcranmer, is that helpful?

Comment 7

[Joshua Cranmer [:jcranmer]]

Okay, I see the certs db now, so I can have a chance to play with it ^_^.

Comment 8

[(not currently active) Ted Mielczarek]

Do feel free to hack ssltunnel as necessary, its sole purpose is to be a test tool. Honza (:mayhemer) has done most of the hacking on it lately, so you can direct reviews his way.

Comment 9

[Joshua Cranmer [:jcranmer]]

Attachment: Sketch of approach

Brief sketch of what this would entail (I think this can read/write SSL data, but telnet -z ssl was balking for me about self-signed certs, so I didn't press too closely).

Things that are nowhere near solved:
1. Controlling which certificates get used.
2. Starting up ssltunnel.
3. Multiple connections. With this approach, we can't have two SSL connections in the handshake stage at the same time.

Potential solutions:
A. Start and shutdown ssltunnel every time we need to use it in a test via nsIProcess.
   Advantages: no changes to m-c
   Disadvantages: seems error-prone, need either one configuration file that
     handles every case or need to switch between several config files
B. Implement nsISSLServerSocket (with STARTTLS support, of course)
   Advantages: I think this gives us maximum flexibility, if ported to m-c,
     could be used by extensions
   Disadvantages: requires a lot of implementation and needs more SSL knowledge
     than I have
C. Modify ssltunnel to use IPC control instead of config files
   Advantages: solves config file issue
   Disadvantages: only good general IPC m-c has right now is sockets... and I'm
     also starting to get annoyed trying to keep track of streams (already up to
     6 per connection)
D. Use xpcshell to generate multiple processes for some tests
   Advantages: should be relatively easy to do with manifests, solves starting
     up ssltunnel
   Disadvantages: IPC issues still remain, m-c needs changes
E. Do #D and come up with a clean IPC in the process?
   Advantages: May fix IPC issues
   Disadvantages: have to come up with better IPC :-(

None of these options seem particularly appealing, but I think B is probably the best in the long run, although it's probably the most work.

Comment 10

[Joshua Cranmer [:jcranmer]]

tl;dr: The approach I have here falls hard, and I don't think I want to continue down this exact line. Layers may be worth saving, if we hardcode them. Finally, I don't think I'm going to continue doing work until there's a better plan in place for this sort of stuff.


Having some experience with the layer-based architecture now, here are my thoughts:

My original use cases were supporting SSL and LDAP's BER-conversion as layers. After thinking about it some more, I thought that IMAP's tagging and parsing conversion might benefit from being pulled out in a layer compared to the current hackish approach, and that perhaps I could test mail-over-proxy and charset concerns via layers as well.

Now, I'm not so sure. First off, writing a generalized asynchronous pump for the SSL layer that might work for other layers is just plain annoying. Much better, I think, to just have a single input/output stream to worry about and shove SSL off to some other management places (hence why I was wishing to opt for option B: if we can promote an XPCOM socket transport to SSL (as a server), we let NSS worry about the real input/output and we just get decrypted data).

The other thing is that it's hard to be general about layers when each one wants to pass data to and from in different ways:
* Proxy, SSL: communicate via async sockets
* charset: needs to do binary array<->JS string conversion (or malformed JS string<->true JS string)
* line buffering: could do either binary arrays or strings, but doing both depending on lower layers is annoying
* IMAP-parser wants to output to generic args
* LDAP-parser needs to output back to binary arrays, and may need buffering

Oh yeah, and everyone needs to communicate data back and forth to individual layers, particularly SSL (starttls, mainly), charset (don't want to translate message bodies! [actually, we may want to, if we want to test "evil" transcoders ;-)]), line buffering (is this line part of a command or not), and *-parser (how do we format the arguments?).

Is this worth saving? Perhaps, but its ambition needs to be tamped down dramatically. There's really not many levels that are actually needed: if we really want to test mail-over-proxy, we can either spin up httpd.js for that or use this proxyserver.py thingy I found laying around somewhere and ignore it in the maild.js stack (which might be a better real-world approximation anyways). Everyone else pretty much has to be in a specific order, although I'm having mental issues trying to figure out where to place charset conversion (probably after line buffering, since I can always change line buffering to output binary arrays instead of strings); at thatp oint, it's a matter of merely enabling or disabling one for a test, with perhaps some variability in the parser layer.

Multiple asynchronous communication in particular is something that I do NOT want to do: while I can easily handle the servers outputting data any old time they please, trying to figure out which stream wants me to send which data where via the layers approach is a maintenance nightmare. I've actually stored this as two distinct patches locally, where moving line buffering to a layer is the first patch and adding the SSL layer is the second one. The first patch is probably worth keeping, so it's the SSL+async layers patch that I'm considering trashing.


Anyways, this is about as far as I am willing to go for now. This is a proof-of-concept which appears to at least partially work (telnet found the certificate at least), so if someone feels compelled that this is absolutely the right way to go, they can charge on ahead. However, going any further would require a fair amount of investment in some architecture decision which I don't feel qualified to make. If we do end up fully using ssltunnel largely as it stands, furthermore, I want to wait until after fakeserver-ipc happens so that I have a clearer idea of what the implications of multiple processes in xpcshell are.

Comment 11

[Joshua Cranmer [:jcranmer]]

Comment on attachment 538438 [details] [diff] [review]
Sketch of approach

Bienvenu: could you tell me what your thoughts on this architecture design are, particularly a response to comment 9? (No real way to request feedback on a comment, so this proof-of-concept patch is the best proxy I have...)

Comment 12

[Joshua Cranmer [:jcranmer]]

Oh yeah, for what it's worth, bug 242448 has a prototype patch which basically allows server sockets to utilize socket providers and then adds a lot of work to get the socket provider for SSL stuff to support socket listening. I doubt I understand NSS well enough to update that patch and know how to use it (while it doubtless handles the generic case better, I don't see how to select specific certificates for testing).

The other implementation idea I had was to grab the PRFileDesc* for the socket and then make a layer which binds SSL server stuff onto that: lower level but easier to understand. That is impossible without modifying m-c, since sockets don't expose PRFileDesc* in any way I can get to it, so it's either duplicate nsServerSocket or modify m-c.

Comment 13

[David :Bienvenu]

Comment on attachment 538438 [details] [diff] [review]
Sketch of approach

One test for the layer strategy is - would it allow you to easily implement COMPRESS=DEFLATE in a layer? Along with SSL at the same time? See http://tools.ietf.org/html/rfc4978. Like STARTTLS, this would require a layer to be added after the connection is established.

Comment 14

[Joshua Cranmer [:jcranmer]]

It still seems to be doable via hard-coded layer ordering:

client <-> SSL <-> SASL <-> compression <-> line <-> charset <-> treat <-> server

Although it does bring up interesting edge cases: what if I did COMPRESS=DEFLATE, then SSL, then SASL (if we use, e.g., GSSAPI to sign/encrypt the entire layer as opposed to just auth).

Comment 15

[David :Bienvenu]

(In reply to comment #14)

> Although it does bring up interesting edge cases: what if I did
> COMPRESS=DEFLATE, then SSL, then SASL (if we use, e.g., GSSAPI to
> sign/encrypt the entire layer as opposed to just auth).

if I'm understanding you correctly, we would not do that (STARTTLS and GSSAPI would always come before COMPRESS=DEFLATE). But shouldn't you be able to adjust the order of those layers based on the commands received from the client?

Comment 16

[Joshua Cranmer [:jcranmer]]

Attachment: SSL without layers

I've given some more thought to layers over the past few days, and I've come to the conclusion that, while layers are a useful addition to fakeserver, they aren't well-suited for SSL, since SSL is sufficiently different and annoying that a generalization that handles SSL causes issues for other people. Thus this is an approach which uses ssltunnel but doesn't handle layers. This doesn't mean I'm abandoning it, it just means that I am going to move layer implementation into another bug where its power can be more readily harnessed, e.g., charset layers or compress=deflate.

The good news is I can confirm that this works, as in, I have carried on an NNTP conversation via telnet -z ssl and a modified qaserver that inits the SSL layer. The bad news is that this requires ssltunnel to be run outside of the process... and my investigation into how mochitest runs ssltunnel can basically be summarized as "it's thoroughly baked into the test framework and automation scripts."

Options:
1. Try to figure out how to invoke ssltunnel from xpcshell itself. I can probably do it fairly easily for regular testing environments, but I would need some hand-holding to figure out how to get it working in the packaged-test world.
2. Modify xpcshell framework to invoke ssltunnel. Apparently mochitest and xpcshell use different automation APIs, so this may not be much easier.
3. Screw ssltunnel and implement SSL server sockets. Fixes process invocation problems, but disables starttls (probably)
4. Screw ssltunnel and write a fakeserver-utils library that implements SSL by reaching down to NSS itself. This may require native code, although at this level I could possibly do it with jsctypes.

Comment 17

[(not currently active) Ted Mielczarek]

Honza has a patch in bug 466524 that implements running ssltunnel via xpcshell in order to test SSL in xpcshell unit tests.

Comment 18

[Joshua Cranmer [:jcranmer]]

Attachment: SSL, with STARTTLS support

Not quite ready for review yet, but I'd thought I'd mention the latest milestone: I now have both SSL connections and STARTTLS working and in tests. At this point, all that should be necessary is cleanup and submitting to try for testing, as well as untangling which patches this depends on (my patch queue is a bit long at this point).

Comment 19

[Joshua Cranmer [:jcranmer]]

I have a locally working approach, but it appears that the buildbots don't unpack ssltunnel to the bin/ directory, which prevents this from working on tinderbox...

Comment 20

[John Hopkins (:jhopkins)]

Buildbot try and try-unittest masters have this patch:

 http://pastebin.mozilla.org/1269650

When we're ready I can land the patch and apply it to the Buildbot production master as well.

Comment 21

[Joshua Cranmer [:jcranmer]]

Attachment: Part 1: Implement a layer-based framework for fakeserver

(Not strictly necessary for SSL support, but it makes doing this easier)

Comment 22

[Joshua Cranmer [:jcranmer]]

Attachment: Part 2: Implement SSL compatibility layer

This does the SSL. It has robustness issues due to raciness with process creation for ssltunnel, and I'm not 100% sure that the SSL certificate stuff is at all correct...

Comment 23

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.