A while ago I got Courier IMAP working with SSL, so for that part at least I no longer needed my ssh tunnel. But being able to use my mail server as an SMTP relay (without completely opening it up to the world) is another matter. So in a comment in Goodbye UW-imapd, hello Maildir and Courier, I whined for someone to figure out how to easily get an SMTP server--Sendmail, Postfix, whatever--set up with authentication over SSL. I'd looked at this in the past with sendmail and it seemed like a lot of work, e.g. you have to generate all these certificates with OpenSSL, there's this add-on package to compile in and set up called SASL, etc, etc.
Inspired by my own whining, I recently bumped this up to a higher slot on my todo list, and finally got it all up and running.
First, a comment on OpenSSL: for some reason using OpenSSL is far more confusing than it really needs to be. It seems no two resources on the web describe the same way to accomplish anything, which is a shame, because there are really only a handful of tasks you ever need to do, but given the plethora of options, wrappers, config files, etc., it gets to be a bit overwhelming. But it helps to keep thinking at the high-level: at the heart of it, SSL is just a mechanism that uses public key encryption: given a pair of keys, one key can decrypt messages encrypted by the other, and vice versa. The keys are not identical, they are specially generated, mathematical complements of one another. You keep one key absolutely private (your "key"), and make the other public (your "certificate"). Swapping certificates is enough to allow private communication between two parties, e.g. for services like HTTP, SMTP, or IMAP. A certificate authority (CA) is nothing more than another guy with yet another key pair, but you keep his certificate in your browser's (or computer's) trusted key ring, and you inherently trust any certificates he's signed to be who they say they are. This allows you to be certain of who you're talking to (e.g. so you don't hand over your credit card info to someone falsely claiming to be amazon.com). Web browsers come with a number of CA's certificates built-in, like Verisign, who charge lots of money to sign people's certificates and guarantee they are who they say they are.
The steps I took were as follows: I created my own CA and copied its certificate to my laptop (and all other computers I had that would be using my mail server). Then, I generated a new certificate and key for the host running the mail services. Finally, I had the CA sign the new certificate. Voila--private, trusted communication. Granted, figuring out the exact OpenSSL commands to do this took some effort to figure out, but that's it at a high level. It almost seems like various installer scripts (e.g. Courier, FreeBSD's ports system) do us a disservice by hiding the details of all of this by handling it automatically, generating self-signed keys that are only useful for that one service, etc.
Next, SASL is an abstraction library for adding authentication to connection oriented services (despite SASL being one letter different than SSL, the two have nothing to do with each other). Last time I looked at it I didn't particularly like it because it looked like you needed to maintain yet another password database on the system, with its own commands for adding passwords, yet another password to keep in sync, etc. What I found out this time is that it can tie into PAM, and thus simply use your system's /etc/passwd database for authentication. Of course, with Unix's hashed password scheme you must have the user's password in clear text to authenticate, so using SASL this way requires cleartext transmission of the password--and that's why we wrap the SMTP connection inside SSL to keep it private (and thus I suppose the appeal of a separate SASL password database, there are auth schemes such as CRAM-MD5 where the password can be sent encrypted, eliminating the need to wrap it in SSL. But I'll take having the whole SMTP conversation wrapped inside SSL for the occasional privacy gained when the other SMTP server supports SSL (which I've found in my logs is surprisingly often).).
Finally, just to add one more item to the mix, while setting this up I decided it was finally time to make the switch from Sendmail to Postfix. Now I like sendmail, I've used it for years, I've written some pretty complex sendmail rulesets, and at times enjoyed the flexibility it has given me. But I had to stop and think about the last time I actually did anything like this and I realized it's been at least 5 years. Over the years I've seen good things about Postfix--mainly, the attraction of fewer exploits than sendmail and a simpler configuration. So, now that I have Postfix set up, I agree: it's simpler to configure, and seems to be working great. The only "bad" thing about Postfix at the moment is the SSL STARTTLS code isn't in the main Postfix distro, you have to patch it (not really a big deal, but the confidence factor would be higher if it were). I'd write more about it, but I really haven't had to mess around with it enough to rant or rave about anything in particular, which I think speaks highly of it.
So it's been a little bit of work to get all this going, but I now have my own CA set up, have all my clients trusting this new CA, and have Courier and Postfix offering IMAP and SMTP over SSL (using the same certificate/key pair). I love it and it works great! Now mail on my laptop works the instant it comes out of sleep. Of course, the devil's in the details, so here are a few resources I found very helpful out there. Good luck!
- Deploying Postfix on Solaris * Excellent
- Postfix with SASL Authentication over TLS (Errata)
- Postfix + TLS + SASL on FreeBSD
- Postfix/TLS - A TLS extension for POSTFIX
- Importing self-signed SSL certificates into a Mac OS 10.3.x system (be sure to read the comment "Even Easier way to manager Certificates" for how to use Keychain Access to do this)