Goodbye UW-imapd, hello Maildir and Courier
Mon Aug 25th 2003, 5:37pm
I've been having some serious problems with my work and home IMAP clients simultaneously accessing mail on my server at home--they would constantly fight, lose their connections, and give me grief when sending mail, often resulting in messages being sent repeatedly. This has been an ongoing annoyance, and recently has been an increasing source of embarrassment when friends and colleagues send me replies such as "umm hi, I got 2 surplus copies of this btw".

It looks to me that the venerable UW imapd is no longer capable of servicing multiple clients simultaneously to the same mailbox, which is weird, because I've done this very thing with it for years using what I believe is essentially the same setup as I have now. It's hard to imagine the point of making an IMAP server that doesn't properly implement this very ability that makes IMAP so desirable in the first place. But... many things have changed, I have new computers, different mail clients, etc, etc, who knows really what the problem is anymore.

So after months of putting it off, and having hit yet another rather embarrassing duplicated email incident in the last few days, I finally bit the bullet and decided it was time to fix this problem. Here is my previous setup, a standard, time-tested configuration:
  • sendmail
  • mbox-formatted mailboxes
  • uw-imapd
When looking at solutions, I wanted something that would require a few changes as possible--i.e., I don't want to bounce any mail or end up debugging ongoing problems. I thought I'd found a reasonable solution in an imap server called Dkimap, which supported mbox mailboxes, but my mail clients seemed to have some weird problems with it. That left Courier and Cyrus. Cyrus looked powerful, but way overkill for my needs. Maybe I'll consider it next time I deploy an enterprise solution from scratch :-). Finally, Courier looked lightweight and clean--and I've been hearing positive reports about it for a long time--but unfortunately it requires mail to be stored in maildir format.

Now, while I'd rather not switch to maildir, I wasn't completely opposed to it, but even if I did it presents the next question of how do I get my mail system to deliver to maildir mailboxes instead of mbox? Qmail and Postfix can do it, sure, but I didn't really want to change out my MTA since 1. I've been using sendmail forever and of course 2. this would be a big pain. Plus 3. based on prior experiences I don't really like Qmail. On the flip side, Postfix looks pretty cool, nice and lightweight--maybe I'll look at it again sometime in the near future. But what finally occurred to me was that Procmail could deliver to maildir mailboxes--and in fact I was already using it! So here is the improved setup:
  • sendmail
  • procmail
  • maildir-formatted mailboxes
  • courier imapd
Sendmail pipes incoming mail to procmail. I changed my procmail recipes to deliver to different locations (under ~/Maildir now instead of ~/mail) and to deliver using maildir format (by providing trailing '/'s at the end of the foldername). I converted my mbox mailboxes to maildir using a script I found on the web (beware, there are some busted ones out there). Courier finds the mail, and serves it up.

One week later: so far it's been working absolutely great. I can finally access my home email simultaneously from work and home and the two clients don't interfere with each other. In fact, as far as I can tell, they get along great. The problem with sending duplicate emails has gone away. And finally--an unexpected bonus--it's a LOT faster.

So thanks for many years of service, UW imapd, but I've had to move on.

Visitor comments
On Thu Apr 29th 2004, 8:07pm, erikburrows posted:
I've been using the IMAP server bundled with RedHat 7.3 for years. It's probably UW... It's been terribly slow, and I've had the same problem with concurrent access. I just finished switching to Courier. I used perfect_maildir to convert, and everything went very smoothly. (Once I figured out subfolders were named Maildir/.foldername) Now, instead of my over-taxed server taking up to a minute to load a message, it's almost instantaneous. Thanks for the inspiration Steve!

On Fri Apr 30th 2004, 10:14am, erikburrows posted:
Followup, 12 hours later: Still fast, still good, but one downside: Courier IMAP-SSL does not seem to be compatable with Evolution! Grrr... I Really wanted IMAPS for my laptop, when I'm consulting, or going over the air waves. So, I've been playing with stunnel. It works fairly well, for my workstation, but crashes Evolution on my laptop. I'll get it worked out eventually, but why does everything have to be so difficult?!

On Fri Apr 30th 2004, 12:12pm, Steve Kehlet posted:
Glad I could be some inspiration... :-) Courier has been working well for me since I installed it.

I also had problems getting Evolution and courier to work over SSL. I found that the default setup, SSL using STARTTLS, did not work. It's some kind of evolution problem. I googled and googled, but to no avail. What did work (and is currently working) for me is instead of using STARTTLS, enable and use imaps on port 993 (IMAPDSSLSTART=YES). Then in Evolution, I set my imap host to <hostname>:993, and Use Secure Connection Always to force it to use imaps instead of starttls.

On Fri Apr 30th 2004, 12:24pm, Steve Kehlet posted:
> I Really wanted IMAPS for my laptop, when I'm consulting, or going over the air waves.

Of course, you could always tunnel your imap and smtp ports over ssh, and set your mail client to send and receive mail from localhost. This avoids the relative nightmare of getting imap and smtp (with the obligatory smtp authentication to avoid opening up a mail relay) over ssl working. For laptops this blows however since every time you turn on your laptop you have to (re)start your ssh connection. We've got imap over ssl figured out--if you or anyone else here figures out an easy way (keyword: easy) to get sendmail or postfix to use ssl and authenticate, please let me know. It's been on my todo list but last time I looked at sendmail it was an ocean of effort to get this to work :-(.

On Fri Apr 30th 2004, 12:56pm, erikburrows posted:
That worked! IMAPDSSLSTART=YES, IMAPDSTARTTLS=NO and TLS_PROTOCOL=SSL2. No more ugly hack with stunnel! I love encryption. Thanks Steve!