Greylisting you ask? Isn’t that a dirty word Danita? Well, it certainly can be. In fact, in its “default” implementation, greylisting can be downright detrimental to the efficient and timely flow of email. For those of you who aren’t sure what I mean, let me explain.
Most of you are familiar with the ideas of “blacklisting” and “whitelisting”. Blacklists are to prevent any mail from a certain sender, domain or even mail server from being delivered to your mailbox or your users’ mailboxes. Whitelisting, on the other hand, does the opposite. It provides for the immediate delivery of all mail from a particular sender, domain or even mail server. So what’s this “greylisting” thing then. Greylisting is the practice of initially rejecting an email, as though the email server is unavailable (usually a generic 450 message, or a 451 message specifically indicating that the message is being delayed), and waiting for the sending server to send again. The theory behind greylisting is that most spam, viruses and trojans are delivered by mail servers or PCs that aren’t interested in specifically getting to “you”. They are just interested in spewing out as much junk as they can in the hopes that a certain percentage “sticks”.
So, then, what is the problem with greylisting overall? The biggest problem is time. It’s bad enough that someone in New York sends me an email, and immediately after clicking “send” picks up the phone to see if I’ve received it. I mean, it takes a COUPLE of minutes to get from their server to mine for goodness sakes. But what if every email sent to my mail server had to “wait” until the sending server tried again on its regularly scheduled “defer” cycle? Most servers are set to try after 20 minutes. By default that’s what GroupWise has also always done. Some servers can be configured to send at a shorter interval. Indeed GroupWise can do that as well. But if you leave most things at their defaults, you will find that 20-30 minutes delay for the first mail deferment if a mail server is down is quite standard. Thus, greylisting gets a really bad name because it just serves to delay the orderly delivery of our email.
So, some bright guy (reportedly SATOH Kiyoshi (http://k2net.hakuba.jp/rgrey/, in Japanese) came up with the idea that not only is most “junk” sent from mail servers and PCs that aren’t really interested in “who” they hit (just that they want to hit a high enough percentage of their recipients), but also that most “junk” is sent from a certain type of mail host. This form of greylisting suggests seven rules to test for whether or not a mail host is “suspect”. These rules are based on the idea that most of these hosts will either not have a reverse DNS look up at all, will have one that doesn’t “match”, or will have a reverse DNS entry that seems to indicate the sender is on a dynamically assigned address. You can find out more about these rules here. If you could use these rules in greylisting to only greylist senders that are caught by one of these rules, well defined and maintained mail servers would never be trapped by the greylisting mechanism (thus ensuring that most “real” mail would be processed immediately), and only those that were “suspect” would be delayed. I’ve been using this mechanism for almost a year now in my own system. Here are some stats from one of the servers that I manage that show you how this works:
Attempted Connects : 1168385 ( 100.0% )
Invalid (No Such Recipient) : 811684 ( 69.4% )
Valid (Recipient Confirmed) : 356701 ( 30.5% )
Total Not Delivered Mail : 299783 ( 25.6% ) ( 84.0% )
Not Delivered – Spam : 5524 ( 0.4% ) ( 1.5% ) ( 1.8% )
Not Delivered – Virus : 82 ( 0.0% ) ( 0.0% ) ( 0.0% )
Not Delivered – Banned : 1 ( 0.0% ) ( 0.0% ) ( 0.0% )
Not Delivered – BadH : 68 ( 0.0% ) ( 0.0% ) ( 0.0% )
Not Delivered – GreyList : 294108 ( 25.1% ) ( 82.4% ) ( 98.1% )
GreyList Validations : 1775 ( 0.6% )
Total Delivered Mail : 56918 ( 4.8% ) ( 15.9% )
Unique Mail Recipients : 1236 ( 100.0% )
The important thing here (although it’s all fascinating) is that of 294108 greylisted entries, only 1775 attempted a second delivery! Now, is it possible that some real mail gets “dropped” in this? It would be possible, but not terribly probable. There are very few mail servers out there who will not retry an email on a 450 or 451 error. And in over a year of using this type of greylisting, I’ve only had to whitelist one IP address to avoid the greylisting process.
So, now I know your next thought is “But I use GroupWise, and we have GWAVA (or Guinevere, or GWGuardian, or whatever), and I have no idea how I would do something like this”. It’s easy to forget that Linux is our friend! I’ve put up Linux PCs (or even Virtual Machines) all over the place as front-ends to the GroupWise Internet Agent to provide this functionality for customers. This doesn’t have to be a “server” either – just plain old vanilla Linux with Postfix installed! There is a Postfix add-on called Postgrey that can provide greylisting services for postfix, and it’s very easy to configure. In fact, to get it to work for this modified greylisting, you can get all of the information in this particular newsgroup post.
Imagine how much more processing power you could give back to GWGuardian, or GWAVA or Guinevere if you dropped 98% of the mail that it’s processing now before it even has to look at it!
Effective greylisting really depends on the other mail server properly implementing timeouts and retries.
One of my new correspondents was taken aback that his mail hadn’t been delivered. I checked my logs, and it had been deferred by the greylister. Unfortunately, his system didn’t retry again for another week! Of course, his complaint was that my anti-spam was too aggressive, but methinks his mail server was a bit too passive…
Yes, this is very true, and of course if someone’s mail server isn’t configures properly for their timeouts and retries, they also might not be configured properly for reverse DNS. However, I think there comes a point where we must put the onus on the sending server. As I said, I DID have to whitelist one server in the past year to avoid greylisting issues. That said, the post office is unlikely to deliver mail that is misaddressed or has no postage on it.
This is what the Deep Six spam filter appliance does. When it doubts, it says wait, then send again to the offending mailer. Spam mailers never send again and in this way the Deep Six box stops all spam but lets the regular mail get through.
You can put this box in front of your Groupwise box. It is a nice solution for people who are not able to build something themselves with Linux.
does anyone knows if there is any other information about this subject in other languages?
If you’re not familiar or comfortable with Linux, there is a Windows solution that is free, providing you have Windows. There is a product called SmarterMail. It runs on Windows and will provide a full mail solution if you want or you can just use it as an MX or relay host. It allows for grey-listing, spf testing and several other anti-spam measures. We’ve been using this for several years and our results are just like yours. In that entire time I’ve had to white list 1 ip address due to a misconfigured mail server on the other end but it was the CEO’s personal accountant.
I have gone to your chats at Brainshare for many years, I currently have a Groupwise 7.0.3 installed on my NOWSSBE2 and it works great, but recently I have getting 550 errors on some emails and in yahoo it gets but into bulk, that´s right my Groupwise 7.0.3. I think the issue is greylisting…
change needed to AllowOverrides None to AllowOverride None. The former will throw an error when restarting Apache.
First thing I’d triple check is AV scanning. Also ensure that Backup software hasn’t locked the files.