Eliminating Spam

----------------- Picture a rainbow line here --------------------

My email addresses are well enough known that I'm on just about every spam list known to man and aliens. In the first 27 days of January, 2002, my address has been sent 4135 pieces of spam, about 153/day.

Update:

July 27, 2002
Spamassassin has filtered 8807 messages in the last month, while only allowing 400 through making it 96% effective!
May 13, 2003
Spam to me is up to 1500-2000/day --- a ten-fold increase over a year and a half ago. Spamassassin is weeding that out to about 10/day...
February 5, 2004
Since installing greylisting last summer, the load has been reduced dramatically! Typically between 50000 and 80000 messages/day are blocked before even getting into the system. I'm down to 150 spams/day to my own account, about 1/2 to 2/3 then filtered by spamassassin (which unfortunately is not as effective as it used to be as the spammers are devising ways around it).
March 18, 2004
I've graphed the spam load for the last year or so at Spam on Agora.

Fortunately, my mailer, mutt, makes it very easy to tag and delete spam (which is usually very obvious to human eyes), but still, I needed a way to cut down the deluge of crap, as real mail was getting buried in it.

The first step is avoiding getting on spam lists in the first place. Some good ideas on how to do that can be found at SpamProof.

SpamAssassin is the solution. It's a rules-based filtering mechanism: when a message comes into an account with spamassassin configured, the message is forwarded to a daemon process that looks at keywords and phrases, where the message came from, if it's been reported as spam to various databases, or the sender has, etc. Each rule has a score associated with it, and if, at the end, the score is greater than 5, the message is tagged as spam. The message is usually deleted, but depending on how you have things set up, it could be saved or delivered with extra headers marking it as spam. Regardless, a new header is added that shows you how a message rated (examples below).

If you want to dramatically reduce the sewage infesting your inbox, then go to the Account Management page and login. There, with a simple checkbox and Update, you can enable the filtering and reduce the din to a whisper...

Another tool that is available on agora is Active Spam Killer. This is so annoying to receivers that I don't recommend it however. Although I'll leave it on the system for existing users, I will no longer support it. The reason it's annoying is because it's a challenge/response system, like SpamCop: if someone sends you mail and they've not been seen before, the message is queued and a confirmation message is sent to them. If they reply to it, the original message is delivered and they're whitelisted so they don't have to confirm again for a month.

SpamAssassin header examples

Since SpamAssassin is the main spam fighting tool here, here's how to interpret what you'll see in your email after SpamAssassin has evaluated it:

Ham (non-spam) gives you an indication of how close it came to being filtered:
X-Spam-Status: No, hits=3.1 required=5.0
        tests=BAYES_80,HTML_50_60,HTML_FONT_COLOR_BLUE,
              HTML_FONT_COLOR_RED
        version=2.53
X-Spam-Level: ***
X-Spam-Checker-Version: SpamAssassin 2.53 (1.174.2.15-2003-03-30-exp)
Spam gives you a lengthy report about why it was considered spam:
X-Spam-Flag: YES
X-Spam-Status: Yes, hits=21.5 required=5.0
        tests=BAD_CREDIT,BAYES_80,CLICK_BELOW_CAPS,DATE_IN_FUTURE_03_06,
              HTML_50_60,HTML_FONT_COLOR_RED,HTML_LINK_CLICK_CAPS,
              HTML_LINK_CLICK_HERE,MIME_HTML_ONLY,MISSING_MIMEOLE,
              MORTGAGE_BEST,MORTGAGE_PITCH,MORTGAGE_RATES,RCVD_IN_NJABL,
              RCVD_IN_OPM,RCVD_IN_RFCI,REMOVE_PAGE
        autolearn=spam version=2.53
X-Spam-Level: *********************
X-Spam-Checker-Version: SpamAssassin 2.53 1.174.2.15-2003-03-30-exp
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----------=_3EC0A287.B453720D"

[-- Attachment #1 --]
[-- Type: text/plain, Encoding: 8bit, Size: 2.2K --]
Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

This mail is probably spam.  The original message has been attached
along with this report, so you can recognize or block similar unwanted
mail in future.  See http://spamassassin.org/tag/ for more details.

Content preview:  g emvxhkezinykbn czi gq unap vmyjbv ol ejrysxmvvkmb
  rqgfhztzyfjemz n g ku caw ikb wpm ejkhsol DON'T MISS THE LOWEST
  MORTGAGE RATES IN HISTORY! [...]

Content analysis details:   (21.50 points, 5 required)
MORTGAGE_PITCH     (0.6 points)  BODY: Looks like mortgage pitch
MORTGAGE_RATES     (0.5 points)  BODY: Information on mortgage rates
BAD_CREDIT         (0.5 points)  BODY: Eliminate Bad Credit
MORTGAGE_BEST      (1.7 points)  BODY: Information on mortgages
CLICK_BELOW_CAPS   (0.5 points)  BODY: Asks you to click below (in capital
+letters)
BAYES_80           (2.8 points)  BODY: Bayesian classifier says spam probability
+is 80 to 90%
                   [score: 0.8596]
HTML_FONT_COLOR_RED (0.1 points)  BODY: HTML font color is red
HTML_50_60         (0.1 points)  BODY: Message is 50% to 60% HTML
HTML_LINK_CLICK_CAPS (1.1 points)  BODY: HTML link text says "CLICK"
HTML_LINK_CLICK_HERE (0.1 points)  BODY: HTML link text says "click here"
REMOVE_PAGE        (0.3 points)  URI: URL of page called "remove"
DATE_IN_FUTURE_03_06 (1.5 points)  Date: is 3 to 6 hours after Received: date
RCVD_IN_NJABL      (1.2 points)  RBL: Received via a relay in dnsbl.njabl.org
                   [RBL check: found 135.105.187.61.dnsbl.njabl.org., type:
+127.0.0.9]
RCVD_IN_RFCI       (6.0 points)  RBL: Received via a relay in
+ipwhois.rfc-ignorant.org
                   [RBL check: found 135.105.187.61.ipwhois.rfc-ignorant.org.,
+type: 127.0.0.6]
RCVD_IN_OPM        (4.3 points)  RBL: Received via a relay in opm.blitzed.org
                   [RBL check: found 135.105.187.61.opm.blitzed.org., type:
+127.1.0.4]
MIME_HTML_ONLY     (0.1 points)  Message only has text/html MIME parts
MISSING_MIMEOLE    (0.1 points)  Message has X-MSMail-Priority, but no X-MimeOLE

The original message did not contain plain text, and may be unsafe to
open with some email clients; in particular, it may contain a virus,
or confirm that your address can receive spam.  If you wish to view
it, it may be safer to save it to a file and open it with an editor.