Checking your domain’s DKIM, SPF and spam record status for outgoing SMTP mail

Here is a brief tutorial how to check if your domain’s SPF and DKIM spam protection measurements are configured correctly. These domain name records are used to prevent spammers sending email and forging the sender’s address to be your domain.

This blog discussed only about testing there measurements; there are plenty of online tutorials available how to setup the protection for your domain.

1. Checking your domain’s SPF and DKIM records, SpamAssassin score

You can check DKIM signature online using this great emailtest service by Brandon Checketts. You will get a random email address where you send in a test email and the results of the email delivery are shown online.

port25 solutions provides an alternative service where you can send inbound email and it will return the SPF and DKIM verification status back in the return message. Getting a reply from this service takes like 30 minutes.

Below is an example how to send out a test email from Django shell. Use the related method of your web framework sending out the test email. For plain UNIX you can always use command line mail command to send out a test email from your shell.

from django.core.mail import send_mail
send_mail("test subject", "test message", from_email="mikko@example.com", recipient_list=["example@www.brandonchecketts.com"])

Then you see the output online on the site above, like SpamAssassin results:

SpamAssassin Score: -2.011
Message is NOT marked as spam
Points breakdown: 
-0.0 RCVD_IN_DNSWL_NONE     RBL: Sender listed at http://www.dnswl.org/, low
                            trust
                            [198.2.128.3 listed in list.dnswl.org]
-0.0 SPF_HELO_PASS          SPF: HELO matches SPF record
-0.0 T_RP_MATCHES_RCVD      Envelope sender domain matches handover relay
                            domain
-0.0 SPF_PASS               SPF: sender matches SPF record
-1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
                            [score: 0.0000]
-0.1 DKIM_VALID_AU          Message has a valid DKIM or DK signature from author's
                            domain
 0.1 DKIM_SIGNED            Message has a DKIM or DK signature, not necessarily valid
-0.1 DKIM_VALID             Message has at least one valid DKIM or DK signature
 0.0 MSGID_FROM_MTA_HEADER  Message-Id was added by a relay

Here is yet another SPF Policy Tester where you can enter your SMTP server IP address and domain name.

2. Testing forged email send

You also need to test  that email receivers (GMail, Yahoo, HotMail) actually filter out messages based on the SPF test. This can be easily done by sending forged emails using the SMTP server of your local ISP. Telnet to SMTP server directly and then type out SMTP commands by hand.

Here is an example SMTP chat with a Finnish ISP’s outgoing SMTP server.

telnet posti.example.com 25

EHLO mikkos-imac
MAIL FROM: <mikko@sender.example.com>
RCPT TO: <mikko9999999@gmail.com>


DATA

test

.

3. GMail and SPF records

GMail does not seem to take spam decisions solely on either SPF hardfail -all or softfail ~all test. Thus, it seems to be very difficult to make GMail to discard spam and phishing attacks send in your domain’s name. Here is an example message from the above, as received by a GMail, which is delivered to inbox regardless of the domain SPF settings.

   
Delivered-To: mikko@receiver.example.com
Received: by 10.14.22.4 with SMTP id s4csp70004ees;
        Wed, 19 Feb 2014 00:49:41 -0800 (PST)
X-Received: by 10.43.65.145 with SMTP id xm17mr25876368icb.35.1392799780856;
        Wed, 19 Feb 2014 00:49:40 -0800 (PST)
Return-Path: <mikko@sender.example.com>
Received: from gw03.mail.visulahti.fi (gw03.mail.visulahti.fi. [1.1.1.1])
        by mx.google.com with ESMTPS id sc10si873225igb.31.2014.02.19.00.49.37
        for <mikko@receiver.example.com>
        (version=TLSv1 cipher=RC4-SHA bits=128/128);
        Wed, 19 Feb 2014 00:49:40 -0800 (PST)
Received-SPF: softfail (google.com: domain of transitioning mikko@sender.example.com does not designate 1.1.1.1 as permitted sender) client-ip=1.1.1.1;
Authentication-Results: mx.google.com;
       spf=softfail (google.com: domain of transitioning mikko@sender.example.com does not designate 1.1.1.1 as permitted sender) smtp.mail=mikko@sender.example.com
Received: from mikkos-imac (a91-154-153-67.pulina-laajakaista.fi [1.1.1.2])
    by gw03.mail.visulahti.fi (Postfix) with ESMTP id D639D216ADF
    for <mikko@receiver.example.com>; Wed, 19 Feb 2014 10:49:13 +0200 (EET)
Message-Id: <20140219084922.D639D216ADF@gw03.mail.visulahti.fi>
Date: Wed, 19 Feb 2014 10:49:13 +0200 (EET)
From: mikko@sender.example.com
To: undisclosed-recipients:;

Test mail

If anyone knows how to make GMail to honor SPF, or to ignore forged sender messages, please let me know.

 

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

Using Postfix and free Mandrill email service for SMTP on Ubuntu Linux server

Update: This blog post is out-of-date. Mandrill recently changed their policy with changes that some of the service users may not approve. See alternatives.

OLD BLOG POST GOES HERE:

This blog posts discuss using Postfix mail server software and Mandrill SMTP service to send email out from your Linux server and web applications.

Getting a good outgoing email service for your self-hosted small business web application is difficult. Mandrill service addresses this issue; Mandrill is outgoing email service provided by Mailchimp email newsletter service – so they have all the money, skill and interest keep their SMTP server working and spam blacklist free.

In this blog post we discuss how to use Mandrill with Postfix SMTP daemon on Ubuntu Linux. Please note that this blog considers only Ubuntu Linux software stack where you have full control. If you run crippled deployment environment where you cannot install software, like Google AppEngine or Heroku, Mandrill offers HTTP API for sending email.

You may also want to see earlier blog posts Using Nullmailer and Mandrill app for sending email on Ubuntu servers and Sendmail using GMail and Nullmailer. However, approach described here is superior and recommended over the former alternatives. Specifically Exim4 and Sendmail are inhuman difficult to configure, MSMTP and Nullmailer do not offer localhost SMTP server.

1. About Mandrill

Screen Shot 2013-03-25 at 12.24.32 PM

Mandrill is

  • Real SMTP server with SMTP and HTTP APIs
  • 12k free emails per month – very generous from them
  • The owner has clear interest keeping it spam blacklist free (Mailchimp, their main product, is a newsletter service)
  • The web admin interface is easy and powerful
  • Mandrill supports converting all links in emails automatically trackable

Sidenote: when dealing with setting From: address in email make sure the domain records of the from email address whitelist the outgoing email server IP addresses via SPF records.

2. Setting up Postfix with Mandrill

The benefits of using Postfix include

  • Relative easy configuration
  • Local email queue on the server
  • SMTP service at localhost port 25. If you are hosting multiple web applications, like Plone CMS, they have configuration panel where to set SMTP server details. Postfix handles centralized SMTP upstream credential management and local queue for your outgoing emails. Postfix approach is more easier to migrate if you change SMTP servers: you change your ourgoing email service credentials just in one place
  • Some older web applications cannot do SMTP SSL/TLS authentication out of the box and thus cannot negotiate with Mandrill server (e.g. Plone 3.x, WordPress). Postfix acts as middleman  for them.

Screen Shot 2013-03-26 at 4.25.04 PM

Install Postfix on Ubuntu 12.04 LTS:

sudo apt-get install postfix mailutils libsasl2-2 ca-certificates libsasl2-modules

You may or may not popped up for Postfix setup wizard in this point. If it doesn’t happen manually run dpkg-reconfigure postfix after apt-get. When the setup wizard asks any questions choose

  • Internet site
  • Add fully qualified hostname for your server (e.g. if your host is foobar, FQDN is foobar.example.com)
  • Otherwise use the default settings

Screen Shot 2013-03-26 at 4.26.27 PM

Then add the following lines to file /etc/postfix/main.cf:

relayhost = [smtp.mandrillapp.com]
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_use_tls = yes

# Make sure Postfix listens to localhost only
inet_interfaces = 127.0.0.1

Set up your Mandrill credentials for Postfix:

echo "[smtp.mandrillapp.com]    MANDRILL_USERNAME:MANDRILL_API_KEY" > /etc/postfix/sasl_passwd
postmap /etc/postfix/sasl_passwd

You can create your API key on Settings page on Mandrill website.

Install SSL certificates, so Postfix trusts smtp.mandrillapp.com (might not be supplied on Ubuntu server by default):

cat /etc/ssl/certs/Thawte_Premium_Server_CA.pem | sudo tee -a /etc/postfix/cacert.pem

Restart Postfix

service postfix reload && service postfix restart

3. Testing email out

Below is the command line to send some mail to yourself:

echo "This is a test message from ${USER}@${HOSTNAME} at $(date)" \
  | sendmail mikko@example.com

Check the system mail queue:

mailq  # Should be empty

You can see Postfix logs for possible detailed error reports:

tail -f /var/log/mail.log

See that Postfix answers in localhost port 25:

telnet localhost 25  # Write crap to the SMTP port until Postfix terminates the connection

Also check Outbound > Activity logs in Mandrill web interface.

4. More information

 

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

Sendmail using Nullmailer and Gmail account on Linux server

When you run VPS servers or other “low end” boxes it is common scenario need to setup a outgoing email mail transfer agent to get reports from cron jobs or enable PHP scripts to send email.

Linux offers some really heavy weight options for sending email, including Sendmail, Postfix and Exim4. However, if you are a low end box user, you run low end services of which requirements are likely to send one email per month from your puny cron job or web contact form. This makes the options above

  • Plain scary
  • Very difficult to configure, maintain, due to extra complexity not necessary for your use case
  • Open some attack surface
  • You still need a master SMTP where you push forward your email messages

They don’t scale down.

Meet Nullmailer. Nullmailer is a simple MTA (mail transfer agent) and sendmail command provider which simply dumps your email output forward to SMTP service.

But that’s not all. If you are using low end boxes you probably don’t need good SMTP service from your VPS provider  (spam makes them expensive to provide). So, here is the trick: you are probably using already a free web email provider like gmail, Hotmail or whatever happens to be the dominating internet company of the day. Nullmailer can be configured to use authenticated SMTP sending through any email provider quite easily.

Below is a setup script for Ubuntu / Debian (tested on Ubuntu 12.04 LTS) which setups xinetd tunnel to connect to GMail’s smtp.google.com and send emails as your authenticated Gmail user. The orignal idea is described in this blog post by Jon Spriggs.

The downside of this hacky solution include

  • No real email From addresses: all email is aliased to your authentcated SMTP user
  • SMTP user password is stored on the server as plain text, so you probably want to create a rogue gmail user for this
  • No locally buffered email (not sure about this with Nullmailer?) – if SMTP acts strangely be prepared for major screw up
  • Using sendmail command with this solution may cause delays and other issues in the script (not sure how timeouting is handled with Nullmailer?)

The script is also available on Github, hosted inside ZtaneSH project. Please contribute your changes back on Github in a case you come across of improvements.

Nothing else is needed, except running the script as instructed.

#!/bin/bash
#
# Setup nullmailer on Ubuntu using your Gmail account as SMTP
# - you get a working sendmail command without requiring to setup complex SMTP
# stuff. Mostly useful with cron scripts.
#
# Orignal script based on https://jon.sprig.gs/blog/post/9
#

# Abort script if any command fails
set -e

#
# Create a gmail SSL wrapper script
#

if [ -z "$GMAIL_USER" ] || [ -z "$GMAIL_PASSWORD" ] || [ -z "$TEST_ADDRESS" ] ; then
    echo "Setup sendmail via gmail proxy on your Ubuntu box"
    echo "Usage:"
    echo "GMAIL_USER=foobar@gmail.com GMAIL_PASSWORD=12312312 TEST_ADDRESS=youremail@example.com sh setup-nullmailer.sh"
    exit 1
fi

# install required software
sudo apt-get install -y openssl xinetd nullmailer

sudo tee /usr/bin/gmail-smtp <<EOF >/dev/null
#!/bin/sh
/usr/bin/openssl s_client -connect smtp.gmail.com:465 -quiet 2>/dev/null
EOF
sudo chmod +x /usr/bin/gmail-smtp

#
# Create xinetd.d entry which wraps SMTP traffic to port 10025 go
# go to gmail
#

sudo tee /etc/xinetd.d/gmail-smtp <<EOF >/dev/null
# default: on
# description: Gmail SMTP wrapper for clients without SSL support
# Thanks to http://ubuntuforums.org/showthread.php?t=918335 for this install guide
service gmail-smtp
{
    disable         = no
    bind            = localhost
    port            = 10025
    socket_type     = stream
    protocol        = tcp
    wait            = no
    user            = root
    server          = /usr/bin/gmail-smtp
    type            = unlisted
}
EOF
sudo /etc/init.d/xinetd reload

#
# Set nullmail to use xinetd
#
sudo tee /etc/nullmailer/remotes <<EOF >/dev/null
127.0.0.1 smtp --port=10025 --user=$GMAIL_USER --pass=$GMAIL_PASSWORD
EOF
sudo /etc/init.d/nullmailer reload

# send test email
echo "This is a test message from ${USER}@${HOSTNAME} at $(date)" | sendmail $TEST_ADDRESS

echo "Test mail send to $TEST_ADDRESS"

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+