logo
my pic [home] [shared news] [resume] [bridge] [publications] [software] [about

Background

Reading E-mail has become an activity that many people have grown used to over the years. Often it is the first thing you do when you wake up, and the last before you go to bed. We've grown so used to it that we want to be able to read our E-mail any time, any place, anywhere.

This, however, is where it gets tricky. When using a conventional mail client, which moves E-mail from the mailserver to a local PC using the POP (Post Office Protocol) protocol, this can be a little tricky at times. This is why the IMAP (Internet Mail Access Protocol) was invented. The trick, then, is to tell your mail client on which server it can find your mailbox and read the messages directly from the server. This way you can always access your mail if you have a IMAP-enabled mail client. Another solution, of course, is to use Webmail.

In this howto I take a different approach. I like the power of UNIX and have grown attached to my mail client (mutt). I simply log into my UNIX-machine over a secure SSH-connection and fetch the mail to this machine. Since I can access my machine at all times I have a 24/7 access to my mailbox.

A second problem that many of us face is the fact that E-mail is insecure in the sense that anyone who puts in a little effort can intercept and read E-mail messages. For typical "hello how are you today" messages this may not be a problem. When telling someone your deepest secrets, or sending someone secret information (such as passwords) this may be very annoying to say the least. Luckily, tools like GnuPG (an implementation of the PGP-protocol) may help. In this system you can encrypt E-mail to ensure that only the addressee can read it.

In this howto I set out to describe how to set this all up. The first step is to configure fetchmail and maildrop (note: previous versions of this howto used procmail) so that the mail can be transported from the mailserver to a local UNIX-machine. Step two is to configure your mail client. In this howto I use mutt. I also show how to configure gnupg and demonstrate how this can be integrated in mutt.

Basic Mail

Some definitions:

MTA: Mail Transport Agent
This is the software that moves E-mail from machine to machine. In our case, we want to move E-mail from a server to our local machine. For this purpose we use fetchmail.
MDA = Mail Delivery Agent
Once mail arrives at your local machine it should be delivered at your mailbox. In this case we use "maildrop" for this. Another nice property of malidropis that you can make filters with it. This, however, is only partially covered in this howto.
MUA = Mail User Agent
This is the software that you use to actually read your mail: mutt.

Fetchmail / Procmail

Before you can do anything you need to create you maildir and the file in which your incoming mail will be stored. Also give it safe permissions:

    $ mkdir ~/mail
    $ touch /var/spool/mail/user-name
    $ chmod -R 700 ~/mail

The second step is to configure fetchmail. This is done by creating a configurationfile ~/.fetchmailrc. First create the file and give it safe permissions:

    $ touch ~/.fetchmailrc
    $ chmod 600 ~/.fetchmailrc

Once you have this file you should put the correct settings in with your favorit editor (e.g. vim, pico or joe). A typical entry looks like this:

    poll "my-mail-server"           # tell fetchmail where to look
    protocol pop3                   # we use the POP-protocol
    username "user-name"            # who am i ?
    password "secret"               # passwords are always secret ;-)
    mimedecode                      # automaticaly deal with mime (attachements)
    mda "/usr/bin/maaildrop"        # tell fetchmail which MDA to use

At this point you're done configuring fetchmail. However, we haven't done anything to configure the MDA (maildrop) yet. This is the next step. First start out by creating the correct file ~/.mailfilter and giving it save permissions:

    $ touch ~/.mailfilter
    $ chmod 600 ~/.mailfilter

A typical .mailfilter looks very simple, especially if you don't do any filtering. The following is enough:

    HOME=/home/user-name              # where is our homedir
    DEFAULT=/var/spool/mail/user-name # what is the default place to store mail
    logfile "$HOME/.maillog"          # keep a log 

The log-file is, of course, optional. It may be convenient though, if you want to check what maildrop has done. Once you start building your own filters (e.g. send all mail of a specific contact to a different folder) you can use this log for debugging your ~/.mailfilter.

At this point your fetchmail / maildrop are configured. You can test it by issuing the fetchmail command on your UNIX-shell. If all goes well, the mail is fetched from the server and stored in your DEFAULT-mailbox. If you add the "-v" parameter to fetchmail it will output more verbose messages and tell you what it does. A typical example on my machine (loki) looks like this (note that my username is "basvg" and that the mail-server is "pop-srv.cs.kun.nl"):

    basvg@loki:~$ fetchmail -v
    fetchmail: 6.2.4 querying pop-srv.cs.kun.nl (protocol POP3) at Thu 18 Sep
    2003 1
    0:01:46 AM CEST: poll started
    fetchmail: POP3< +OK POP3 server (version 1.1.5) at pandora.cs.kun.nl
    starting.

    fetchmail: POP3> CAPA
    fetchmail: POP3< +OK POP3 server (version 1.1.5) at pandora.cs.kun.nl
    starting.

    fetchmail: POP3> USER basvg
    fetchmail: POP3< +OK Password required for basvg.
    fetchmail: POP3> PASS *
    fetchmail: POP3< +OK basvg has 1 message (940 octets).
    fetchmail: POP3> STAT
    fetchmail: POP3< +OK 1 940
    fetchmail: POP3> LAST
    fetchmail: POP3< +OK 0 is the last read message.
    1 message for basvg at pop-srv.cs.kun.nl (940 octets).
    fetchmail: POP3> LIST
    fetchmail: POP3< +OK 1 messages (940 octets)
    fetchmail: POP3< 1 940
    fetchmail: POP3< .
    fetchmail: POP3> TOP 1 99999999
    fetchmail: POP3< +OK 940 octets
    reading message basvg@pandora.cs.kun.nl:1 of 1 (940 octets) #** flushed
    fetchmail: POP3> DELE 1
    fetchmail: POP3< +OK Message 1 has been deleted.
    fetchmail: POP3> QUIT
    fetchmail: POP3< +OK Pop server at pandora.cs.kun.nl signing off.
    fetchmail: 6.2.4 querying pop-srv.cs.kun.nl (protocol POP3) at Thu 18 Sep
    2003 10:02:22 AM CEST: poll completed
    fetchmail: normal termination, status 0

Note that fetchmail contacts the mailserver (pandora.cs.kun.nl), finds 1 message and moves it to my machine. There are many other options that you can put in your ~/.fetchmailrc and ~/.mailfilter. For example, you can specify options for leaving mail on the server after retrieving it etcetera. See the manual pages for fetchmail and maildrop to get more insight in the things you can do with these tools!. As an example I will include some filter rules that you could put in a ~/.mailfilter so that you can easilly filter incoming mail.

    # Throw SpamAssassin-tagged spam directly into spam box. This prevents our
    # own Bogofilter word list to explode overnight.
    xfilter "/usr/bin/spamassassin -C /usr/share/spamassassin"
    if ( /^X-Spam-Flag: YES$/ )
    {
        to "Mail/spam-assassin"
    }

    # Run Bogofilter over the remaining stuff. See man bogofilter for more.
    xfilter "bogofilter -u -e -p"
    if (/^X-Bogosity: Spam/)
    {
        to "Mail/spam-bogofilter"
    } 


    # if the address 'ubunutu-users@lists.ubuntu.com' occurs
    # then mail should be redirected to the ubuntu folder
    if ( hasaddr("ubuntu-users@lists.ubuntu.com") )
    {
        to "Mail/ubuntu"
    }
  
    # if there is a subject line (in the headers) which 
    # contains the string [isworld] in it then move the mail
    # to the isworld folder
    if ( /^Subject:.*\[isworld\]/:h )
    { 
        to "Mail/isworld"
    }

Mutt

After configuring fetchmail/maildrop to fetch E-mail to your machine, the next step is to configure your MUA. Several good clients exist (such as PINE, elm, mail). In this howto, thoug, I focus on mutt. The first step is to make the configuration file ~/.muttrc and give it safe permissions:

    $ touch ~/.muttrc
    $ chmod 600 ~/.muttrc

Again, use your favorit editor to edit this file:

    set realname            = "John Doe"
    set from                = "john_do@some.server"
    set use_from            = yes
    set envelope_from       = yes

    set spoolfile           = /var/spool/mail/user-name

With these lines you tell mutt who you are and where it can find your mailbox (this is the same spot that you defined in ~/.mailfilter with the DEFAULT variable). Other options include:

    # from: http://www.mutt.org/doc/manual/manual-3.html#ss3.8
    # this way you get a "clean" view on the important headers of mail
    ignore *
    unignore from date subject to cc bcc
    unignore organization organisation x-mailer: x-newsreader: x-mailing-list:

    # which editor do you want to use? In my case it is vim. I like to
    # ensure that the textwidth is maximum 70 characters.
    set editor              = "vim \"+set textwidth=70\""

    # define how often mut should check for mail
    set mail_check          = 2

    # using the 'pager' option you can tell mutt to split the screen
    # the index (top) has 6 lines. Also specify what should be shown
    # in this pager
    set pager_index_lines   = 6
    set pager_format        = "%S [%C/%T] %n (%l) %s"
    
    # specify where the read messages, sent messages  and the postponed
    # messages are kept.
    set postponed           = ~/mail/postponed
    set mbox                = ~/mail/mbox
    set record              = ~/mail/sent-mail

    # the status-line, at the bottom, can be configured too.
    set status_format       = "%v: %f (%s) [%M/%m] [N=%n,*=%t,post=%p,new=%b]"

    # With hooks you can customize mutt even further. In this case I specify
    # that all folders should be sorted with threads (and also specify which
    # headers I want to see), except for sent-mail. This should be sorted
    # according to the date-sent.
    folder-hook .*          'set sort=threads'
    folder-hook .*          'set strict_threads'
    folder-hook .*          'set hdr_format="%4C %Z %[!%y%m%d] %-17.17F (%3l) %s"'
    folder-hook sent-mail   'set hdr_format="%4C %Z %{%b %d} %-15.15t (%4l) %s"'
    folder-hook sent-mail   'set sort=date-sent'

    # auto_view text/html
    set implicit_autoview

At this point you hae configured mutt and are ready to play with it. The manpage and website for mutt provide a wealth of information on other things that you could try out!

PGP

PGP (Pretty Good Privacy) is a standard for private/public key encryption. One of its applications lies in securing E-mail (but also files). There are 4 concepts of interest:

private/public keypair
The private key is the thing to keep secret. Once someone knows your private key (and the password on it) then s/he can decrypt all your files and Emails. The public key is (as the name suggests) public. Anyone may obtain a copy of your public key. It is a good idea to put your key on a public keyserver (example is given later on in this section).
passphrase
A password on your keys so that not everyone can simply use them.
keyring
You keep a local "database" of keys of people you know about. If you meet someone new: add their public key to your keyring. Once in your keyring you can specify how carefuly you checked if the key is indeed correct, and also express your level of trust in this person.
signing
By signing an E-mail with your private key you tell the world: "yes, I sent this E-mail and no one else". Anyone who has a copy of your public key can verify that it was indeed you who sent the E-mail.
crypting
You encrypt an E-mail with the recepient's public key. Since only the recepient has the private key, only s/he can decrypt the E-mail.

Basic configuration

In short, before you can start using PGP you have to generate your own private/public keypair. After reading gpg --help you'll quickly find that gpg --gen-key is the way to go. A series of questions are asked:

Once you've answered all the questions, GnuPGwil generate the keypair for you. Everything will be stored in the directory ~/.gnupg. For my implementation of gnupg (version 1.2.3), a typical session would look like this:

    $ gpg --gen-key
    gpg (GnuPG) 1.2.3; Copyright (C) 2003 Free Software Foundation, Inc.
    This program comes with ABSOLUTELY NO WARRANTY.
    This is free software, and you are welcome to redistribute it
    under certain conditions. See the file COPYING for details.

    gpg: /root/.gnupg: directory created
    gpg: new configuration file `/root/.gnupg/gpg.conf' created
    gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during
    this run
    gpg: keyring `/root/.gnupg/secring.gpg' created
    gpg: keyring `/root/.gnupg/pubring.gpg' created
    Please select what kind of key you want:
       (1) DSA and ElGamal (default)
       (2) DSA (sign only)
       (5) RSA (sign only)
    Your selection? 1
    DSA keypair will have 1024 bits.
    About to generate a new ELG-E keypair.
                  minimum keysize is  768 bits
                  default keysize is 1024 bits
        highest suggested keysize is 2048 bits
    What keysize do you want? (1024) 2048
    Requested keysize is 2048 bits
    Please specify how long the key should be valid.
               0 = key does not expire
              = key expires in n days
            w = key expires in n weeks
            m = key expires in n months
            y = key expires in n years
    Key is valid for? (0) 0
    Key does not expire at all
    Is this correct (y/n)? y
                            
    You need a User-ID to identify your key; the software constructs the user id
    from Real Name, Comment and Email Address in this form:
        "Heinrich Heine (Der Dichter) "

    Real name: "John Doe "

    Comment: 
    You selected this USER-ID:
        "John Doe "

    Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
    You need a Passphrase to protect your secret key.    

    You need a Passphrase to protect your secret key.

    We need to generate a lot of random bytes. It is a good idea to perform
    some other action (type on the keyboard, move the mouse, utilize the
    disks) during the prime generation; this gives the random number
    generator a better chance to gain enough entropy.
    +++++++++++++++.+++++.+++++.++++++++++.+++++++++++++++.+++++.++++++++++....
    +++++.+++++..+++++.+++++++++++++++..++++++++++.+++++..+++++.+++++++++++++++
    gpg: /root/.gnupg/trustdb.gpg: trustdb created
    public and secret key created and signed.
    key marked as ultimately trusted.

    pub  1024D/097D3958 2003-10-28 John Doe 
         Key fingerprint = DAEC E5AE 70BB 3643 6BE9  F828 9172 CDE7 097D 3958
    sub  2048g/81AE1790 2003-10-28

Note: Different versions / implementations of PGP software may produce different output. The core idea, though, remains the same.

At this point you have your private/public key-pair. These are stored as binary files (best not to look at them ;-) in the directory ~/.gnupg (if you want to check whether your key is actually there use the following command: gpg --list-key john.doe). Time to configure the program some more. The file you need is ~/.gnupg/gpg.conf (which is the default configuration file for my implementation. There's also an "old style" configuration file ~/.gnupg/options. By making a softlink from one to the other you can fix this: ln -sf link-from link-to). At the bottom of the configuration file you can tell it which key-servers it should look for. Also you can tell it to ingore the "secmem-warning":

    # GnuPG can import a key from a HKP keyerver if one is missing
    # for certain operations. Is you set this option to a keyserver
    # you will be asked in such a case whether GnuPG should try to
    # import the key from that server (server do syncronize with each
    # other and DNS Round-Robin may give you a random server each time).
    # Use "host -l pgp.net | grep www" to figure out a keyserver.
    keyserver wwwkeys.us.pgp.net
    keyserver wwwkeys.nl.pgp.net
    keyserver blackhole.pca.dfn.de
    keyserver horowitz.surfnet.nl

    # get rid of annoying error/warning
    no-secmem-warning 

With these options set you're ready to go. Your keyring is still empty, though. Adding new people to the keyring consists of the following steps:

    $ gpg --search-keys bas.vangils@cs.kun.nl
    gpg: searching for "bas.vangils@cs.kun.nl" from HKP server horowitz.surfnet.nl
    Keys 1-1 of 1 for "bas.vangils@cs.kun.nl"
    (1)     Bas van Gils 
              1024 bit DSA key 2768A493, created 2002-07-03
    Enter number(s), N)ext, or Q)uit > 1
    gpg: key 2768A493: public key "Bas van Gils " imported
    gpg: Total number processed: 1
    gpg:               imported: 1

The above output already shows that one key is imported into your keyring.

You still have to check the key and set various options for it. This is done by editting the key with gpg --edit-key john.doe@some.domain. Once you do this you enter a menu with a prompt. First you issue the command "sign" to sign the key. The program will ask how carefully you checked whether the key is, indeed, correct. You can check by asking the other person to read out the first part of the key. Once this is done you can set the level of trust by issuing the "trust" command. Once that is done too, simply type "save" to save what you've done. A typical sessions looks like this:

    $ gpg --edit-key bas.vangils@cs.kun.nl
    gpg (GnuPG) 1.2.3; Copyright (C) 2003 Free Software Foundation, Inc.
    This program comes with ABSOLUTELY NO WARRANTY.
    This is free software, and you are welcome to redistribute it
    under certain conditions. See the file COPYING for details.
    
    
    gpg: checking the trustdb
    gpg: checking at depth 0 signed=0 ot(-/q/n/m/f/u)=0/0/0/0/0/1
    pub  1024D/2768A493  created: 2002-07-04 expires: never      trust: -/-
    sub  1024g/EE1483AA  created: 2002-07-04 expires: never     
    (1). Bas van Gils 
    Command> sign
                 
    pub  1024D/2768A493  created: 2002-07-04 expires: never      trust: -/-
     Primary key fingerprint: 3805 664A F24E 90D5 6A77  C624 E13D 0440 2768 A493
    
         Bas van Gils 
    
    How carefully have you verified the key you are about to sign actually belongs
    to the person named above?  If you don't know what to answer, enter "0".
    
       (0) I will not answer. (default)
       (1) I have not checked at all.
       (2) I have done casual checking.
       (3) I have done very careful checking.
    
    Your selection? (enter '?' for more information): 3
    Are you really sure that you want to sign this key
    with your key: "John Doe " (097D3958)
    
    I have checked this key very carefully.
    
    Really sign? yes
    
    You need a passphrase to unlock the secret key for
    user: "John Doe "
    1024-bit DSA key, ID 097D3958, created 2003-10-28
    
    
    Command> trust
    pub  1024D/2768A493  created: 2002-07-04 expires: never      trust: f/-
    sub  1024g/EE1483AA  created: 2002-07-04 expires: never     
    (1). Bas van Gils 
    
    Please decide how far you trust this user to correctly
    verify other users' keys (by looking at passports,
    checking fingerprints from different sources...)?
    
     1 = Don't know
     2 = I do NOT trust
     3 = I trust marginally
     4 = I trust fully
     5 = I trust ultimately
     m = back to the main menu
    
    Your decision? 4
                    
    pub  1024D/2768A493  created: 2002-07-04 expires: never      trust: f/-
    sub  1024g/EE1483AA  created: 2002-07-04 expires: never     
    (1). Bas van Gils 
    Command> save

That's all... the key is in your keyring and you're all done. You could add more keys though. You might also want to explore the "--export" options to export your key (and trust) to the servers. Don't forget to do this at least once so that your public key is on the servers as well!

Integration with mutt

With everything configured and functioning just fine, it is time to integrate with mutt. This is done by putting some options in ~/.muttrc again. These options can be found on the Web and it's best to simply cut-and-paste them there:

    # Command formats for gpg.
    # 
    # This version uses gpg-2comp from 
    #   http://muppet.faveve.uni-stuttgart.de/~gero/gpg-2comp.tar.gz
    #
    # $Id: gpg.rc,v 3.1 2002/03/26 22:23:58 roessler Exp $
    #
    # %p    The empty string when no passphrase is needed,
    #       the string "PGPPASSFD=0" if one is needed.
    #
    #       This is mostly used in conditional % sequences.
    #
    # %f    Most PGP commands operate on a single file or a file
    #       containing a message.  %f expands to this file's name.
    #
    # %s    When verifying signatures, there is another temporary file
    #       containing the detached signature.  %s expands to this
    #       file's name.
    #
    # %a    In "signing" contexts, this expands to the value of the
    #       configuration variable $pgp_sign_as.  You probably need to
    #       use this within a conditional % sequence.
    #
    # %r    In many contexts, mutt passes key IDs to pgp.  %r expands to
    #       a list of key IDs.
    
    # Note that we explicitly set the comment armor header since GnuPG, when used
    # in some localiaztion environments, generates 8bit data in that header, thereby
    # breaking PGP/MIME.
    
    # decode application/pgp
    set pgp_decode_command="/usr/bin/gpg  --charset utf-8   %?p?--passphrase-fd 0? --no-verbose --quiet  --batch  --output - %f"
    
    # verify a pgp/mime signature
    set pgp_verify_command="/usr/bin/gpg   --no-verbose --quiet  --batch  --output - --verify %s %f"
    
    # decrypt a pgp/mime attachment
    set pgp_decrypt_command="/usr/bin/gpg   --passphrase-fd 0 --no-verbose --quiet  --batch  --output - %f"
    
    # create a pgp/mime signed attachment
    # set pgp_sign_command="/usr/bin/gpg-2comp --comment '' --no-verbose --batch  --output - --passphrase-fd 0 --armor --detach-sign --textmode %?a?-u %a? %f"
    set pgp_sign_command="/usr/bin/gpg    --no-verbose --batch --quiet   --output - --passphrase-fd 0 --armor --detach-sign --textmode %?a?-u %a? %f"
    
    # create a application/pgp signed (old-style) message
    # set pgp_clearsign_command="/usr/bin/gpg-2comp --comment ''  --no-verbose --batch  --output - --passphrase-fd 0 --armor --textmode --clearsign %?a?-u %a? %f"
    set pgp_clearsign_command="/usr/bin/gpg   --charset utf-8 --no-verbose --batch --quiet   --output - --passphrase-fd 0 --armor --textmode --clearsign %?a?-u %a? %f"
    
    # create a pgp/mime encrypted attachment
    # set pgp_encrypt_only_command="pgpewrap gpg-2comp  -v --batch  --output - --encrypt --textmode --armor --always-trust -- -r %r -- %f"
    set pgp_encrypt_only_command="pgpewrap /usr/bin/gpg  --charset utf-8    --batch  --quiet  --no-verbose --output - --encrypt --textmode --armor --always-trust -- -r %r -- %f"
    
    # create a pgp/mime encrypted and signed attachment
    # set pgp_encrypt_sign_command="pgpewrap gpg-2comp  --passphrase-fd 0 -v --batch  --output - --encrypt --sign %?a?-u %a? --armor --always-trust -- -r %r -- %f"
    set pgp_encrypt_sign_command="pgpewrap /usr/bin/gpg  --charset utf-8 --passphrase-fd 0  --batch --quiet  --no-verbose  --textmode --output - --encrypt --sign %?a?-u %a? --armor --always-trust -- -r %r -- %f"
    
    # import a key into the public key ring
    set pgp_import_command="/usr/bin/gpg  --no-verbose --import -v %f"
    
    # export a key from the public key ring
    set pgp_export_command="/usr/bin/gpg   --no-verbose --export --armor %r"
    
    # verify a key
    set pgp_verify_key_command="/usr/bin/gpg   --verbose --batch  --fingerprint --check-sigs %r"
    
    # read in the public key ring
    set pgp_list_pubring_command="/usr/bin/gpg   --no-verbose --batch --quiet   --with-colons --list-keys %r" 
    
    # read in the secret key ring
    set pgp_list_secring_command="/usr/bin/gpg   --no-verbose --batch --quiet   --with-colons --list-secret-keys %r" 
    
    # fetch keys
    # set pgp_getkeys_command="pkspxycwrap %r"
    
    # pattern for good signature - may need to be adapted to locale!
    
    # set pgp_good_sign="^gpg: Good signature from"
    
    # OK, here's a version which uses gnupg's message catalog:
    set pgp_good_sign="`gettext -d gnupg -s 'Good signature from "' | tr -d '"'`"

This is, more or less, all there is to it. After composing a mail in mutt, the 'p' command gives you the PGP-options. You can either 'sign' or 'crypt' or 'both'. Mutt will only ask you for your passphrase...

Two more options in mutt are of interested, though. When you start mutt, the "esc-k" command (that is, escape-k) is used to send a key to someone. All you have to do is tell mutt which key you want to send. Conversely, "^K" (that is, control-k) is used to extract a key. If someone has mailed you his/her public key then issuing this command will extract the key and add it to your keyring. All you need to do then is exit mutt and use "gpg --edit-key" again.