That said, most of us have better things to do than trawl our plethora of overly verbose logfiles every few minutes. One way of meeting our systems halfway is to receive summaries of uncommon and important log entries on a regular basis. In my case, I have these summaries sent to a dedicated mail account, which I check via my phone, at my leisure.
This way, the most important information is collected and sent to me, for perusal when I have time. It's a win-win situation.
For the task of logfile distillation and email preparation, I use a handy tool called logcheck.
It contains a large collection of rules which filter out "uninteresting" log entries, leaving just the tastier morsels for my daily commuting pleasure. It also contains a set of rules for "violations"; significant events, which logcheck separates into another email to highlight their occurrence.
Over time, you'll find yourself tuning these rules to reduce your reading burden. With each rule being a simple regex string, this is no great ordeal.
Installing Logcheck
sudo apt-get install logcheck
Now to configure it:
sudo vim /etc/logcheck/logcheck.conf
The one change you do need to make is to tell logcheck where to send its reports to:
SENDMAILTO="recipient@domain.com"
Aside from that, it's worth reading over this file to see what else you'd care to change. I like to set the following; shortening the subject line and the message:
INTRO=0
DATE="$(date +'%H:%M')"
FQDN=0
ATTACKSUBJECT="Security Alert"
SECURITYSUBJECT="Security Event"
EVENTSSUBJECT=
Logcheck executes via cron, so there's no service to restart after updating the config file. By default, it runs 2 minutes past every hour.
You can force a report to run by invoking logcheck manually. Try it out now:
sudo -u logcheck logcheck
However, unless you're already running a mailserver, you're likely to find that nothing interesting happens at this stage. This is because logcheck expects to use the sendmail command to send the email message; and this is where things get interesting.
Sendmail
Sendmail has been in existence since the 1980s. It's a veritable swiss army knife of a mailserver. However, it's an interesting beast to try to configure.
Most end users, like myself, will simply want logcheck to send their status emails via an external mailserver - say, a gmail server, or one provided by their hosting company. Well, it should be easy enough to configure a relay to an external mailserver with a powerful tool like sendmail. But that wasn't my experience. Between a mess of dnl terminators, FEATUREs, bizarre config parameter names and the magical world of sendmail's string quotes, I started losing the will to live.
Switching from sendmail to postfix (a simpler, drop-in replacement) was a welcome improvement; but it was still too heavy a solution for a task as simple as this. Hash dbs, stunnel... it all involved an unnecessary number of moving parts. Ergo, for simple mail relay to an external mailserver, I sought an alternative.
Configuring Mail Relay with MSMTP
First, I tried ssmtp, which had a good reputation for being a simple, lightweight, and reliable solution for mailserver relay. However, that coredumped straightaway on my x64 box, so I switched to msmtp, which is very similar.
sudo apt-get install msmtp
sudo vim /etc/msmtprc
A simple configuration looks like:
defaults
tls on
tls_trust_file /etc/ssl/certs/ca-certificates.crt
logfile /home/USERNAME/.msmtp.log
# Account fooBar
account fooBar
host smtp.foo.bar.com
port 587
# Note that this is simply envelope-from, not the mail's "From" header
from me@foo.bar.com
auth on
user me@foo.bar.com
password ySSwbUsnEz5M
# Use fooBar as the default account
account default : fooBar
msmtp will use STARTTLS where possible to enter TLS mode, so that passwords are sent over an encrypted channel. The documentation claims that it will not try to send the password in an insecure form over an insecure channel: "If you really want to risk your authentication data, you have to force msmtp to do that by manually setting the authentication method while TLS is off." If you like, you can confirm this via Wireshark or tcpdump.
For simplicity here, I've hardcoded the password. Obviously, this is not good, and you'll want to take steps to remedy this once you have it working. A quick workaround involves limiting access to /etc/msmtprc to a particular group, e.g. "msmtp", and adding the logcheck user to that group. An arguably slightly more rugged approach involves using msmtp's passwordeval to invoke gpg (backed by gpg-agent to support non-interactivity) to decrypt a password file.
Either way, it's a very good idea to use a separate dedicated email account for logcheck notifications, as I have done; this way, there is much less to lose via password exposure (an attacker with local system access has the ability to view your logfiles anyway).
Now, verify you can send a mail with:
echo "Test from msmtp" | msmtp recipient@domain.com
If you received that mail, congratulations. You've configured a command-line mailer. Now you need to override lsb-invalid-mta and tell your system to use msmtp as its sendmail-compliant mailer:
sudo ln -s /usr/bin/msmtp /usr/local/sbin/sendmail
sudo ln -s /usr/bin/msmtp /usr/local/bin/sendmail
Test with:
echo -e "Subject: I hope I receive this\nTest from sendmail" | sendmail recipient@domain.com
If you want, you can also configure it as your system's "mail" command mailer too. (It's not necessary for logcheck, but you may find it useful for command line e-mailing)
echo 'set sendmail="/usr/bin/msmtp"' | sudo tee -a /etc/.mailrc
Test with:
echo "Test from mail" | mail -s "This too"
recipient@domain.com
So, in future, should you want to send mail from the command line, you can use either the "mail" or "sendmail" utilities.
Logcheck
With an external email relay configured, and the sendmail command working, logcheck should start sending its updates each hour. Try it out by creating a security event (try to sudo using an invalid password, for example).
Logcheck does have its flaws. Ironically, it doesn't offer much by way of logging itself, and its verbose mode (-dd) could provide more detail. Additionally, I'd like to see more configuration options than are on offer at present.
However, compared to letting your logfiles wrap around out-of-sight - and out-of-mind - this old tool still has much to offer for simple home system monitoring.
Happy log scouring!
