Putting Process Accounting to Use

by J. Edward Durrett

Process accounting on a *NIX system keeps track of commands run on that system. Most systems like RHEL, Solaris
and FreeBSD have process accounting installed by default and it just needs to be activated by the accton command.
It is a separate package on Debian and descendants. This is a great way to keep tabs on what has been done on a
system for change management and security purposes. Just enabling process accounting, though, does not do much
besides log commands and statistics. Just looking at the commands run on a system really does not do much and, as there
is a lot of data, it is not easy for a human to notice things that should not be there. Here is a simple way to
keep tabs on the processes.

The first thing to do is create a baseline. What are the normal commands run on the system? This of course varies.
This will dump commands that are run into a file called normal commands:


lastcomm | awk '{print $1}' | sort -n | uniq > normal_commands 

And this will compare what is normally run to what is being run on a particular day:

lastcomm | awk '{print $1}' | sort -n | uniq | grep -v -f normal_commands 


This will show you if unauthorized commands were run on a system. A practical use would to be monitoring a
production system where development work should not normally done. Commands such as gcc probably is not run on a
regular basis.

There is a limited security benefit as well. Noisy, generally scripted intrusions will easily be detected with
this method; however there are several reasons why this should not be relied upon as a sole intrusion detection
method. For one, an attacker with root could just turn off process accounting with acctoff. Also, it is possible
to run a command with an arbitrary name. See the example below on running top as the command blah:


cp /usr/bin/top blah 
./blah

And in another terminal:

ps | grep blah 
90775 3 S+ 0:00.03 ./blah

That said, there is still some usefulness.

Of course, doing this by hand isn’t practical across many systems and is quite boring to do daily even on one
system. The script below, also available on github/durrettj automates this check and mails basic system system
information when an anomaly is detected. It works on Linux and FreeBSD. It is possible with modification to run
this on Solaris. The grep utility in Solaris 11 does not accept the -v flag .


#!/bin/sh 
WHITELIST=""
TEMPFILE=""
EMAIL=""
HASH=""
OS=`uname`
lastcomm | awk '{print $1}' | sort -n | uniq | grep -v -f $WHITELIST > $TEMPFILE
if [ -s $TEMPFILE ]
then
if [ $OS == Linux ]; then
printf "\nThe above unauthorized commands were found. SHA256 hash to follow. \n" >> $TEMPFILE
printf "Diagnostic information below:\n\n" >> $TEMPFILE
date >> $TEMPFILE
printf "\nListening and established network info:\n" >> $TEMPFILE
netstat -pant >> $TEMPFILE
printf "\nProcess information:\n" >> $TEMPFILE
ps -ef >> $TEMPFILE
#WARNING: lsof can provide really useful information, but on a heavily loaded system
#like a terminal server that hosts 100s of Desktops, this file can be huge as in
#Gigabytes - Make sure you know what you are getting into before enabling!
#printf "\nOpen Files via lsof:\n"
#lsof >> $TEMPFILE
mail -s "WARN: EXTRA COMMANDS FOUND" $EMAIL < $TEMPFILE
mail -s "SHA256: ECF" $EMAIL < $HASH
mail -s "SHA256: ECF" $EMAIL < $HASH
exit 1
elif [ $OS == FreeBSD ]; then
printf "\nThe above unauthorized commands were found. SHA256 hash to follow. \n" >> $TEMPFILE
printf "Diagnostic information below:\n\n" >> $TEMPFILE
date >> $TEMPFILE
printf "\nActive firewall connections:\n" >> $TEMPFILE
pftop -b >> $TEMPFILE
printf "\n\nListening sockets from sockstat:\n\n" >> $TEMPFILE
sockstat -l >> $TEMPFILE
printf "\n\nProcesses from procstat:\n\n" >> $TEMPFILE
procstat -a >> $TEMPFILE
printf "\n\nOpen files from fstat:\n\n" >> $TEMPFILE
fstat >> $TEMPFILE
sha256 $TEMPFILE > $HASH
mail -s "WARN: EXTRA COMMANDS FOUND" $EMAIL < $TEMPFILE
mail -s "SHA256: ECF" $EMAIL < $HASH
exit 1
fi
fi
rm $TEMPFILE
rm $HASH







Copyright (c) 2019, Jason Edward Durrett - All content on this site, unless otherwise noted, is subject to this license.

Please contact me if any errors, such as erroneous / misleading content or missing / incomplete attribution, are found.