Building Simple Graphs


by J. Edward Durrett

Visualizing data can help people see patterns more easily as well as communicate complex issues to a wider
audience. There is great software out there that helps you do this. The following is an example on how to graph
top pf rejected connections logged in /var/log/pflog with some just some python and a bit of pipe-fu.

The python module matplotlib makes some nice graphs [1] and the following is based on the pie chart demo they have
on their web page [1].

Before going into the python, we need to get our data and this does the trick:

 
tcpdump -nr /var/log/pflog | awk '{print$5}' | cut -d . -f5 | sort -n | uniq -c | sort -n | tr -d : | awk '{print$2" "$1}' | tail -5


I’ll unpack that a bit. First we call tcpdump to read the log file because it is in pcap format (and, by the
way, the whole packet is in there so there is a ton of information especially useful in active response). Then we
call awk to print the 5th position which is the destination ip address and port. We use cut to give us just
the last field, 5, which is the port. Then we sort the ports numerically. Using uniq we get a count of each port.
Then we sort them again to put them in order from lowest to highest instances. Using tr we can get rid of a pesky extra
character, the colon, that our previous filters missed. Then, we awk again, reversing the order of the fields so
they fit what our python script expects. And, finally, tail gives us just the top 5.

Now, to get a graph, all we have to do is pipe the output of the above command into this python script:

 
#!/usr/bin/python

import matplotlib.pyplot as plt
import sys

labels = []
sizes = []

while True:
line_prep=sys.stdin.readline().rstrip('\n')
if ' ' in line_prep:
label, value = line_prep.split(" ")
if line_prep == '':
break
else:
labels.append(label)
sizes.append(value)

exp = []
for x in sizes:
exp.append(0.1)

explode = exp
fig1, ax1 = plt.subplots()
ax1.pie(sizes, explode=explode, labels=labels, autopct='%1.1f%%',
shadow=True, startangle=90)
ax1.axis('equal') # to get circle
plt.title('Top Restricted Port Access Attempts')
plt.show()


And voilla! We get this graph:



Now, here is the interesting thing. It took me longer to write this explanation than actually making the graph. If
you can set aside just 15 or 20 minutes a week, at the end of the year, you will have a dashboard of 52 graphs.
The graphs can also be used in automating reports distributed to staff or management. Automating the routine, and
less face it, boring, aspects of your job frees you to add value to the organization.

References:

[1] https://matplotlib.org/examples/pie_and_polar_charts/pie_demo_features.html







Copyright (c) 2019, Jason Edward Durrett - All content on this site, unless otherwise noted, is subject to this license.

Please contact me if any errors, such as erroneous / misleading content or missing / incomplete attribution, are found.