This is a simple, quick-and-dirty, copy/paste guide to install a great software, pmacct, on a fresh Ubuntu 14.04.1 LTS (Trusty Tahr) setup. I’ll use this simple setup as the basis for other related posts I plan to publish soon.
Tl;dr: pmacct is a suite of tools to collect, filter and aggregate IP accounting data, which works with live traffic (libpcap), NetFlow v1/v5/v7/v8/v9, IPFIX, sFlow and ULOG.
A blog post is not enough to show the great features and possibilities that this tool offers, so I really recommend whoever may be interested to read author’s documentation on the official web site.
On a next post I plan to show some ideas to deploy pmacct together with ElasticSearch and Kibana, in order to build useful dashboards full of graphs. Add my RSS feed to your reader or follow me on Twitter to stay updated!
EDIT: the Integration of pmacct with ElasticSearch and Kibana post has been published.
Let’s start from a really simple setup here.
Compilation and installation
Since the Ubuntu packaged version is quite old (0.14.0) I preferred to download and compile the latest release from the official web site (1.5.0 at the time of writing):
# cd /usr/local/src/ # curl -O http://www.pmacct.net/pmacct-1.5.0.tar.gz # tar -zxvf pmacct-1.5.0.tar.gz # cd pmacct-1.5.0/
pmacct depends on libpcap, and because we also want to use JSON for the output (we will see why in another post), we need to install the following libraries:
# apt-get install libjansson-dev libpcap-dev make
Now, configure, make and install the program:
# ./configure --enable-ipv6 --enable-jansson # make # make check # make install
(in case of compilation problems, add the –enable-relax option to ./configure)
# ./configure --enable-ipv6 --enable-jansson --enable-relax
The INSTALL file provided along with the source code may offer additional details.
When done, if other options have not been modified, we can find the client program in /usr/local/bin/ and the daemons in /usr/local/sbin/:
# whereis pmacct pmacct: /usr/local/bin/pmacct # whereis pmacctd pmacctd: /usr/local/sbin/pmacctd # whereis nfacctd nfacctd: /usr/local/sbin/nfacctd
pmacct needs and creates some files, so I decided to use the following directories for them:
# mkdir /etc/pmacct # for configuration files # mkdir /var/run/pmacct # for PID files # mkdir /var/spool/pmacct # for plugins pipes # mkdir /var/lib/pmacct # for plugins output
Now, the installation is complete and some configurations are needed to make it work.
In this post I show only a really simple configuration based on pmacctd, the daemon which uses libpcap to collect live traffic from a NIC: similar configurations may be used for the other daemons too and for more useful scenarios, for example to acquire NetFlow data (nfacctd) or sFlow packets (sfacctd), merge them with BGP routing table information and build traffic engineering reports: the basic principle is that a daemon collects accounting data while its plugins process and aggregate them and, finally, produce output results.
The official Quickstart guide is the best document to start reading from, along with the Collecting NetFlow with pmacct (PDF) tutorial from RIPE SEE3.
That said, a quick-and-dirty approach is to build a really simple configuration and save it to /etc/pmacct/pmacctd.conf; here it is:
syslog: daemon promisc: true interface: eth0 imt_mem_pools_number: 0 plugins: memory[plugin1] imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe aggregate[plugin1]: src_host, src_port, dst_host, dst_port
This configuration allows pmacctd to enters promiscuous mode on eth0 and store data in an in-memory-table (imt), taking into account only source host:port and destination host:port. Since the imt_mem_pools_number key is set to zero, pmacctd will continue to allocate memory in order to store its output, so it’s important to periodically empty it (below we’ll see how).
Run it (-D option is used to daemonize it)…
# pmacctd -f /etc/pmacct/pmacctd.conf -D
… and wait some seconds (and some network traffic), then query the in-memory-table using the pmacct client:
# pmacct -p /var/spool/pmacct/plugin1.pipe -s SRC_IP DST_IP SRC_PORT DST_PORT PACKETS BYTES 172.16.1.195 172.16.1.15 22 59635 13 1396 172.16.1.15 10.0.0.100 59693 443 1 41 192.168.0.1 172.16.1.30 443 52901 3 220 [...]
Network traffic captured on eth0 will be aggregated on the basis of src host / dst host / src port / dst port and counters will be shown.
To empty the in-memory-table, add the -e option:
# pmacct -p /var/spool/pmacct/plugin1.pipe -s -e
To add some details to the output we can add two additional fields to the aggregate, etype (Ethernet Ethertype) and proto (IP protocol); so edit the /etc/pmacct/pmacctd.conf file and change the aggregate[plugin1] key:
aggregate[plugin1]: etype, proto, src_host, src_port, dst_host, dst_port
When done, kill and restart the daemon…
killall -INT pmacctd -w ; pmacctd -f /etc/pmacct/pmacctd.conf -D
… wait some seconds, then run the client again to show the new output:
# pmacct -p /var/spool/pmacct/plugin1.pipe -s -e ETYPE SRC_IP DST_IP SRC_PORT DST_PORT PROTOCOL PACKETS BYTES 86dd 2001:DB8::A 2001:DB8::B 60129 80 tcp 3 1468 800 172.16.1.15 172.16.1.195 53384 22 tcp 3 204 [...]
Now the testbed is ready for playing with pmacct! The official documentation page is full of intriguing presentations and examples from which to take inspiration.
/etc/init.d System V initscript
In order to manage pmacct daemons I’ve built an init.d script: you can find it on my GitHub page. If you need and like it, you can install it (please read how it works in the comments) and use update-rc.d to install links to automatically start/stop daemons in your system:
# cd /etc/init.d # curl -O https://raw.githubusercontent.com/pierky/pmacct-initscript/master/pmacct # chmod a+x pmacct # update-rc.d pmacct defaults
Feel free to edit/share/ignore it!
When finished playing with pmacct, if you are still using a configuration with imt_mem_pools_number: 0 remember to kill the daemon or be sure you have a client consuming its results, otherwise memory will be filled up!
Latest posts by Pier Carlo Chiodi (see all)
- Good MANRS for IXPs route servers made easier - 11 December 2020
- Route server feature-rich and automatic configuration - 13 February 2017
- Large BGP Communities playground - 15 September 2016
My service is frequently crashed. When I look into the log file, I can see the below error message.
Can you please suggest me what was reason behind this error message.
Sep 3 13:51:17 fc-feed06 kernel: pmacctd: segfault at 7ff800000000 ip 00000030dc67b55c sp 00007ffffb430c98 error 4 in libc-2.12.so[30dc600000+18a000]
Sep 3 13:51:17 fc-feed06 abrt: Not saving repeating crash in ‘/usr/local/sbin/pmacctd’
Sep 3 13:51:37 fc-feed06 kernel: pmacctd: segfault at 7f9ffc000000 ip 00000030dc67b55c sp 00007fff5868dcb8 error 4 in libc-2.12.so[30dc600000+18a000]
Sep 3 13:51:38 fc-feed06 abrt: Saved core dump of pid 407 (/usr/local/sbin/pmacctd) to /var/spool/abrt/ccpp-2015-09-03-13:51:37-407 (11698176 bytes)
Sep 3 13:51:38 fc-feed06 abrtd: Directory ‘ccpp-2015-09-03-13:51:37-407’ creation detected
Sep 3 13:51:38 fc-feed06 abrtd: Executable ‘/usr/local/sbin/pmacctd’ doesn’t belong to any package and ProcessUnpackaged is set to ‘no’
Sep 3 13:51:38 fc-feed06 abrtd: ‘post-create’ on ‘/var/spool/abrt/ccpp-2015-09-03-13:51:37-407’ exited with 1
Sep 3 13:51:38 fc-feed06 abrtd: Deleting problem directory ‘/var/spool/abrt/ccpp-2015-09-03-13:51:37-407’
Sep 3 13:52:01 fc-feed06 snmpd: Connection from UDP: [126.96.36.199]:42742->[188.8.131.52]
Sep 3 13:53:19 fc-feed06 kernel: pmacctd: segfault at 7fd450000000 ip 00000030dc67b55c sp 00007fff76aacde8 error 4 in libc-2.12.so[30dc600000+18a000]
Sep 3 13:53:19 fc-feed06 abrt: Saved core dump of pid 428 (/usr/local/sbin/pmacctd) to /var/spool/abrt/ccpp-2015-09-03-13:53:19-428 (11698176 bytes)
Sep 3 13:53:19 fc-feed06 abrtd: Directory ‘ccpp-2015-09-03-13:53:19-428’ creation detected
Sep 3 13:53:20 fc-feed06 abrtd: Executable ‘/usr/local/sbin/pmacctd’ doesn’t belong to any package and ProcessUnpackaged is set to ‘no’
Sep 3 13:53:20 fc-feed06 abrtd: ‘post-create’ on ‘/var/spool/abrt/ccpp-2015-09-03-13:53:19-428’ exited with 1
the crash seems to be on the pmacctd process, please refer to the official pmacct project website for support.
if you have any chance, please provide an example for how to integrate L7-filer in pmacctd. As I’m unable to enable this on my config
please refer to pmacct website for this kind of support: http://www.pmacct.net/#contacts
Hello, thanks for the tutorial, it is pretty clear.
I am new to pmacct and i was trying to capture traffic in an openvirtualswitch switch, following your post. However when i issue the command “pmacct -p /var/spool/pmacct/plugin1.pipe -s”, i can not see traffic from the hosts that are interconnected via the ovs switch. I can only see the interface of the ovs switch. May be i am doing something wrong, or is it that pmacct doesn’t support ovs switch network traffic?.
Thanks in advance.
please refer to pmacct website for this kind of support: http://www.pmacct.net/#contacts
First, I wanna thank you so much for this tutorial, you saved my life 😀
I just want ask, I’m trying to run pmacctd and save the output into MySQL DB, I did the ./ configure –enable_mysql but I want to ask what to change in the pmacctd.conf file?
And Thank you in advance. :))
you can find what you’re looking for in the official quickstart guide (many examples are provided there) and on the config keys documentation.
Hi Pier, Can you write a blog or refer to any document on making bgp neighborship from linux server to router running bgp via pmaact?
hello can you refer me to a place where i can find the pmacct command line options list