In this post I want to show a solution based on a script (pmacct-to-elasticsearch) that I made to gather data from pmacct and visualize them using Kibana/ElasticSearch. It’s far from being the state of the art of IP accounting solutions, but it may be used as a starting point for further customizations and developments.
I plan to write another post with some ideas to integrate pmacct with the canonical ELK stack (ElasticSearch/Logstash/Kibana). As usual, add my RSS feed to your reader or follow me on Twitter to stay updated!
The big picture
This is the big picture of the proposed solution:
There are 4 main actors: pmacct daemons (we already saw how to install and configure them) that collect accounting data, pmacct-to-elasticsearch, which reads pmacct’s output, processes it and sends it to ElasticSearch, where data are stored and organized into indices and, at last, Kibana, that is used to chart them on a web frontend. Read more …
This is a simple, quick-and-dirty, copy/paste guide to install a great software, pmacct, on a fresh Ubuntu 14.04.1 LTS (Trusty Tahr) setup. I’ll use this simple setup as the basis for other related posts I plan to publish soon.
Tl;dr: pmacct is a suite of tools to collect, filter and aggregate IP accounting data, which works with live traffic (libpcap), NetFlow v1/v5/v7/v8/v9, IPFIX, sFlow and ULOG.
A blog post is not enough to show the great features and possibilities that this tool offers, so I really recommend whoever may be interested to read author’s documentation on the official web site.
On a next post I plan to show some ideas to deploy pmacct together with ElasticSearch and Kibana, in order to build useful dashboards full of graphs. Add my RSS feed to your reader or follow me on Twitter to stay updated!
EDIT: the Integration of pmacct with ElasticSearch and Kibana post has been published.
Let’s start from a really simple setup here. Read more …
The number of IPv4 prefixes in the global BGP table is approaching the limit of many Cisco products, such as 7600/6500 RSP720/Sup720 and some ASR1000, which may hold a maximum of ~500K routes in their FIB (the Forwarding Information Base, where only best paths are stored).
These routers can usually handle a bigger load of prefixes, they can also be used to receive the full BGP table from many upstream providers concurrently, but they can’t manage more than 500/512K entries in their RIB or FIB.
Read more …
I’m happy to see that more and more tools are developed to increase the security level and trustworthiness of Internet applications. I already talked about DNSSEC and tools to check the validity of domain names, many others blogged about DANE and TLSA validation support in browsers; this time I would like to focus on DKIM and on a Thunderbird add-on to verify its signatures taking advantage of DNSSEC end-to-end validation.
Read more …
Fencing is a vital component in a virtualization cluster; when a cluster member fails it must be inhibited to access shared resources such as network disks or SAN, so that any virtual machine still running on it could be restarted on other members, being sure that no data will be corrupted because of simultaneous access.
Many methods exist to fence failed cluster members, mostly based on powering them off or on disconnecting their network cards; here I would like to show how to use network fencing on a Linux cluster environment (Cman based), using the fence_ifmib against a Cisco managed switch.
The logic behind this mechanism is very simple: once a node has been marked as dead the agent uses the SNMP SET method to tell the managed switch to shut the ports down. Read more …