SecurityCloud

SecurityCloud project aims at developing innovative solution for up-to-date services and infrastructure enabling to detect operational and security risks (i.e. attacks, anomalies, failures or decreased service quality). The developed solution will meet the need for processing and analysis of enormous amounts of specific network-related data. The Security Cloud collector will enable collection, storage and ad-hoc user queries over large amount of flow data with interactive response as well as  deep streaming analysis in real time.  The developed solution will be by inherently distributed and scalable. Participating partners are Masaryk University in Brno and Flowmon Networks.

 

Architecture

SecurityCloud collector is a distributed flow-based processing software based on master, slaves and proxy architecture. Proxy serves to receive flow records and distributes (round-robin) these flow records to the slaves while master serves as a central point to query the stored flow records. Please note that a node may serve as master, proxy or slave at the same time.

SecurityCloud collector consists of two core tools – IPFIXcol and fdistdump. IPFIXcol receives, distributes and stores flow data while fdistdump executes ad hoc user queries upon stored data. IPFIXcol must be instantiated as proxy at proxy node and as collector at slave nodes. Fdistdump is a command line tool that utilizes MPI to communicate with slaves during query execution and utilizes libnf library to read flow data.

Download

You can download SecurityCloud collector source code from git:

• Fdistdump on github
• IPFIXcol on github
• Scripts to deploy collector in cloud environment

Or download deb packages:

• Fdistdump package
• IPFIXcol package
• Libnf package

Publications

• M. Čermák et al: Performance Benchmark of NetFlow Data Analysis on Distributed Stream Processing Systems, In Proceedings of the Network Operations and Management Symposium,  IEEE Xplore Digital Library, 2016
• M. Žádník: Distribuovaný kolektor záznamů o IP tocích: Experimenty s big data platformami, technical report, link.
• M. Žádník: Distribuovaný kolektor záznamů o IP tocích: Návrh a první experiment, technical report, link.

Acknowledgement

The SecurityCloud project is supported by the Technology Agency of the Czech Republic under No. TA04010062 Technology for processing and analysis of network data in big data concept.