Alamut: a high‐performance network intrusion detection system in support of virtualized environments

ABSTRACTOne of the benefits of virtualization technology is the provision of secure and isolated computing environments on a single physical machine. However, the use of virtual machines for this purpose often degrades the overall system performance that is due to emulation costs, for example, packet filtering on every virtual machine. To allow virtual machines to be favorably used as before for the provision of secure environments but with comparably less performance degradation, we propose a new architecture called Alamut in this paper for restructuring any typical network intrusion detection system (NIDS) to run in a Xen‐based virtual execution environment. In the proposed architecture, primitive mechanisms for implementing the security concerns of typical NIDSs such as signature matching are placed at the kernel level of driver domain (dom0), whereas security policies and management modules are kept in user space of that domain. Separation of mechanisms from policies allows network packets to be verified at the kernel level first hand more efficiently without requiring costly context switches to push them to user space for validation. In addition, system administrators can easily define new policies at user level and determine on which virtual machines these policies should be enforced. A proof‐of‐concept implementation of Alamut has been prototyped on the Xen hypervisor using Bro open‐source NIDS. Experimental results show approximately 3.5‐fold increase in the overall system performance when our prototype is run compared with when Bro is run. Results also show 19% improvement in network throughput. The comparison of Alamut with Snort with the same set of signatures and attacks shows that our prototyped NIDS has lower processor utilization and has captured more packets in heavy network loads. Copyright © 2013 John Wiley & Sons, Ltd.