Abstract. Our work proposes a generic architecture for runtime monitoring and optimization of IDS based on the challenge insertion. The challenges, known instances of malicious or legitimate behavior, are inserted into the network traffic represented by NetFlow records, processed with the current traffic and the system’s response to the challenges is used to determine its effectiveness and to fine-tune its parameters. The insertion of challenges is based on the threat models expressed as attack trees with attached risk/loss values. The use of threat model allows the system to measure the expected undetected loss and to improve its performance with respect to the relevant threats, as we have verified in the experiments performed on live network traffic. 1

We present a mechanism for autonomous self-adaptation of a network-based intrusion detection system (IDS). The system is composed of a set of cooperating agents, each of which is based on an existing network behavior analysis method. The self adaptation mechanism is based on the insertion of a small number of challenges, i.e. known instances of past legitimate or malicious behavior. The response of individual system components to these challenges is used to measure and eventually optimize the system performance in terms of accuracy. In this work we show how to choose the challenges in a way such that the IDS attaches more importance to the detection of attacks that cause much damage.