Protocol reverse engineering, the process of extracting the application-level protocol used by an implementation, without access to the protocol specification, is important for many network security applications. Recent work [17] has proposed protocol reverse engineering by using clustering on network traces. That kind of approach is limited by the lack of semantic information on network traces. In this paper we propose a new approach using program binaries. Our approach, shadowing, uses dynamic analysis and is based on a unique intuition—the way that an implementation of the protocol processes the received application data reveals a wealth of information about the protocol message format. We have implemented our approach in a system called Polyglot and evaluated it extensively using real-world implementations of five different protocols: DNS, HTTP, IRC, Samba and ICQ. We compare our results with the manually crafted message format, included in Wireshark, one of the state-ofthe-art protocol analyzers. The differences we find are small and usually due to different implementations handling fields in different ways. Finding such differences between implementations is an added benefit, as they are important for problems such as fingerprint generation, fuzzing, and error detection.

Abstract. Defending a server against Internet worms and defending a user’s email inbox against spam bear certain similarities. In both cases, a stream of samples arrives, and a classifier must automatically determine whether each sample falls into a malicious target class (e.g., worm network traffic, or spam email). A learner typically generates a classifier automatically by analyzing two labeled training pools: one of innocuous samples, and one of samples that fall in the malicious target class. Learning techniques have previously found success in settings where the content of the labeled samples used in training is either random, or even constructed by a helpful teacher, who aims to speed learning of an accurate classifier. In the case of learning classifiers for worms and spam, however, an adversary controls the content of the labeled samples to a great extent. In this paper, we describe practical attacks against learning, in which an adversary constructs labeled samples that, when used to train a learner, prevent or severely delay generation of an accurate classifier. We show that even a delusive adversary, whose samples are all correctly labeled, can obstruct learning. We simulate and implement highly effective instances of these attacks against the Polygraph [15] automatic polymorphic worm signature generation algorithms. Key words: automatic signature generation, machine learning, worm, spam 1

Abstract. In this paper, we give an overview of the BitBlaze project, a new approach to computer security via binary analysis. In particular, BitBlaze focuses on building a unified binary analysis platform and using it to provide novel solutions to a broad spectrum of different security problems. The binary analysis platform is designed to enable accurate analysis, provide an extensible architecture, and combines static and dynamic analysis as well as program verification techniques to satisfy the common needs of security applications. By extracting security-related properties from binary programs directly, BitBlaze enables a principled, root-cause based approach to computer security, offering novel and effective solutions, as demonstrated with over a dozen different security applications.

Recent work has established the importance of automatic reverse engineering of protocol or file format specifications. However, the formats reverse engineered by previous tools have missed important information that is critical for security applications. In this paper, we present Tupni, a tool that can reverse engineer an input format with a rich set of information, including record sequences, record types, and input constraints. Tupni can generalize the format specification over multiple inputs. We have implemented a prototype of Tupni and evaluated it on 10 different formats: five file formats (WMF, BMP, JPG, PNG and TIF) and five network protocols (DNS, RPC, TFTP, HTTP and FTP). Tupni identified all record sequences in the test inputs. We also show that, by aggregating over multiple WMF files, Tupni can derive a more complete format specification for WMF. Furthermore, we demonstrate the utility of Tupni by using the rich information it provides for zeroday vulnerability signature generation, which was not possible with previous reverse engineering tools.

The vulnerabilities that plague computers cause endless grief to users. Slammer compromised millions of hosts in minutes; a hit-list worm would take under a second. Recently proposed techniques respond better than manual approaches, but require expensive instrumentation, which limits deployment. Although spreading “antibodies ” (e.g. signatures) ameliorates this limitation, hosts depending on antibodies are defenseless until inoculation; to the fastest hit-list worms this delay is crucial. Additionally, most recently proposed techniques cannot provide recovery to provide continuous service after an attack. We propose a novel solution called Sweeper that provides both fast and accurate post-attack analysis and efficient recovery with low normal execution overhead. Sweeper innovatively combines several techniques: (1) Sweeper uses lightweight monitoring techniques to detect a wide array of suspicious requests, providing a first level of defense. (2) By cleverly leveraging lightweight checkpointing, Sweeper postpones heavyweight monitoring until absolutely necessary — after an attack is detected. Sweeper rolls back and re-executes multiple times to dynamically apply heavyweight analysis techniques via dynamic binary instrumentation. Since only the execution involved in the attack is analyzed, the analysis is efficient, yet thorough. (3) Based on the analysis results, Sweeper automatically generates lowoverhead antibodies to prevent future attacks of the same vulnerability. (4) Finally, Sweeper again re-executes to perform fast recovery for continuous service. We implement Sweeper in a real system. Our experimental results with three real-world servers and four real security vulnerabilities show that Sweeper can detect an attack and generate antibodies in under 60 milliseconds. Our results also show that Sweeper imposes under 1 % overhead during normal execution, clearly suitable for widespread production deployment (especially since Sweeper also allows partial deployment). Finally, we analytically show that, for a

In biology, a vaccine isaweakenedstrainofavirusorbacterium that is intentionally injected into the body for the purpose of stimulating antibody production. Inspired by this idea, we propose a packet vaccine mechanism that randomizes address-like strings in packet payloads to carry out fast exploit detection, vulnerability diagnosis and signature generation. An exploit with a randomized jump address behaves like a vaccine: it will likely cause an exception in a vulnerable program’s process when attempting to hijack the control flow, and thereby expose itself. Taking that exploit as a template, our signature generator creates a set of new vaccines to probe the program, in an attempt to uncover the necessary conditions for the exploit to happen. A signature is built upon these conditions to shield the underlying vulnerability from further attacks. In this way, packet vaccine detects and filters exploits in a black-box fashion, i.e., avoiding the expense of tracking the program’s execution flow. We present the design of the packet vaccine mechanism and an example of its application. We also describe our proof-of-concept implementation and the evaluation of our technique using real exploits.

Current taint tracking systems suffer from high overhead and a lack of generality. In this paper, we solve both of these issues with an extensible system that is an order of magnitude more efficient than previous software taint tracking systems and is fully general to dynamic data flow tracking problems. Our system uses a compiler to transform untrusted programs into policy-enforcing programs, and our system can be easily reconfigured to support new analyses and policies without modifying the compiler or runtime system. Our system uses a sound and sophisticated static analysis that can dramatically reduce the amount of data that must be dynamically tracked. For server programs, our system’s average overhead is 0.65% for taint tracking, which is comparable to the best hardware-based solutions. For a set of compute-bound benchmarks, our system produces no runtime overhead because our compiler can prove the absence of vulnerabilities, eliminating the need to dynamically track taint. After modifying these benchmarks to contain format string vulnerabilities, our system’s overhead is less than 13%, which is over 6 × lower than the previous best solutions. We demonstrate the flexibility and power of our system by applying it to file disclosure vulnerabilities, a problem that taint tracking cannot handle. To prevent such vulnerabilities, our system introduces an average runtime overhead of 0.25 % for three open source server programs.

Software failures in server applications are a significant problem for preserving system availability. We present AS-SURE, a system that introduces rescue points that recover software from unknown faults while maintaining both system integrity and availability, by mimicking system behavior under known error conditions. Rescue points are locations in existing application code for handling a given set of programmer-anticipated failures, which are automatically repurposed and tested for safely enabling fault recovery from a larger class of (unanticipated) faults. When a fault occurs at an arbitrary location in the program, ASSURE restores execution to an appropriate rescue point and induces the program to recover execution by virtualizing the program’s existing error-handling facilities. Rescue points are identified using fuzzing, implemented using a fast coordinated checkpoint-restart mechanism that handles multiprocess and multi-threaded applications, and, after testing, are injected into production code using binary patching. We have implemented an ASSURE Linux prototype that operates without application source code and without base operating system kernel changes. Our experimental results on a set of real-world server applications and bugs show that ASSURE enabled recovery for all of the bugs tested with fast recovery times, has modest performance overhead, and provides automatic self-healing orders of magnitude faster than current human-driven patch deployment methods.

We present a new technique that enables software recovery in legacy applications by retrofitting exception-handling capabilities, error virtualization using rescue points. We introduce the idea of “rescue points ” as program locations to which an application can recover its execution in the presence of failures. The use of rescue points reduces the chance of unanticipated execution paths thereby making recovery more robust by mimicking system behavior under controlled error conditions. These controlled error conditions can be thought of as a set erroneous inputs, like the ones used by most quality-assurance teams during software development, designed to stress-test an application. To discover rescue points applications are profiled and monitored during tests that bombard the program with bad/random inputs. The intuition is that by monitoring application behavior during these runs, we gain insight into how programmer-tested program points are used to propagate faults gracefully. 1

Complex computer systems are plagued with bugs and vulnerabilities. Worms such as SQL Slammer and hit-list worms exploit vulnerabilities in computer programs and can compromise millions of vulnerable hosts within minutes or even seconds, bringing down vulnerable critical services. In this paper, we propose an end-to-end self-healing approach to achieve the following goal: for a large class of vulnerabilities and attacks, we can protect a large fraction of critical services and enable them to be highly available even in the case of a zero-day hit-list worm. Moreover, our techniques do not require access to source code and thus work on COTS software. We achieve this goal by designing an end-to-end self-healing approach: (1) programs use light-weight techniques to efficiently self-monitor the execution behavior and reliably detect a large class of errors and exploits, (2) we use sophisticated techniques to self-diagnose the root cause of detected errors and exploits, (3) programs self-harden to be resilient against further attacks on the same vulnerability, and (4) safely and efficiently self-recover to a safe state. Self-hardening does not result in false positives of legitimate traffic, and adds little performance overhead. Moreover, our approach allows a community of nodes to efficiently share Self-Verifiable Antibody Alerts (SVAAs), which are produced by the self-diagnosis engine. Nodes can verify that SVAAs fix real vulnerabilities without trusting the SVAA senders, and self-harden quickly and efficiently based upon SVAAs. By employing a new approach of combining proactive protection and reactive anti-body defense, we show for the first time that it is possible to protect vulnerable programs and enable critical services to remain undisrupted even under extremely fast worm attacks such as hit-list worms. 1