Offloading to hardware components that support the primary task of a system enables sea-paration of concerns and allows both the primary and offloaded components of a system to be easy to understand, manage, and evolve independent of other components. In this dissertation, we explore the software mechanisms required to effectively offload functionality to idle processing elements. We present the design, implementation, and evalua-tion of three system architectures – TCPServers, Orion, and FileWall, which offload functional-ity for improving performance (TCPServers), improving availability (Orion), and for extending functionality (FileWall). We explore software mechanisms to offload functionality to a subset of processors in an Symmetric Multiprocessor (SMP) system, a programmable network inter-face, and an interposing network middlebox to realize the three system architectures. TCPServers is a system architecture that offloads network processing to a subset of proces-sors in an SMP system. Network processing imposes direct and indirect overheads on server systems. It directly affects system performance since it executes at a higher priority than ap-plication tasks and prevents other components of the system from executing simultaneously on the processors. It indirectly affects performance by causing cache pollution and Trans-