According to cognitive science literature, human memory is predicated on contextual cues (e.g., room, music) in the environment. During recall tasks, we associate information/activities/objects with contextual cues. However, computer systems do not leverage our natural process of using contextual cues to facilitate recall. We present a new interaction technique, Pivoting, that allows users to search for contextually related activities and find a target piece of information (often not semantically related). A sample motivation for contextual search would be, “what was that website I was looking at when Yesterday by The Beatles was last playing? ” Our interaction technique is grounded in the cognitive science literature, and is demonstrated in our system YouPivot. In addition, we present a new personal annotation method, called TimeMarks, to further support contextual recall and the pivoting process. In a pilot study, participants were quicker to identify websites, and preferred using YouPivot, compared to current tools. YouPivot demonstrates how principles of human memory can be applied to enhance the search of digital information.