Abstract

Abstract Joint attention between hearing children and their caregivers is typically achieved when the adult provides spoken, auditory linguistic input that relates to the child's current visual focus of attention. Deaf children interacting through sign language must learn to continually switch visual attention between people and objects in order to achieve the classic joint attention characteristic of young hearing children. The current study investigated the mechanisms used by sign language dyads to achieve joint attention within a single modality. Four deaf children, ages 1;9 to 3;7, were observed during naturalistic interactions with their deaf mothers. The children engaged in frequent and meaningful gaze shifts, and were highly sensitive to a range of maternal cues. Children's control of gaze in this sample was largely developed by age two. The gaze patterns observed in deaf children were not observed in a control group of hearing children, indicating that modalityspecific patterns of joint attention behaviors emerge when the language of parent-infant interaction occurs in the visual mode. The child's ability to engage in joint attention with people and objects in the world is a fundamental cognitive process requiring perceptual, memory, categorization, and information processing abilities. Much research grounded in a social interactional framework From the earliest months of life, caregivers engage in face-to-face interactions with their infants, responding to their vocalizations, displaying positive affect, and speaking in a special register known as motherese, or child-directed speech NIH Public Access Author Manuscript Lang Learn Dev. Author manuscript; available in PMC 2015 January 01. NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript attention, for example by pointing to an object and then checking their interaction partner's gaze direction after pointing In signed languages such as ASL, all linguistic information is presented visually, through signs, facial expressions, and subtle body movements. Perceiving language in the visual mode poses a challenge to the typical developmental course of joint attention. Language cannot be perceived without visual attention to the interlocutor; however, if a child's attention is drawn away from an object or event to which he or she is attending, the relationship between the linguistic input and the object may not be apparent. A system that is typically based on multimodal input must therefore adapt to a situation in which all input is uni-modal. Understanding how each partner's behavior during dyadic interaction differs when all information is perceived through the visual mode can inform theories of development of joint attention across a range of situations revealing adaptive skills in both adults and children. Prior work on interactions between deaf children and their caregivers has focused primarily on how the mothers adapt to interaction in the visual mode. For example, studies of interactions with young deaf infants have established that deaf mothers alter their sign language input in systematic ways using a unique register known as child-directed signing (Maestas y Moores, 1980; A few studies have also examined changes in maternal signing strategies that occur with development. Specifically, over the child's second year, deaf parents have been found to use less explicit attention-getting strategies, and instead rely more upon the deaf child's developing ability to interact in the visual mode A further complication in studying deaf children's joint attention is the fact that the vast majority are not exposed to language on a typical time course, due to the fact that approximately 95% of children are born to hearing parents who have no prior experience communicating through sign language Among hearing children, one type of interaction that has received much study is joint book reading, due to the established links between the frequency and quality of caregiver-child book reading and children's language and literacy outcomes Despite the well-documented strategies used by deaf mothers, the way in which deaf children acquire the ability to interact in the visual mode remains largely undocumented. Studies have yet to reveal the types of skills that enable deaf children to engage in coordinated joint attention with their caregivers. That is, there is little understanding of how children adapt to the cognitive requirements of continuously alternating their own visual attention in order to achieve joint attention with their interlocutors. The goal of the present study was to provide a detailed account of children's development of joint attention in the single modality of vision. This study builds on previous work in two Lieberman et al. Page NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript important ways. First, while earlier studies have focused only on maternal modifications and strategies, in the present study we conducted a micro-level analysis of children's own gaze switching patterns. Second, while earlier research includes deaf children up to the age of two, the participants in the current study were between the ages of two and four. This allowed us to track development of joint attention after infancy and observe interactions at a time period in which children are typically acquiring and producing language at a rapid rate. We investigated joint visual attention by analyzing deaf children's gaze behavior during two types of dyadic interactions with their deaf mothers-a joint book reading interaction and an interaction centered around a set of toys. We chose book reading because it represents a complex and linguistically rich setting in which joint attention is crucial for a successful interaction. The toy interactions were included as a contrastive interaction so that we could determine whether the observed gaze patterns were specific to book reading or instead whether they would generalize across settings. We then compared deaf children's gaze patterns to those of a control group of hearing children to determine which patterns of gaze are unique to joint attention in the visual mode. We hypothesized that there would be significant modality-specific behaviors in deaf children's gaze patterns that would enable them to engage in joint attention episodes with their mothers. Beyond the adaptations that have been previously documented in deaf mothers, we predicted that the deaf children would make specific adaptations allowing them to coordinate and control their own visual attention from an early age. Finally, we sought to discover features of interactions among deaf dyads that might further our understanding of the resilience of the joint attention system in a range of situations. Methods Participants Deaf Dyads-Deaf participants were recruited through direct contact with families in an early intervention program at a large school for the deaf. Four deaf children and their deaf mothers participated. The children in the dyads each had two deaf parents, were identified as deaf at birth, had hearing losses ranging from moderate to profound, and had at least one deaf sibling. All of the mothers reported that ASL was the primary language used in the home. All of the children also attended a center-based early intervention program in which ASL was the primary mode of instruction. Thus the children were immersed in ASL exposure both at home and in the classroom setting. Hearing Dyads-A control group of hearing participants was obtained from the Providence Corpus In each session, two cameras were placed in the room to capture head-on views of both the mother and the child. The book reading sessions ranged from approximately 8 minutes to over 30 minutes, and the free play sessions ranged from approximately 15 to 30 minutes. The time was not controlled by the experimenter, but instead was determined by the amount of time that the child remained engaged in the activity. Although two of the participants were siblings (DD2 and DD4), they were filmed on separate occasions and did not participate in the interaction in which they were not the focus child. Vocabulary measure-For three of the four deaf children, ASL vocabulary was assessed using the MacArthur-Bates Communicative Developmental Inventory for ASL (ASL-CDI; Coding and Analysis For the deaf dyads, a five-minute segment of book reading and a five-minute segment of free play were identified for each child from the videotapes. The book reading segments were obtained by identifying the onset of the first observed interaction around one of the provided books that lasted for at least five minutes. From this interaction, the first five minutes during which both the mother's and child's eyes and hands were clearly visible were extracted for analysis. The rationale for choosing a five minute sample was that this represented the length of time for which each dyad had at least one continuous interaction centered around books. This enabled comparison of the interactions across dyads and situations while keeping the time window constant. To obtain the free play sample, the videotapes were reviewed to identify a sustained interaction that lasted for at least five minutes and centered around a single toy or set of toys. As with the book reading episodes, the first five-minute segment of sustained play during which both the mothers' and child's eyes and hands were clearly visible were extracted for coding. For the hearing dyads, a five-minute segment of book reading was identified for each child from the videotaped interactions, beginning at the onset of reading in which the child's gaze was clearly visible. Book reading was chosen as the comparative analysis for the hearing dyads as it was thought to represent a linguistically-rich joint attention interaction. The identified segments were coded using the linguistic annotation system ELAN (the Eudico Linguistic Annotator; Crasborn, Sloetjes, For the deaf mothers and children, all ASL signs were transcribed, and all accompanying non-linguistic activity (i.e. manual gestures, actions, and object manipulation) were coded by a deaf native-user of ASL or by a hearing, highly skilled user of ASL. In addition, the locus of the child's gaze as well as all shifts in gaze were coded at each point in time. The locus of gaze was coded as partner, book/object, or away. Gaze to the partner refers specifically to looks to the interlocutor's face, which is the typical locus of gaze for signers during comprehension (Emmorey, For the hearing dyads, the child's locus of gaze and all gaze shifts were coded. Maternal gaze was not coded as the hearing-dyad interactions were used to compare child gaze patterns only. Reliability-A portion of the interactions (roughly 25%) were coded by a second coder to determine inter-rater agreement. A portion of each dyad's data was re-coded by selecting a random sample that represented approximately 25% of that dyad's total data. Reliability scores were calculated as percent agreement (A/A+D). Percent agreement scores ranged from 82% to 96%. Any disagreements in coding were discussed until a consensus was reached. In cases where it was not possible to reach a consensus, the native signer's codes were used. Percent agreement scores for specific analyses were: locus of child gaze: 87%; number of gaze shifts: 89%; modified signs: 96%; prompts for gaze shifts: 82%. Results Gaze Patterns Mutual Gaze-To determine whether deaf mothers and children spent the majority of their dyadic interactions in mutual gaze, defined as mother and child directing gaze towards one another's eyes, we analyzed gaze patterns. For each dyad, we computed the percent of total interaction time that was spent in mutual gaze for both the book and toy conditions, as well as the average duration of mutual gaze episodes. The percent of time spent in mutual gaze ranged from 4% to 39%, with a mean of 22% of total interaction time spent in mutual gaze. Individual episodes of mutual gaze were extremely short, lasting only 1.4 seconds on average (range .8 to 2.2 seconds). This suggests that parent-child interaction among deaf dyads is not primarily achieved through prolonged mutual gaze, and that other gaze patterns are employed. Similarly, in the toy condition, an average of 15% of interaction time was spent in mutual gaze (range 10% to 23%), and mutual gaze episodes lasted 1.3 seconds (range 1.1 to 1.5 seconds). There were no significant differences in the book vs. toy 1 There was one exception to this, which occurred when a mother produced a classifier sign for STRIPES, in which she signed on her own foot to indicate that the character had striped feet. In this instance, the child looked down to the mother's hands and not her face. Lieberman et al. Page 6 Lang Learn Dev. Author manuscript; available in PMC 2015 January 01. NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript conditions with regard to either the amount of time spent in mutual gaze (Wilcoxon signed rank t-test, S = −3.0, p = .38), or the duration of mutual gaze episodes (S = 0, p = 1.0). The hearing dyads spent significantly less time in mutual gaze than the deaf dyads (Wilcoxon rank sum t-test, Z = −2.17, p = .03), with an average of only 1% of interaction time spent in mutual gaze (range 0% to 4%). Mean gaze duration for the hearing dyads was 0.9 seconds (range 0 to 1.8 seconds); the mean duration of mutual gaze episodes did not differ between the hearing and deaf dyads (Z = −.44, p = .66). Locus of Gaze-Given that the majority of the five-minute interaction was not spent with mother and child looking directly at one another, we asked how gaze was allocated for each interaction partner in the dyad. Specifically, we measured the proportion of maternal and child gaze to either the interaction partner, the object (book or toy), or off-task, (i.e., to a non-related object or to a person other than the interaction partner). For maternal gaze during book reading, there was a fourth potential gaze direction, namely signing space. This refers to a particular gaze in which the mother looks in a specific direction, often straight ahead, as part of ASL discourse, with an accompanying non-manual marker and/or body shift to show character inflection or reported speech (Reilly, McIntire, & Anderson, 1994). Analysis revealed that, although there was individual variation among the deaf dyads, on average both partners were on-task (i.e., looking at either the interlocutor or the joint focus of attention) over 90% of the time, see Gaze Shifts and Duration-Next we asked how the children attended to both the mother and the book simultaneously. We calculated the total frequency of gaze shifts, including those from the book or toy to the interaction partner and those from the partner to the book or toy. Gaze shifts "away," defined as those shifts to or from other objects or people in the room were not included, as those shifts were considered off-task. In addition, as a measure of sustained attention, the duration of each gaze to the book or toy and the mother was computed Children's gazes were generally very short, lasting on average 4.2 seconds to the book and 2.6 seconds to the mother (in the book condition). There were no significant differences in mean gaze duration to either the mother or object for the deaf dyads in book vs. toy conditions (Wilcoxon signed rank, S = 4.0, p = .25 for both mother and object). As expected, the hearing children had longer mean gazes to the book than deaf children, approaching significance (Wilcoxon rank sum, Z = 1.88, p = .06). Conversely, hearing children had shorter mean gaze duration to the mother than did the deaf children, although the differences were not significant (Z = −1.74, p = .08). Hearing children's gazes to the book averaged 58.0 seconds long, while gazes to the mother averaged only 1.4 seconds long. Analysis of gaze shifts by the deaf children revealed extremely frequent shifts between the object and the mother. In each five-minute interaction, deaf children shifted gaze approximately every three to eight seconds. The total number of gaze shifts in the five- 03). One hearing child never shifted gaze towards the mother during the interaction, and the other three shifted gaze 4-12 times throughout the entire episode. Thus, in the present sample frequent and rapid gaze shifting was a behavioral adaptation unique to deaf children who sign to achieve joint attention in the visual mode. Developmental differences-The four deaf dyads represented two different ages; DD1 and DD2 were roughly 24 months, and DD3 and DD4 were roughly 42 months. Developmental trends, as indicated by contrasts between the younger and older dyads, were most apparent in the total number of gaze shifts during the book reading sessions. In the younger dyads, DD1 shifted gaze between the book and the mother 67 times, and DD2 shifted gaze 36 times in the five-minute interaction. In the older dyads by contrast, DD3 shifted gaze 112 times, and DD4 shifted gaze 105 times in five minutes. In the toy condition, there was a less pronounced but still sizeable age-related increase in gaze shifting: in the younger dyads DD1 shifted gaze 65 times and DD2 shifted gaze 57 times, while in the older dyads DD3 shifted gaze 99 times, and DD4 shifted gaze 77 times. Age effects were also observed in the percent of time off-task (i.e. looking away), which was higher for the two younger dyads (15% and 8%) than for the older dyads (6% and 1%), but only in the book reading condition. This suggests that the older children may have been more engaged during these interactions than the younger children. The number of gaze shifts by child also patterned with vocabulary score, although as all deaf children had been exposed to ASL from birth, age and vocabulary were closely related in this sample. Maternal Scaffolding of Visual Attention There were two broad ways in which the deaf mothers attempted to ensure that their signs were visible to the child. The first way was to modify signs by producing them outside of the typical signing space such that they were visible to the child. For example, the mothers occasionally signed on the book or near an object on which the child was currently focusing attention. Each of these modifications was coded. In the book condition, mothers modified an average of 8% of their utterances (range 3% to 15%) either by signing on or near the book, or by leaning in to sign in the child's visual focus of attention. In the toy condition, mothers modified an average of 12% of their utterances (range 2% to 25%), again by signing on or near the toy, or in the child's visual focus of attention. However in both conditions the majority of maternal utterances were produced in the conventional signing space. This meant that, in order to perceive maternal utterances, the child had to either maintain an existing focus on or shift gaze towards the mother. The second way mothers made their signs visible was to elicit a gaze shift in the child using an overt or non-overt prompt. To identify these maternal prompts, the events and actions immediately preceding each gaze shift by the child were coded. These maternal behaviors were found to fall into three categories of behavior that served to adapt to the child's current focus and/or prompt a gaze shift. The categories were as follows: 1. Physical prompts: attention-getters (e.g. a tap or a wave directed towards the child), or a point on or near the book. 2. Linguistic prompts: maternal utterance onset as a prompt to look up, or maternal end of utterance as a prompt to look down. In order to count as a linguistic prompt, NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript the child's gaze shift had to occur within the temporal space of one sign, i.e. the child looked up at or during the first sign in the mother's utterance, or looked down at or within one sign following the end of her utterance. 3. Gaze-based prompts: maternal gaze shifts to or from the book or child that prompt the child to follow her gaze. Gaze and linguistic prompts often occurred together, when the mother simultaneously began an utterance and looked up, or ended an utterance and looked down. These combinations were noted and coded as "Linguistic + Gaze" prompts. In addition to child gaze shifts that were preceded by a maternal prompt, some gaze shifts were initiated by the child, such that the child's gaze shift was either immediately preceded by, or co-occurred with, the child's own sign or point, indicating the child's desire to communicate with the mother. These were coded as "child-initiated" shifts. Finally, any child gaze shift that occurred spontaneously in the middle of a maternal utterance without any observable prompt or associated child action was coded as "no prompt." The distribution of the children's gaze shifts were analyzed according to the type of maternal prompt (or lack thereof) that directly preceded them (see To further investigate the specific maternal behaviors that prompted the children to look at the mother in order to perceive her signing, we looked at the subset of child gaze shifts from the book to the mother. In the book condition, there were 165 gaze shifts that fell into this category. Across all four dyads, 18% were prompted by an attention-getter (i.e. a wave or a tap), 14% were prompted by the mother's utterance onset, and 20% were prompted by a simultaneous maternal gaze up and an utterance onset. Only 8% of shifts were prompted by maternal gaze shift alone, and 3% were preceded by maternal contact with the book (i.e. pointing or turning the page). The remaining gaze shifts to the mother were either childinitiated (17%), or were not preceded by a prompt (19%). Thus, mothers were most effective in getting the child to look up at them when they tapped or waved at the child, or when the mother began a new utterance. In the toy condition, of the 144 gaze shifts to the mother, 24% were prompted by an attention-getter, 22% were prompted by an utterance onset, 42% were child-initiated, and 12% had no prompt. Maternal gaze shift alone was never a prompt for the child to look up, perhaps because it was too subtle to draw children's attention away from the toys. Maternal prompts to look up were also examined by individual dyad Page NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript when the mother lifted her hands to begin an utterance, with our without an accompanying gaze shift, both younger and older children responded to this prompt to look up at the mother, suggesting that the children understood that signs were about to be produced, and they looked up to perceive the input. Similarly, in the toy condition, the two younger children had a higher proportion of looks to the mother in response to an attention-getter than any other maternal prompt, while the two older children had a higher proportion of looks in response to a maternal utterance onset than any other prompt. Thus there were observable developmental differences in the types of prompts to which children responded, with an increase in responsiveness to subtle prompts (i.e. maternal gaze shifts) and linguistic cues (i.e. maternal utterance onset) in the older two children. Discussion The goal of the present study was to determine how the development of joint attention, which is typically distributed across both auditory and visual channels, is adapted to a situation in which language and visual information is perceived within a single modality. We analyzed the gaze behavior of four deaf mother-deaf child dyads in two settings, comparing this to a control group of hearing mother-hearing child dyads. Significant modality-based differences in gaze behavior were found at every level of analysis. Deaf mothers and children spent more time gazing directly at one another's faces than hearing dyads, although the majority of the interaction in both groups was not spent in mutual gaze. Whereas hearing children directed gaze almost entirely to the book, deaf children's gaze was divided between the book or object and the mother. Deaf children gazed to the book in short bouts lasting only a few seconds, while hearing children maintained gaze to the book for a minute or longer. Most strikingly, deaf children shifted gaze between the mother and the book every few seconds throughout the interaction, whereas hearing children rarely shifted gaze. The deaf children's gaze shifts were largely synchronized with specific maternal behaviors, ranging from overt, physical attention-getting devices, to more subtle prosodic and non-manual cues in the sign language stream. Gaze control was largely in place by the age of two, however there were important increases in the degree of sophistication of gaze shifting that occurred after this age. Below we discuss each of these findings with regard to how they might further our understanding of development of joint attention in deaf and hearing populations. The first major finding of this study concerns the unique patterns of gaze observed in deaf children during dyadic interaction. In order to achieve the type of simultaneous attention to both linguistic and non-linguistic information that hearing children receive through two modalities, in a signed interaction both partners must carefully and continually monitor one another's actions, and react to them by shifting gaze at appropriate times. In the joint book reading interactions observed, the hearing children could easily engage in joint attention by maintaining gaze to the book while perceiving auditory language input and rarely if ever looked towards their mothers. The deaf children could not employ the same strategy, as sustained gaze to the book would preclude receiving language input. While mutual gaze comprised a small proportion of the interaction among the deaf dyads, this was not an effective strategy to establish joint attention, as it would not allow the children to make visual connections with objects and pictures. Instead, coordinated joint attention was actively achieved by the children with frequent and carefully timed shifts to the mother and the book or toy. These gaze shifts enabled the child to perceive linguistic input and connect it to the non-linguistic context in a sequential but organized and dynamic fashion. Despite the small sample size of the current study, the findings were robust and consistent across dyads, and we speculate that similar behaviors would be observed in a larger group of deaf children. Thus it appears that joint attention can be achieved in the visual modality through a The deaf children's gaze shifts were meaningful in that they were largely executed in response to cues provided by their mothers. While maternal gaze shifts alone were rarely sufficient to prompt a child's gaze shift, maternal utterance boundaries (with or without an accompanying gaze shift) did serve as an effective cue. Furthermore, the greater number of child gaze shifts that occurred at maternal utterance onset (i.e. with a prompt) relative to those that occurred mid maternal utterance (i.e. with no maternal prompt) suggests that children may have been learning to coordinate their gaze shifts with the timing of maternal linguistic input. This required knowledge of both the physical behavior that tends to precede a signed utterance (i.e. lifting the hands into signing position), as well as a cognitive awareness that meaningful linguistic information would be provided at that moment. The children also responded successfully to overt bids for attention, i.e. taps and waves, which suggests that children at this age had learned that these pragmatic devices in ASL are intended to direct attention to the mother The second major finding of the present study is that children's control of gaze appears to be largely in place by the age of two. The four children in the deaf dyads represented two ages, approximately two years and three-and-a-half years. Across all dyads, mothers produced the majority of signs in conventional signing space and only modified a small proportion of signs into the child's existing focus of attention, which places the burden of attention shifting largely on the child. This suggests that by the age of two, children were already directing their own gaze and switching gaze between the focus of attention and the interlocutor in order to coordinate visual attention to objects and to language. This remarkable control exhibited by children as young as 21 months is likely due to the fact that in their early interactions, their deaf mothers provided visually accessible language, modeled appropriate attention-getting prompts, and supported the children's development of these important skills Although the two younger children showed sophisticated gaze control, the behavior of the two older children suggests that there is continued development and refinement of children's control of visual attention beyond the age of two. The two older children showed a marked increase in the number of meaningful gaze shifts between the book or toy and the mother, producing roughly twice the number of purposeful gaze shifts as the younger children. Thus in the developmental time span from 24 to 48 months, deaf children engaged in the interaction and controlled their own attention with increased sophistication. The younger children, while still shifting frequently, had more shifts away from the interaction, and were still learning when and how to optimally shift attention between the mother and the visual scene of joint attention. Furthermore, the two younger children had either the same number or fewer shifts during the book vs. toy conditions, while the two older children both shifted gaze more frequently during the book than the toy conditions. Joint book reading represents a complex type of visual interaction that is language-dependent. Thus the finding that the older children increased their gaze shifts in this language-laden situation provides further Lieberman et al.