Abstract

What is the impact of garden-based learning on academic outcomes in schools? To address this question, findings across 152 articles Keywords: garden-based learning, academic outcomes, school gardens, environmental learning. Since the early 1990s, school grounds previously covered with asphalt or grass have instead increasingly become sites for growing school gardens. This resurgence of interest in school gardens over the past 20 years has resulted in the establishment of thousands of school gardens across rural and urban areas in the United States of America. Schools in California, Colorado, Florida, Illinois, Iowa, New York, Ohio, Oregon, North Carolina, Texas, and Vermont, among other states, have aligned subject area standards with newly designed garden curricula The first strand centers around obesity, health, and food insecurity. There is heightened interest in teaching students how to grow food, and school grounds are considered prime places for local food production and garden-based learning Although the vast array of topics covered and increased interest in garden-based pedagogy are laudable, there is need for evidence about the extent to which garden-based learning meets academic outcomes if school gardens are to gain legitimacy. Since gardens are on school grounds, there needs to be justification for their academic value. Hence, the overall goal of this exploratory study was to determine what the landscape of research indicates about the impact of gardenbased learning on academic outcomes. A synthesis of existing research was undertaken with a view to identify, select, and analyze the knowledge base of academic What Is Garden-Based Learning? Garden-based learning is an instructional strategy that utilizes a garden as an instructional resource, a teaching tool. "It encompasses programs, activities and projects in which the garden is the foundation for integrated learning, in and across disciplines, through active, engaging, real-world experiences" (Desmond, Grieshop, & Subramaniam, 2002, p. 7). There is proliferation of garden programs and curriculum, including the Boston Schoolyards Initiative, the Common Roots Program in Vermont, Denver Urban Gardens, the Garden-Based Learning Program at Cornell, the Learning Gardens Laboratory in Portland, the Life Lab Science Program at Santa Cruz, the San Francisco Green Schoolyard Alliance, the Edible Schoolyard in Berkeley, the Garden Initiative in Chicago, Urban Harvest in Houston, the Junior Master Gardener programs, 4-H Youth garden programs, and the National Wildlife Federation Schoolyard Habitat Program across most states Method Conceptual Framework for Research In a recent article in Review of Educational Research, Suri and Clarke (2009) presented a new framework of "methodologically inclusive advancements in research synthesis" (MIRS) methods. Going beyond the traditional positivist approaches to meta-analyses, they developed a methodology that we used, as it is conceptualized by distilling and synthesizing ideas, theories, and strategies. Their theoretical approach to advancing research syntheses is appealing because it builds connections. As garden-based learning research tends to be interdisciplinary, we believe MIRS provided the best match for this research synthesis. Furthermore, in Synthesizing Research: A Guide for Literature Reviews, Cooper (1998) stated, Research syntheses focus on empirical studies and seek to summarize past research by drawing overall conclusions from many separate investigations that address related or identical hypotheses. The research synthesist hopes to present the state of knowledge concerning the relation(s) of interest and to highlight important issues that research has left unresolved. (p. 3) As with all exploratory research, we did not have a hypothesis going into the process. The trends emerged in the analysis. We used a wide net to identify empirical research between 1990 and 2010 that examined the impact of garden-based learning on academic performance from the primary grades through the end of high school (12th grade). We used descriptive rather than parametric tools, "paying close attention to details before invoking more technically involved procedures" (Howell, 2002, p. 4). Not only is garden-based learning new, but the research is relatively new and limited. Hence, we gathered extant research in garden-based learning. We selected studies for detailed analysis based on a set of criteria and analyzed the results along the lines of positive, negative, or no impact, and we summarized their findings and trends. at Stockholm University Library on August 14, 2013 http://rer.aera.net Downloaded from Garden-Based Learning and Academic Outcomes 214 Search Process Although gardens have been part of the school landscape in the United States from time to time since the late 19th century, and were particularly popular during World War I and World War II, for our review we chose studies from 1990 to 2010 since much of the recent momentum for establishing school gardens started in the early 1990s and we were able to cover databases until mid-2010 as a cutoff point. The 48 studies that are reviewed resulted from a three-phase process we followed for the identification, selection (elimination or inclusion), and analysis of studies. Studies were selected if they met the following criteria: (a) There was gardenbased curriculum, (b) academic outcomes were measured, (c) exposure consisted of a minimum of an hour at least every 2 weeks, (d) there was connection with schools, and (e) assessment measures were specific to the age group being studied. These criteria were developed after we had randomly reviewed 20 publications of garden programs prior to undertaking Phase 1 of the search process and were also based on pilot site visits to school gardens in eight states undertaken by the primary author Phase 1: Search process to access studies. In the first phase we did a search of electronic databases and journals: Academic Search Complete (EBSCO), Agricola (EBSCO), Biological and Agricultural Index Plus, BioOne, Child Welfare League of America, ProQuest Dissertations & Theses Abstracts Online, Education Full Text, Environmental Science and Pollution Management, ERIC (EBSCO and U.S. government interfaces), Google Scholar, GreenFILE (EBSCO), HortTechnology, informaworld, JSTOR, PsycINFO, and Web of Science (SocSciCitIndex). We queried each database separately rather than use a meta-search engine in order to have a clear record of the source of results, a procedure we followed throughout the research. In formulating the query terms, we used a text editor that allowed word wrapping to be disabled. This assured accurate pasting into the database search windows without line breaks or any unseen symbols typically generated by word processors. Based on the research question, three major search parameters were developed: (a) garden-based education, (b) academic achievement, and (c) evaluation. The most elegant, simple, and effective search strings were developed. Following standard search process, the final search terms were as follows, separated by the Boolean term "AND": • garden or (garden-based learning) or (school garden) or (green school yard) or (green school ground) or horticultur* • academic or (academic achievement) or (academic effect) or (academic outcome) or (academic performance) or (academic skill) or (standard test) or (test score) or outcom* • (program evaluation) or (evaluation research). at Stockholm University Library on August 14, 2013 http://rer.aera.net Downloaded from Williams & Dixon 215 We found that grouping the terms resulted in fewer duplicate sources, eliminating extraneous articles without jeopardizing the final results. For example, we used the term academic achievement instead of academic and achievement by themselves. Several search terms were eliminated because they were found to be contained in a shorter final search. We tested this by adding terms, then comparing the results. For example, when we added the term math, it gave rise to no additional results when compared to the results from the previously described second search term. Furthermore, we did not need to list individual direct or indirect variables because they emerged from the simple search terms used. We excluded from our search terms those that did not add to the outcome of the searches: knowledge, science, motivation, attendance, engagement, literacy, academic efficiency, program effectiveness, academic impact, and academic measure. In performing the queries, we used "apply related terms" but found that it did not result in additions. Also, we eliminated many terms that were rendered redundant with the use of a "wildcard" to make searches more efficient. For example, competenc* was effective in the search for competence, competency, and competencies. Dissertations and Theses Abstracts and JSTOR had limits to the number of terms that could be used. For instance, JSTOR could only take three wildcards, which necessitated modification as follows: • garden* or (garden-based learning) or (school garden) or (green school yard) or (green school ground) • academic* or (academic achievement) or (academic performance) or (standard* test) or • (test score) or outcome or (program evaluation) or (evaluation research). As we were progressing through the search, on the recommendation of the research librarian, we used RefWorks primarily because it allowed for online collaboration from any computer by the two investigators as well as other researchers if they were to be added at a later date. RefWorks permitted backups and exportation of citations into multiple formats, including EndNote and CSV (comma separated values) delimited text files. Next, we used Zotero to acquire references from Google Scholar, informaworld (where Applied Environmental Education & Communication, among others, is housed), and HortTechnology. Published books on the topic were utilized for their reference sources to hand trace further relevant studies. After collecting the citations into RefWorks, we traced and eliminated any duplicate articles that resulted from overlaps in the databases, resulting in 235 citations. Phase 2: Preliminary analysis of studies and fine-tuning. The citations were next color coded: neither pertinent nor relevant to our synthesis (blue), relevant to garden-based education in general but not pertinent to our synthesis (yellow), or directly pertinent to our synthesis (green). For instance, if an article had the term garden in the name of a school or even a city such as Garden Grove, California, but did not use garden-based learning, it was considered neither pertinent nor relevant and marked blue. Articles were considered relevant to garden-based learning but not pertinent and marked yellow if (a) they dealt with garden programs that at Stockholm University Library on August 14, 2013 http://rer.aera.net Downloaded from Garden-Based Learning and Academic Outcomes 216 were based in the community with no explicit connection to schools and/or student learning, (b) they were commentaries on garden-based learning, (c) there was no "treatment" of a garden, or (d) there was no research component. Abstracts of all articles (including journals, dissertations, theses, and reports) were read by the primary author to determine which ones were empirical studies of school-based garden programs. If abstracts were unclear, then the full articles were read. The second author randomly checked 20% of the abstracts; before proceeding, we discussed discrepancies mostly to gain clarity and accuracy over the coding. In keeping with Suri and Clarke (2009), as an emergent synthesis design, we attempted to keep our approach open, raise questions of each other, explore possibilities, and stay in dialogue. Nevertheless, independent coding of the rest of the articles was undertaken next by the second author. Overall discrepancy between the authors for 21 (9%) articles was minor, bordering on green and yellow coded articles. Each discrepancy was discussed to reach consensus. After eliminating the blue and yellow ratings, 152 articles remained. A matrix was developed to classify and catalog all 152 unduplicated articles in a garden-based learning database we created based on reviewing the abstracts and several full-length manuscripts as samples. The two authors then filled in the matrix to further refine the search. We used a compendium of variables from previous readings of the articles as well as the headings from the matrix. The categories (column headings) for analysis in the matrix were modified after the first 35 articles were read at random. Articles often resulted in population of several columns in the matrix. The emerging list of categories included author, date, title, school/district, grade, school demographics, student demographics, research design, data collection, learning outcomes, and subjects covered (e.g., science, math, language arts, social studies, writing). After a few reviews, it became clear to us that we also needed universal categories such as "direct academic outcomes" and "indirect academic outcomes," which were added to the matrix. Any differences in deciding whether to include or exclude an article were discussed until consensus was found. In order to include the articles in the next phase of the synthesis, we asked the following questions: (a) Is the study empirically based, (b) are there pertinent research questions, (c) is the methodology described, (d) is research evidence provided, (e) are links made between garden and school/curriculum/subjects, and (f) are outcomes measured? The caliber of research became critical during our review as we examined each study further for the following characteristics: clear statement of intentions, problem, setting, and arguments; coherent framework of theory, methodology, validity, analysis, and conclusions; detailed information about the study context, methods, and sources of data; well-defined and articulated sample; presentation of an adequate data set; description of data collection methods; findings that emerged from the research process; and conclusions that referred back to the original framework. As an initial assessment, the authors reviewed each article independently and rated it on a 1 to 10 scale; the greater the relevance to the research question, namely, examining the impact of school gardens on academic learning, the higher the score on relevance. Furthermore, studies with a higher quality of research and writing received higher ratings. Based on aforementioned categories, studies that reported an impact on indirect academic variables (e.g., discipline, bonding, social skills) but did not measure direct academic impact (e.g., test scores, grades, GPA) were at Stockholm University Library on August 14, 2013 http://rer.aera.net Downloaded from Williams & Dixon 217 scored no higher than 7 if written clearly. We operationalized direct academic variables as subjects including science, math, language arts, writing, and social studies. Apart from the national urgency for improvement in science, language arts, math, and writing, another reason for the focus on direct academic outcomes is the relative standardization of assessments for these subjects when compared to indirect variables. Both investigators reviewed all articles, with surprisingly close results. None of the ratings were more than 2 points off the other investigator's ratings. Although 38 out of 152 had slightly differing scores, these studies were re-read and the scores were discussed to reach consensus. For the next phase, only those studies rated 6 or higher (i.e., above the mean) were included, as the overall pool of articles was weak in research methodology and the distribution resulted in a natural break between the ratings of 5 and 6, resulting in the inclusion of 52 studies. Four were dropped because they were dissertations or theses that were subsequently published in a peer-reviewed journal already in this set of studies. These procedures resulted in 48 studies for final review. During the search process, we found two peer-reviewed articles Ozer Phase 3: Design of template and analysis of studies. After the matrix was developed and populated and the list of articles was narrowed to 48, a template was designed and The data from the template were entered into an Excel spreadsheet directly from the 48 templates using the categories enlisted in the template. Once data were verified, an SPSS database was developed from the headings and variable types (numeric, string, etc.) used in the spreadsheet. These data were then imported into SPSS. Variable names and labels were developed to mirror the template, and missing cases labels were developed where appropriate. Also, new variables were developed from original data to enhance statistical analyses. For example, a study that indicated "Grades 1-3" resulted in positive dummy variables for first, second, and third grades. The results of all additional variables and transformations were verified on a case-by-case basis and computations performed according to generally accepted techniques in educational and sociological research.