#### DMCA

## A comparison of traditional homework to computer-supported homework (2009)

Venue: | Journal of Research on Technology in Education |

Citations: | 12 - 1 self |

### BibTeX

@ARTICLE{Mendicino09acomparison,

author = {Michael Mendicino and Leena Razzaq and Neil T Heffernan},

title = {A comparison of traditional homework to computer-supported homework},

journal = {Journal of Research on Technology in Education},

year = {2009},

pages = {331--359}

}

### OpenURL

### Abstract

Abstract This study compared learning for fifth grade students in two math homework conditions. The paper-and-pencil condition represented traditional homework, with review of problems in class the following day. The Web-based homework condition provided immediate feedback in the form of hints on demand and step-by-step scaffolding. We analyzed the results for students who completed both the paper-and-pencil and the Web-based conditions. In this group of 28 students, students learned significantly more when given computer feedback than when doing traditional paper-and-pencil homework, with an effect size of .61. The implications of this study are that, given the large effect size, it may be worth the cost and effort to give Web-based homework when students have access to the needed equipment, such as in schools that have implemented one-to-one computing programs. (Keywords: online homework, intelligent tutoring systems, online tutoring, homework.) Web-based homework assistance is already popular in colleges. Blackboard (www.blackboard.com), WebAssign, (www.webassign.com), MasteringPhysics (www.masteringphysics.com), and WebWorK (http://webwork.rochester. edu) are all systems that have thousands of student users at the college level, but K-12 Web-based homework assistance lags behind. Systems such as Study Island (www.studyisland.com) and PowerSchool (www.powerschool.com) are gaining popularity with K-12 teachers, and it seems likely that the use of Webbased homework assistance for K-12 will increase as the digital divide between students narrows, teachers become more comfortable with the technology, and teachers gain access to systems that are low cost or free. The important question is, do such systems help students learn more than traditional paper-and-pencil homework? With recent advances in educational technology, teachers now have a multitude of tools to assist and enhance student learning and motivation. New intelligent tutoring systems that guide students through math problems much the same way human tutors do have been successful in helping students learn math in the classroom. Some systems attempt to imitate a human tutor by reproducing the interactive dialogue patterns and strategies that were likely to be used by a human tutor, whereas others provide immediate feedback by highlighting each step attempted in either red or green to indicate a right or wrong answer. They may also provide hint sequences to students asking for help. A U.S. Congress-mandated study In this study, we attempt to determine if fifth grade students can learn more by doing their math homework with a Web-based intelligent tutoring system than when doing traditional paper-and-pencil homework. We conducted this experiment using the ASSISTment System, a Web-based system that provides both interactive scaffolding and hints on demand. We will review studies of Web-based homework assistance systems before describing the ASSISTment system used in this evaluation. LiTerATure review The Use of Web-based Systems for Homework Web-based systems that allow students to do their homework online such as Blackboard, WebCT (www.webct.com), Homework Service (https://hw.utexas. edu/bur/overview.html), and WebWorK are becoming more widely used in higher education. At the K-12 level, systems such as Study Island and PowerSchool are gaining popularity among teachers. Some states in the United States, including Maine, Indiana, Michigan, and Virginia, have begun to implement one-to-one computing Journal of Research on Technology in Education 333 Some advantages of homework-assistance systems are immediate feedback to students and automatic grading and recording of grades for instructors. Automatic grading saves time for teachers who would like to grade all of their students' paper-and-pencil homework carefully by hand but do not have time. In turn, this can prompt students to take homework more seriously because they know it will be graded and the grade will be recorded. With these systems, students can often get immediate feedback on their answers and work and sometimes help toward solving problems. Although these Web-based homework-assistance systems can provide benefits, they can have disadvantages, as well. Many of these systems do not take students' work into consideration when they require students to enter a single answer for each problem. Students may be less organized because they do less scrap work on paper and try to do more math in their heads. Teachers may be less able to figure out exactly where students are having difficulties without seeing their work. Finally, because these systems often do not consider student work, cheating may be easier among students because they could possibly get the answers from their friends without having to show how they arrived at them. Web-based Homework Versus Paper-and-Pencil Homework Previous research has shown positive results for using Web-based homework assistance instead of traditional paper-and-pencil homework. MasteringPhysics, a Web-based physics homework tutor developed at MIT, uses mastery learning to help students reach mastery when solving physics homework problems. Students can ask for hints on problems and receive feedback on common student errors. Some hints will ask the student a question that behaves like a "scaffolding question" in the ASSISTment system (described in the next section). Warnakulasooriya & Pritchard (2005) found that twice as many students could complete a set of problems in a given time with the help provided by MasteringPhysics when compared to students who worked on the problems without help (administered by MasteringPhysics but without hints or feedback). The Andes system is an intelligent tutoring system that provides support for problem solving for physics homework. Students using Andes complete whole derivations step by step and receive feedback after each step. Students can also request hints for each step to find out where their errors are (What's Wrong Help) or to find out what to do next (Next Step Help). VanLehn et al. (2005) presented evidence from introductory physics courses taught at the U.S. Naval Academy from 1999 to 2003 that students who used Andes for homework got significantly higher exam scores than students in control groups who did paperand-pencil homework. Other studies of Web-based physics homework versus paper-and-pencil homework did not find significant differences between the two homework conditions The ASSISTment System Assistance and assessment are integrated in a Web-based system called the ASSISTment System, which offers instruction to students while providing a detailed evaluation of their abilities to teachers. Each time students work on the Web site, the system "learns" more about their abilities. The ASSISTment System is being built to identify the difficulties individual students--and the class as a whole--are having, and teachers will be able to use this detailed feedback to tailor their instruction to focus on those difficulties. Unlike other assessment systems, the ASSISTment system also provides students with intelligent tutoring assistance while assessment information is collected. Our hypothesis is that ASSISTments can do a better job of getting students to learn from homework than traditional paper-and-pencil homework by providing immediate feedback on answers and tutoring for items that students get wrong. Teachers can also assess their students' knowledge limitations better by noting the amount and nature of the assistance students need to finish their homework. The ASSISTment system was developed using grants from the U.S. Department of Education and the National Science Foundation and will be freely available for use by teachers and students. The ASSISTment System provides students with two types of tutoring assistance, scaffolds and hints, when they answer a question incorrectly. With scaffolds, the student who answers an ASSISTment incorrectly receives the message, "Hmm, no. Let me break this down for you," and is immediately presented with a scaffolding question that the student must answer correctly in order to continue and receive the next scaffolding question. The scaffolding questions break the problem into steps, and students must answer the scaffolding questions before returning to the original question; that is, the student is forced to work through the problem. With hints, the student may request a hint by pressing the hint button. Students may request more hints until they reach the "bottom-out" hint, which will typically give them the answer to the question. When students log in to the ASSISTment system, they are presented with math problems. Journal of Research on Technology in Education 335 Ms. Lindquist, developed by Heffernan and Koedinger The ASSISTment system also breaks problems down for students in the way that Cognitive Tutors and Ms. Lindquist do, but it is not rule based. During the design stage of the ASSISTment system, middle school math teachers were involved in the process Journal of Research on Technology in Education 337 form," which they used to implement the ASSISTment. Teachers gave their opinions on the first draft of the ASSISTment and were asked to edit it. They also videotaped these review sessions and revised the design form as needed. When the teacher was satisfied, they released the ASSISTment for use. Initial studies of the ASSISTment system MeTHod Setting and Participants The setting for this study was 4 fifth grade classrooms and students' home computers. The school was located in a small town in a rural county and was a sample of convenience. Approximately 350 students were enrolled in the school at the time of the study, with at least 50% receiving free or reduced lunch. All four classes were typical elementary classes with a mix of below-average, average, and above-average students. Teachers gave a total of 92 students (54 with Internet access at home) this homework assignment, depending on their access. The breakdown of the participants is shown in Content We used two problem sets in both the Web-based homework and the paperand-pencil homework assignments, each consisting of 10 problems. One problem set consisted of Number Sense problems, and the other was a mix of problems. The Number Sense problem set included problems for which students had to demonstrate understanding of numbers, ways of representing numbers, and relationships among numbers and number systems. The Mixed problem set included problems in the algebra, geometry, data analysis, and probability domains. Students had to demonstrate understanding of patterns, relations, and functions; describe spatial relationships using coordinate geometry and other representational systems; develop and evaluate inferences and predictions that are based on models; and apply and demonstrate an understanding of basic concepts of probability. (See Appendix B for the problems in both homework sets.) Students in the classes had prior learning experience with the homework material during the course of the school year. However, as the experiment took place at the end of the school year, it was not recent experience and was more of a review.