As intelligent tutoring system (ITS) issues are investigated and intelligent tutoring systems are developed, evaluation methodology becomes important. Basic researchers, system developers, and educators working with ITS all have motives for becoming involved in ITS evaluation. In formative evaluation, researchers examine a system under development, to identify problems and guide modifications. By contrast, summative evaluation is carried out to support formal claims about the construction, behaviour of, or outcomes associated with a completed system. Different methodologies are suitable for different types of evaluation, some focusing on internal considerations, such as architecture and behaviour, others on external considerations, such as educational impact. This paper draws upon the areas of intelligent tutoring systems research, expert systems design, computer-based instruction, education, and psychology to identify techniques for the formative and summative evaluation of ITS. Evalu...

The advent of second-generation intelligent computer tutors raises an important instructional design question: when should tutorial advice be presented in problem solving? This paper examines four feedback conditions in the ACT Programming Tutor. Three versions offer the student different levels of control over error feedback and correction: (a) immediate feedback and immediate error correction; (b) immediate error flagging and student control of error correction; (c) feedback on demand and student control of error correction. A fourth, No-tutor condition offers no step-by-step problem solving support. The immediate feedback group with greatest tutor control of problem solving yielded the most efficient learning. These students completed the tutor problems fastest, and the three tutorsupported groups performed equivalently on tests. Questionnaires revealed little student preference among the four conditions. These results suggest that students will need explicit guidance to benefit from learning opportunities that arise when they have greater control over tutorial assistance.

Across many domains, Intelligent Tutoring Systems (ITSs) are used to facilitate practice, providing a customized learning environment and personal tutoring experience for students to learn at their own pace through effective student modeling and feedback. Most current ITSs are built around cognitive learning theories including Ohlsson’s theory on learning from performance errors and Anderson’s ACT theories of skill acquisition which focus primarily on providing negative feedback or corrective feedback, facilitating learning by correcting errors. Research into the behavior and methods used by expert tutors suggest that experienced tutors use positive feedback quite extensively and successfully. This research investigates positive feedback; learning by capturing and responding to correct behavior, supported by cognitive learning theories. The research aim is to develop and implement a systematic approach to delivering positive feedback in Intelligent Tutoring Systems, in particular SQL-Tutor, a constraint-based tutor which instructs users in the design of Structured Query Language (SQL) database queries. An evaluation study was conducted at the University of Canterbury involving a control group of students who used the original version of SQL-Tutor giving only

Iteration can help people improve ideas. It can also give rise to fixation, continuously refining one option without considering others. Does creating and receiving feedback on multiple prototypes in parallel, as opposed to serially, affect learning, self-efficacy, and design exploration? An experiment manipulated whether independent novice designers created graphic Web advertisements in parallel or in series. Serial participants received descriptive critique directly after each prototype. Parallel participants created multiple prototypes before receiving feedback. As measured by clickthrough data and expert ratings, ads created in the Parallel condition significantly outperformed those from the Serial condition. Moreover, independent raters found Parallel prototypes to be more diverse. Parallel participants also reported a larger increase in task-specific self-confidence. This article outlines a theoretical foundation for why parallel prototyping produces better design results and discusses the implications for design education.

This paper reviewed the extensive evidence on the effectiveness of feedback on learning. The research supported five claims about feedback. First, informational feedback is effective in domains with clear right or wrong answers when tested immediately after training. Second, when the same maximal feedback conditions are tested for retention or transfer, they are less effective than conditions with less feedback. Third, feedback can draw attention away from the learning task. Fourth, feedback apparently plays a minor role in actual classroom situations. Fifth, teaching students to provide their own feedback and explanation is an effective alternative. These findings suggest that instructors may be more effective if they put less effort into grading and commenting on students ' products and more effort into structuring their courses to help students learn how to assess and reflect on their state of learning themselves. Two specific pedagogical strategies are suggested. First, giving students more assignments than the instructor could grade or comment on will provide more of the kinds of practice they need to develop expertise. Second, helping students to learn how to assess and reflect on their state of learning will help them learn how to provide their own feedback and thus help them to become independent life-long learners. I.

Online course offerings have not only gained in popularity among technology-mediated training methods, but they also have produced a prominent change in the landscape of academia. It is therefore imperative to obtain a solid understanding of the important elements that contribute to effective online learning. The major contribution of this paper is to investigate a complex set of interrelated factors in the relatively new sphere of online learning. The intercombination of these particular factors appears to be important from past research, but it has never been explicitly addressed, and never in an experimental setting. Findings of this study have shown that feedback and learner control have a significant interaction effect for declarative knowledge acquisition. For satisfaction, however, feedback is only salient for declarative knowledge learning, and not for procedural knowledge learning. Other factors, such as interest and comfort level produce effects in most situations.

Abstract not found