## Natural Language Dialog with a Tutor System for Mathematical Proofs (2007)

Venue: | JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY |

Citations: | 8 - 5 self |

### BibTeX

@ARTICLE{Benzmüller07naturallanguage,

author = {Christoph Benzmüller and Helmut Horacek and Ivana Kruijff-Korbayová and Manfred Pinkal and Jörg Siekmann and Magdalena Wolska},

title = {Natural Language Dialog with a Tutor System for Mathematical Proofs},

journal = {JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY},

year = {2007}

}

### OpenURL

### Abstract

Natural language interaction between a student and a tutoring or an assistance system for mathematics is a new multi-disciplinary challenge that requires the interaction of (i) advanced natural language processing, (ii) flexible tutorial dialog strategies including hints, and (iii) mathematical domain reasoning. This paper provides an overview on the current research in the multi-disciplinary research project Dialog, whose goal is to build a prototype dialog-enabled system for teaching to do mathematical proofs. We present the crucial sub-systems in our architecture: the input understanding component and the domain reasoner. We present an interpretation method for mixed-language input consisting of informal and imprecise verbalization of mathematical content, and a proof manager that supports assertion-level automated theorem proving that is a crucial part of our domain reasoning module. Finally, we briefly report on an implementation of a demo system.

### Citations

560 |
Untersuchungen über das logische Schliessen
- Gentzen
- 1935
(Show Context)
Citation Context ...the mere existence of proofs. For instance, evaluation of (a) boils down to judging the complexity of the generated proof task (P1). Let us, for example, use Gentzen’s natural deduction (ND) calculus =-=[12]-=- as the proof system ⊢. As a first and naive logical granularity measure, we may determine the number of ⊢-steps in the smallest ⊢-proof of the proof task for the proof step utterance in question; thi... |

130 |
The Psychology of Proof
- Rips
- 1994
(Show Context)
Citation Context ...A ⇔ C) may actually expand into base-level ND proofs of completely different size. Related research has shown that the standard ND calculus does not adequately reflect human-reasoning in this respect =-=[24]-=-. Two important and cognitively interesting questions thus concern the appropriate choice of a proof system ⊢ and ways to measure the ‘argumentative complexity’ of an admissible proof step. 3.2 Releva... |

84 |
Collaborative dialogue patterns in naturalistic one-toone tutoring. Applied cognitive psychology
- Graesser, Person, et al.
- 1995
(Show Context)
Citation Context ...error correction are among the most prominent features of naturalistic one-to-one tutoring and that efficient tutoring exhibits certain dialog patterns characteristic of these collaborative processes =-=[16]-=-. In our project, we aim at a flexible tutorial dialog in which students interact with the system by proposing proof steps using an unconstrained mixture of natural language and mathematical symbols, ... |

80 |
The meaning of the sentence in its semantic and pragmatic aspects
- Sgall, Hajiìova
- 1986
(Show Context)
Citation Context ...l-formed fragments, and constructs a representation of their linguistic meaning (LM). The LM is represented as a relational dependency structure closely corresponding to the tectogrammatical level in =-=[26]-=-. To obtain the LM, we use the OpenCCG parser (openccg.sourceforge.net) for which we develop a lexically-based Combinatory Categorial Grammar for German [13]. Our motivation for using this grammar fra... |

63 | Reconstructing proofs at the assertion level
- Huang
- 1994
(Show Context)
Citation Context ...ine. The application of our approach to mathematical domains that are more challenging than naive set theory, and its evaluation therein is ongoing work. The hypothesis that assertion level reasoning =-=[20]-=- plays an essential role in evaluating appropriateness of underspecified partial proofs has been confirmed. The fact that assertion level reasoning may be highly underspecified in human-constructed pr... |

58 |
The information state approach to dialogue management
- Traum, Larsson
- 2001
(Show Context)
Citation Context ...user interface, an input analyzer, a proof manager, a tutorial manager, and a natural language generator. The modules are connected and controlled by an Information State Updatebased dialogue manager =-=[29]-=-. Our research and, in particular, the implementation focus has been mainly on the input analyzer, whose task is to interpret and formally represent the linguistic content of the student’s dialog cont... |

41 | MBase: Representing knowledge and context for the integration of mathematical software systems
- Kohlhase, Franke
- 2001
(Show Context)
Citation Context ...epant views of linguistic analysis and deduction systems’ representation [17]; it thus has a potential of providing a direct link to logical definitions in a mathematical knowledge base, suchas MBase =-=[21]-=-. The motivation for using an intermediate representation instead of directly accessing a mathematical knowledge base will become clear when we discuss ambiguity in Section 2.2. Let us return to the e... |

30 | Towards tutorial dialog to support self-explanation: Adding natural language understanding to a cognitive tutor
- Aleven, Popescu, et al.
- 2001
(Show Context)
Citation Context ...oaches more or less suitable for the tutorial dialog setting. For instance, scripting is employed in AutoTutor [23] and a knowledge-based approach similar to ours is implemented in the Geometry Tutor =-=[1,2]-=-. Outside the tutorial domain, the framework of Information State Update (ISU) has been developed in the EU projects TRINDI (http://www.ling.gu.se/research/projects/trindi/) and SIRIDUS (http://www.li... |

23 |
and the TRG. Using latent semantic analysis to evaluate the contributions of students in autotutor. Interactive Learning Environments
- Graesser, Wiemer-Hastings, et al.
(Show Context)
Citation Context ...potting difficult because of the variety of possible verbalizations. Statistical methods are often employed in tutorial systems to compare student responses with pre-constructed gold-standard answers =-=[15]-=-. In our context, such a static modeling solution is impossible because of the wide quantitative and qualitative range of acceptable proofs, i.e., generally, our set of gold-standard answers is infini... |

20 | Abductive theorem proving for analyzing student explanations to guide feedback in intelligent tutoring systems
- Makatchev, Jordan, et al.
- 2004
(Show Context)
Citation Context ...uantitative and qualitative range of acceptable proofs, i.e., generally, our set of gold-standard answers is infinite. In this respect our approach is closely related to the Why2Atlas tutoring system =-=[22]-=-. This system presents students with qualitative physicssNatural Language Dialog with a Tutor System for Mathematical Proofs 11 questions and encourages them to explain their answers with natural lang... |

20 | Understanding Informal Mathematical Discourse
- Zinn
- 2004
(Show Context)
Citation Context ...stant systems, such as Ωmega [28], are capable of interactive proof search and correctness verification. Soundness is a fairly tractable criterion for which different techniques are readily available =-=[34]-=-. However, proof step evaluation with respect to granularity and relevance is a novel and interesting application challenge for theorem proving systems. C. Handling underspecification. A typical chara... |

18 | Hierarchical Contextual Reasoning
- AUTEXIER
- 2003
(Show Context)
Citation Context ...plemented proof manager demonstrator helps to resolve underspecification and to evaluate proof steps based on heuristically guided abstract-level domain reasoning realized of the Ωmega-CoRe framework =-=[3]-=-. The PM has been integrated also into the overall Dialog system to communicate with the other components of the system. The evaluation of the system so far concentrates mainly on individual analysis ... |

16 | Pilot-testing a tutorial dialogue system that supports self-explanation
- Aleven, Popescu, et al.
- 2002
(Show Context)
Citation Context ...oaches more or less suitable for the tutorial dialog setting. For instance, scripting is employed in AutoTutor [23] and a knowledge-based approach similar to ours is implemented in the Geometry Tutor =-=[1,2]-=-. Outside the tutorial domain, the framework of Information State Update (ISU) has been developed in the EU projects TRINDI (http://www.ling.gu.se/research/projects/trindi/) and SIRIDUS (http://www.li... |

15 | Assertion-level proof representation with under-specification
- Autexier, Benzmüller, et al.
(Show Context)
Citation Context ...ertion is interpreted as the Fact and the Rule as Justification in a proof-step. Applying re-writing transformations, we obtain the following underspecified representation used by the domain reasoner =-=[4]-=-: “Fact K((A ∪ B) ∩ (C ∪ D)) = (K(A ∪ B) ∪ K(C ∪ D)) By DeMorgan-1 From .”. The baseline processing described so far covers simple cases of the mixed language: it suffices to interpret utterances wher... |

14 | 2006b. A corpus of tutorial dialogs on theorem proving; the influence of the presentation of the study-material
- Benzmüller, Horacek, et al.
(Show Context)
Citation Context ...ucted to enter proof steps, rather than complete proofs, to encourage a dialogue with the system. More details on the setup of the first experiment can be found in [5] and the second in [8]. [32] and =-=[9,33]-=- present the first and the second corpus respectively. Fig. 1 shows an excerpt from a typical session. 2 The experience gained in the Wizard-of-Oz experiments and the analysis of the collected data le... |

13 | Processing language input in the CIRCSIM-Tutor intelligent tutoring system
- Glass
- 2000
(Show Context)
Citation Context ...nt step. 4 Related Work Input analysis in dialog systems is commonly done with shallow syntactic analysis combined with keyword spotting where short answers may be elicited by asking closed-questions =-=[14]-=-. Slot-filling templates, however, are not suitable representations of the content in our domain. Moreover, the interleaving of natural and symbolic language makes keyword spotting difficult because o... |

12 | 2003b. A Wizard-ofOz experiment for tutorial dialogues in mathematics
- Benzmüller, Fiedler, et al.
(Show Context)
Citation Context ...l interface. The students were instructed to enter proof steps, rather than complete proofs, to encourage a dialogue with the system. More details on the setup of the first experiment can be found in =-=[5]-=- and the second in [8]. [32] and [9,33] present the first and the second corpus respectively. Fig. 1 shows an excerpt from a typical session. 2 The experience gained in the Wizard-of-Oz experiments an... |

12 | Abstract saturation-based inference
- Dershowitz, Kirchner
- 2003
(Show Context)
Citation Context ...cally good’ proofs and proof steps and pedagogically ‘less acceptable’ proofs and proof steps in the space of all proofs for a given problem. A notion of a ‘good proof’ is, for instance, presented in =-=[11]-=-. Soundness is, however, only one of the criteria along which a proof step should be evaluated in a tutorial context. For instance, a proof step may be formally relevant in purely logical terms, but c... |

10 |
An annotated corpus of tutorial dialogs on mathematical theorem proving
- Wolska, Vo, et al.
- 2004
(Show Context)
Citation Context ...ere instructed to enter proof steps, rather than complete proofs, to encourage a dialogue with the system. More details on the setup of the first experiment can be found in [5] and the second in [8]. =-=[32]-=- and [9,33] present the first and the second corpus respectively. Fig. 1 shows an excerpt from a typical session. 2 The experience gained in the Wizard-of-Oz experiments and the analysis of the collec... |

9 | 2004. Analysis of Mixed Natural and Symbolic Language Input in Mathematical Dialogs
- Wolska, Kruijff-Korbayová
(Show Context)
Citation Context ...red to analyze simple cases of mixed language. Then we discuss extensions for some of the more complex phenomena. For a more detailed discussion of language phenomena and interpretation procedure see =-=[30,31,18,19]-=- 2.1 Baseline Processing A simple utterance consisting of a mixture of mathematical expressions and natural language is the utterance S presented in Section 1: “by the deMorgan rule K((A ∪ B) ∩ (C ∪ D... |

9 |
The be&e tutorial learning environment (beetle
- Zinn, Moore, et al.
- 2003
(Show Context)
Citation Context ...://www.ling.gu.se/projekt/siridus/) [29], and applied in various projects targeting flexible dialog. An ISU-based approach with several layers of planning is used in the tutorial dialog system BEETLE =-=[35]-=-. 5 Conclusion We presented our approach to interpreting informal mathematical discourse in the context of tutorial dialogue and to evaluating the proof steps proposed by the student by a back-end dom... |

7 | Q (2005) Mathematical domain reasoning tasks in natural language tutorial dialog on proofs
- Benzmüller, Vo
(Show Context)
Citation Context ... clearly needed. In the following sections, we discuss some of the issues related to evaluating granularity and relevance. We illustrate the challenges using a constructed example in Fig. 4. See also =-=[7]-=-. 3.1 Granularity Granularity judgment refers to analysis of the ‘complexity’ or ‘size’ of proofs instead of the mere existence of proofs. For instance, evaluation of (a) boils down to judging the com... |

7 |
Interpreting Semi-Formal Utterances in Dialogs about Mathematical Proofs
- Horacek, Wolska
- 2004
(Show Context)
Citation Context ...red to analyze simple cases of mixed language. Then we discuss extensions for some of the more complex phenomena. For a more detailed discussion of language phenomena and interpretation procedure see =-=[30,31,18,19]-=- 2.1 Baseline Processing A simple utterance consisting of a mixture of mathematical expressions and natural language is the utterance S presented in Section 1: “by the deMorgan rule K((A ∪ B) ∩ (C ∪ D... |

6 | Mechanizing Proof Step Evaluation for Mathematics Tutoring - the Case of Granularity - Schiller - 2005 |

5 | Discourse phenomena in tutorial dialogs on mathematical proofs
- Benzmüller, Fiedler, et al.
- 2003
(Show Context)
Citation Context ...level, the discourse-level as well as at the level of domain interpretation. The language phenomena are by themselves not new, but the genre of an informal mathematical dialog adds new twists to them =-=[6,18]-=-. The mixed language and the imprecision call for deep syntactic and semantic analysis to ensure a correct mapping of the surface input to the formal representation of the proof step. B. Evaluation of... |

3 | DiaWOz-II - a tool for wizard-of-oz experiments in mathematics
- Benzmüller, Horacek, et al.
- 2006
(Show Context)
Citation Context ...nts were instructed to enter proof steps, rather than complete proofs, to encourage a dialogue with the system. More details on the setup of the first experiment can be found in [5] and the second in =-=[8]-=-. [32] and [9,33] present the first and the second corpus respectively. Fig. 1 shows an excerpt from a typical session. 2 The experience gained in the Wizard-of-Oz experiments and the analysis of the ... |

3 |
System description: A dialog manager supporting tutorial natural language dialogue on proofs
- Buckley, Benzmüller
- 2005
(Show Context)
Citation Context ... domain reasoner: the automated theorem prover Ωmega.For the other modules we provided baseline functionality required to carry out the dialogs. More details on the Dialog demo system can be found in =-=[10]-=-. The remainder of this paper is organized as follows: In Sections 2 and 3 we describe our approach to mixed language interpretation and proof step evaluation, respectively. In Section 4, we overview ... |

3 |
Introducing Topological Field Information into CCG
- Gerstenberger, Wolska
- 2005
(Show Context)
Citation Context ...esponding to the tectogrammatical level in [26]. To obtain the LM, we use the OpenCCG parser (openccg.sourceforge.net) for which we develop a lexically-based Combinatory Categorial Grammar for German =-=[13]-=-. Our motivation for using this grammar framework is two-fold: first, it is well known for its account of coordination phenomena, widely present in mathematical discourse. Second, mathematical express... |

3 |
2005. Interpretation of mixed language input in a mathematics tutoring system
- Horacek, Wolska
(Show Context)
Citation Context ...level, the discourse-level as well as at the level of domain interpretation. The language phenomena are by themselves not new, but the genre of an informal mathematical dialog adds new twists to them =-=[6,18]-=-. The mixed language and the imprecision call for deep syntactic and semantic analysis to ensure a correct mapping of the surface input to the formal representation of the proof step. B. Evaluation of... |

2 |
Proof presentation
- Siekmann
- 2006
(Show Context)
Citation Context ...oof alternatives that have to considered in the analysis tasks. To analyze human-oriented mathematical proofs, shaped-up textbook proofs have been investigated in the deduction systems community (see =-=[34,27]-=-). Claus Zinn [34], for instance, addresses complete, carefully structured textbook proofs, and relies on given text-structure, typesetting and additional information that identifies mathematical symb... |

2 |
Applied Logic series (28
- Siekmann, Benzmüller, et al.
- 2003
(Show Context)
Citation Context ...‘size’ of the proof step logically and pedagogically acceptable? Relevance: Is the proof step logically and pedagogically useful given the goal? Existing Mathematical Assistant systems, such as Ωmega =-=[28]-=-, are capable of interactive proof search and correctness verification. Soundness is a fairly tractable criterion for which different techniques are readily available [34]. However, proof step evaluat... |

2 | á. 2004b. LexicalSemantic Interpretation of Language Input in Mathematical Dialogs
- Wolska, Kruijff-Korbayov
(Show Context)
Citation Context ...red to analyze simple cases of mixed language. Then we discuss extensions for some of the more complex phenomena. For a more detailed discussion of language phenomena and interpretation procedure see =-=[30,31,18,19]-=- 2.1 Baseline Processing A simple utterance consisting of a mixture of mathematical expressions and natural language is the utterance S presented in Section 1: “by the deMorgan rule K((A ∪ B) ∩ (C ∪ D... |

2 |
Factors Influencing Input Styles in Tutoring Systems: The Case of the Study-Material Presentation Format
- Wolska, Kruijff-Korbayovà
- 2006
(Show Context)
Citation Context ...ucted to enter proof steps, rather than complete proofs, to encourage a dialogue with the system. More details on the setup of the first experiment can be found in [5] and the second in [8]. [32] and =-=[9,33]-=- present the first and the second corpus respectively. Fig. 1 shows an excerpt from a typical session. 2 The experience gained in the Wizard-of-Oz experiments and the analysis of the collected data le... |

1 |
Representation of mathematical objects for inferencing and for presentation purposes
- Horacek, Fiedler, et al.
- 2004
(Show Context)
Citation Context ...rule K((A ∪ B) ∩(C ∪ D)) = (K(A ∪ B) ∪ K(C ∪ D)) holds.”; DM stands for DeMorgan representation that mediates between the discrepant views of linguistic analysis and deduction systems’ representation =-=[17]-=-; it thus has a potential of providing a direct link to logical definitions in a mathematical knowledge base, suchas MBase [21]. The motivation for using an intermediate representation instead of dire... |

1 | Benzmüller et al - unknown authors |

1 |
and the Tutoring Research Group. Dialog move generation and conversation management in AutoTutor
- Person, Graesser, et al.
- 2000
(Show Context)
Citation Context ...finement procedures. Recent research into dialog modeling has delivered a variety of approaches more or less suitable for the tutorial dialog setting. For instance, scripting is employed in AutoTutor =-=[23]-=- and a knowledge-based approach similar to ours is implemented in the Geometry Tutor [1,2]. Outside the tutorial domain, the framework of Information State Update (ISU) has been developed in the EU pr... |