Results 1 
4 of
4
Accuracy vs. Simplicity: A Complex TradeOff∗
, 2003
"... Inductive learning aims at finding general rules that hold true in a database. Targeted learning seeks rules for the prediction of the value of a variable based on the values of others, as in the case of linear or nonparametric regression analysis. Nontargeted learning finds regularities without ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Inductive learning aims at finding general rules that hold true in a database. Targeted learning seeks rules for the prediction of the value of a variable based on the values of others, as in the case of linear or nonparametric regression analysis. Nontargeted learning finds regularities without a specific prediction goal. We model the product of nontargeted learning as rules that state that a certain phenomenon never happens, or that certain conditions necessitate another. For all types of rules, there is a tradeoff between the rule’s accuracy and its simplicity. Thus rule selection can be viewed as a choice problem, among pairs of degree of accuracy and degree of complexity. However, one cannot in general tell what is the feasible set in the accuracycomplexity space. Formally, we show that finding out whether a point belongs to this set is computationally hard. In particular, in the context of linear regression, finding a small set of variables that obtain ∗Earlier versions of this paper circulated under the title “From Cases to Rules: Induction
Earlier versions of this paper circulated under the title “From Cases to Rules: Induction
, 2002
"... Inductive learning aims at finding general rules that hold true in a database. Targeted learning seeks rules for the prediction of the value of a variable based on the values of others, as in the case of linear or nonparametric regression analysis. Nontargeted learning finds regularities without a ..."
Abstract
 Add to MetaCart
Inductive learning aims at finding general rules that hold true in a database. Targeted learning seeks rules for the prediction of the value of a variable based on the values of others, as in the case of linear or nonparametric regression analysis. Nontargeted learning finds regularities without a specific prediction goal. We model the product of nontargeted learning as rules that state that a certain phenomenon never happens, or that certain conditions necessitate another. For all types of rules, there is a tradeoff between the rule’s accuracy and its simplicity. Thus rule selection can be viewed as a choice problem, among pairs of degree of accuracy and degree of complexity. However, one cannot in general tell what is the feasible set in the accuracycomplexity space. Formally, we show that finding out whether a point belongs to this set is computationally hard. In particular, in the context of linear regression, finding a small set of variables that obtain a certain value of R 2 is computationally hard. Computational complexity may explain why a person is not always aware of rules that, if