Evaluating Hypotheses Two De nitions of Error

Abstract

The true error of hypothesis h with respect to target function f and distribution D is the probability that h will misclassify an instance drawn at random according to D. error D (h) Pr x2D f(x) 6 = h(x)] The sample error of h with respect to target function f and data sample S is the proportion of examples h misclassiies error S (h) 1 n X x2S (f(x) 6 = h(x)) Where (f(x) 6 = h(x)) is 1 if f(x) 6 = h(x), and 0 otherwise. How well does error S (h) estimate error D (h)? 75 lecture slides for textbook Machine Learning, T. Mitchell, McGraw Hill, 1997