Lower bounds on learning decision lists and trees

Abstract

k-decision lists and decision trees play important roles in learning theory as well as in practical learning systems, k-decision lists generalize classes such as monomials, k-DNF, and k-CNF and like these subclasses is polynomially PAC-learnable [19]. This leaves open the question of whether k-decision lists can be learned as efficiently as k-DNF. We answer this question negatively in a certain sense, thus disproving a claim in a popular textbook [2]. Decision trees, on the other hand, are not even known to be polynomially PAC-learnable, despite their widespread practical application. We will show that decision trees are not likely to be efficiently PAC-learnable. We summarize our specific results. The following problems cannot be approximated in polynomial time within a factor of 2log δ n for any δ < 1, unless NP ⊂ DTIME[2polylog n]: a generalized set cover, k-decision lists, k-decision lists by monotone decision lists, and decision trees. Decision lists cannot be approximated in polynomial time within a factor of nδ, for some constant δ > 0, unless NP = P. Also, k-decision lists with 1 0-1 alternations cannot be approximated within a factor log1 n unless NP ⊂ DTIME[no(log log n)] (providing an interesting comparison to the upper bound recently obtained in [1]).