On-line learning of binary and n-ary relations over multi-dimensional clusters

Abstract

We consider the on-line learning problem for binary relations defined over two finite sets, each clustered into a relatively small number κ, 1 of 'types' (such a relation is termed a (κ, l)-binary relation), extending the models of [2] [3]. We investigate the learning complexity of (κ, l)-binary relations with respect to both the 'self-directed' and 'adversary-directed' learning models. We also generalize this problem to the learning of (κ1,.., κd)-d-ary relations. In the self-directed model, we exhibit an efficient learning algorithm which makes at most κl + (n - κ) log κ + (m - l) log l mistakes, where n and m are the number of rows and columns, roughly twice the lower bound we show for this problem, 1/4 [log κ][log l] + 1/2(n - κ) [log κ] + 1/2 (m - l) [log 1]. In the adversary-directed model, we exhibit an efficient algorithm for the (2, 2)-binary relations, which makes at most n + m + 2 mistakes, only 2 more than the lower bound we show for this problem, n + m. We also obtain two general lower bounds for learning (κ1,.., κd)-d-ary relations. Finally we show that, although the sample consistency problem for (2, 2)-binary relations is solvable in polynomial time, the same problem for (2,2,2)-ternary relations is already NP-complete.