Statistical Learning for Relational and Structured Data

Abstract

Learning with relational and structured data has gained a growing interest within the machine learning community in recent years. The great devel- opment of this research area has been mainly due to the large amount of available data, organized in complex relational structures, coming from a vari- ety of fields, like molecular biology, social networks analysis, natural language parsing, and many others. Traditional machine learning algorithms, which typically consider the examples as independent, had to be extended and gen- eralized in order to handle relational and structured data. For this reason, Statistical Relational Learning (SRL) community was born in recent years with the aim of applying statistical learning methodologies to the relational setting. SRL algorithms usually combine a first-order logic representation of the domain of interest, with the probabilistic framework of graphical mod- els. Many different methods have been developed following this idea, often providing very similar formalisms and solving similar problems. This thesis is organized in three distinct parts, each addressing a different aspect of statistical relational learning. In the first part of this dissertation, a semi-generative approach to parameter learning for stochastic grammars is pro- posed, showing applications to natural language parsing and RNA secondary structure prediction. In the second part, Markov logic networks (MLNs) are applied to bioinformatics tasks, and an extension of standard MLNs is pre- sented, in order to handle continuous features and to embed discriminative classifiers within that framework. Finally, the third part of this thesis consid- ers a classical problem of Inductive Logic Programming (ILP), which consists in learning logical representation of objects directly from data: a new scoring function, called relational information gain, is used to capture the potential informativeness of literals.