Image search enhanced by using external data sources and reasoning

Abstract

In the paper, the concept of a system is presented, that makes use of the reasoning to perform digital image search using the information present in an annotated images combined with external text data sources in order to answer complex queries. The workflow in based the Prolog reasoning that allows for both modeling ontologies and combining the knowledge from both data sources. Prior to reasoning, both data sources are transformed into the form of Prolog predicates. An example application of the proposed concept is also presented in the paper. It deals with photographs database with annotated faces of politicians combined with the external knowledge stored in the Parliament-members database. It provides with a mechanism that supports formulating complex questions like: “Find a photograph where a politician belonging to party A stands between a woman and another political that was a member of Parliament of B-th term of office”.