Studying Databases of Intentions: Do Search Query Logs
Capture Knowledge about Common Human Goals?
Markus Strohmaier
Graz University of Technology and Know-Center
Graz, Austria
markus.strohmaier@tugraz.at
Mark Kröll
Graz University of Technology
Graz, Austria
mkroell@tugraz.at

ABSTRACT
Access to knowledge about common human goals has been
found critical for realizing the vision of intelligent agents
acting upon user intent on the web. Yet, the acquisition of
knowledge about common human goals represents a major
challenge. In a departure from existing approaches, this
paper investigates a novel resource for knowledge acquisi-
tion: The utilization of search query logs for this task. By
relating goals contained in search query logs with goals
contained in existing commonsense knowledge bases such
as ConceptNet, we aim to shed light on the usefulness of
search query logs for capturing knowledge about common
human goals. The main contribution of this paper consists
of insights generated from an empirical study comparing
common human goals contained in two large search query
logs (AOL and Microsoft Research) with goals contained
in the commonsense knowledge base ConceptNet. The
paper sketches ways how goals from search query logs
could be used to address the goal acquisition and goal cov-
erage problem related to commonsense knowledge bases.
Categories and Subject Descriptors
H.3.1 [Information Storage and Retrieval]: Content Analysis
and Indexing; I.2.6 [Artificial Intelligence]: Learning; I.2.7 [Ar-
tificial Intelligence]: Natural Language Processing
General Terms
Algorithms, Experimentation
Keywords
Knowledge acquisition, human goals, commonsense knowledge
INTRODUCTION
To realize the vision of common-sense enabled and goal-
oriented agents on the web, agents must have programmat-
ic access to the set and variety of common human goals, in
order to reason about them and to provide services that
help satisfy users’ needs [14][23]. In Berner-Lee’s vision,
an agent aiming to, for example, “plan a trip to Vienna”
would need to have some means to understand that “plan a
trip” is likely to involve a set of other goals or services,
such as “contact a travel agency” and “book a hotel”. This
type of knowledge has been characterized as commonsense
knowledge, i.e. knowledge that humans are generally as-
sumed to possess, but which is extremely difficult for com-
puters to acquire [16]. Having such knowledge available is
a prerequisite for applications such as GOOSE – a com-
monsense-enabled search engine [15] - or EventMinder – a
commonsense-enabled calendar application [25], which are
inspiring prototypes for the potential of traditional applica-
tions augmented with commonsense knowledge.
Current research projects aiming to capture and organize
commonsense knowledge include CyC [11] or Openmind /
ConceptNet [24]. These projects utilize human knowledge
engineering [11], volunteer-based [16], game-based [14] or
semi-automatic approaches [5] for knowledge acquisition,
where common human goals can be considered a subset of
the enormous breadth of commonsense knowledge. How-
ever, existing attempts to capture knowledge about com-
mon human goals generally suffer from a number of prob-
lems, including: 1) the goal acquisition problem (or bottle-
neck), which refers to the costs associated with knowledge
acquisition [14] and 2) the goal coverage problem, which
refers to the difficulty of capturing the tremendous variety
and range in the set of common human goals [5]. These
problems have hindered progress in capturing broad know-
ledge about common human goals, and have hindered the
development of intelligent agents and applications on the
web. In this paper, we are seeking to explore the utility of
search queries, i.e. user needs expressed in small textual
fragments, to help address the above challenges.
Using textual contributions of volunteers on the web for
knowledge acquisition purposes is not a new idea (cf.
MIT’s Openmind initiative [16]). However, in contrast to
volunteer-based systems, Search Query Logs provide an
abundant and seemingly endless stream of information
about human needs. This has led query logs being referred
to as “Databases of Intentions” in the past. Databases of
intentions
1
refer to the observation that Search Query Logs
provide unique, up-to-date and detailed insights into human
motivations, needs and goals (such as “buying a house”) that
can be analyzed, studied and used for different purposes.
While search query logs have been utilized successfully for
knowledge acquisition in a range of different contexts [18],
they have not been used to capture explicit knowledge
about human goals, partly because query logs pose a num-

1
http://battellemedia.com/archives/000063.php, last accessed on
April 15, 2009

ber of challenges: The majority of search queries is usually
short [18] and ambiguous [1], they convey user goals at
different degrees of intentional explicitness [27], and they
are often consisting of arbitrary concatenations of terms
that frequently contain misspellings. Yet, recent research
revealed that a number of search queries actually contain
explicit statements of human goals [27], and that the space
of queries in search query logs is vast and topically diverse
[19]. This would make query logs a seemingly attractive
resource for acquiring knowledge about a diverse range of
common human goals. But how useful are Search Query
Logs for capturing common human goals? In this paper,
we are investigating the following guiding research ques-
tions related to this task:
RQ1: Do Search Query Logs contain knowledge about
common human goals?
RQ2: If they do, what is the nature of common human
goals shared by ConceptNet and two large Query Logs?
RQ3: Do goals contained in ConceptNet and Search Query
Logs differ w.r.t. scope, and if so - how?
RQ4: Can goals contained in Search Query Logs be used
to refine ConceptNet?
RQ5: Can goals contained in Search Query Logs be used
to add novel nodes and relations to ConceptNet?
Finding answers to these questions would help assess the
usefulness of search query logs to lower the costs often
associated with commonsense knowledge capture, and
would help to assess the potential to improve the coverage
of existing knowledge bases such as ConceptNet.
Contributions
In previous work, we have developed an automatic classifi-
cation approach focused on identifying search queries that
contain explicit statements of human goals [26]. In this
paper, we apply our approach to two very large query logs
provided by AOL and Microsoft Research, yielding a set of
~115.000 queries that contain common human goals in 77
out of 100 cases (77% precision).
The paper makes the following contributions: Using the set
of goals acquired from search query logs, we conduct com-
parative analyses to assess the nature of these goals in rela-
tion to an exemplary commonsense knowledge base, in our
case ConceptNet. Subsequently, we discuss preliminary
proposals on how to use common human goals acquired
from query logs to refine and expand existing knowledge
bases. Our findings suggest that goals acquired from search
query logs have the potential to expand coverage of com-
mon human goals in existing knowledge bases while main-
taining reasonable precision scores. To the best of our
knowledge, this work represents the first comparative
study of the utility of search query logs for capturing com-
mon human goals. Our findings suggest that search query
logs indeed represent a viable, yet largely untapped alterna-
tive for acquiring knowledge about common human goals.
AUTOMATIC GOAL ACQUISITION FROM
SEARCH QUERY LOGS
In this paper, we use a machine learning approach pre-
viously developed by the authors of this paper [26] for the
purpose of automatically classifying queries regarding
whether they contain an explicit goal statement or not. This
approach can be used to acquire a set of queries containing
goals from search query logs. Table 1 gives some examples
of actual queries containing/not containing statements of
human goals (obtained from [19]).
Table 1 Examples of queries obtained from [19]
Queries containing
explicit goal statements
Queries not containing
explicit goal statements
“sell my car” “Mazda dealership”
“play online poker” “online games”
“find home to rent in Florida” “Miami beach houses”
“passing a drug test” “drug test”
“raising your credit score” “credit cards”
Based on work that highlights the crucial role of verbs in
explicit statements of goals ([13],[22]), we define queries
containing human goals in the following way:
A search query is regarded to contain an explicit user goal
whenever the query 1) contains at least one verb and 2) de-
scribes a plausible state of affairs that the user may want to
achieve or avoid (cf. [22]) in 3) a recognizable way [26].
“Recognizable” refers to what [9] defines as “trivial to
identify” by a subject within a given attention span. “Plaus-
ible” refers to an external observer’s assessment whether
the goal contained in a query could likely represent the
goal of a user who formulates the given query. This defini-
tion has been shown to produce reasonable inter-rater
agreements among independent subjects [26]. It is impor-
tant to note that it would be rather difficult to assess queries
solely based on data from an anonymous query log due to
the inherent goal verification problem of such a task [26].
However, the objectives of this work are more modest: In
this paper, we are interested in acquiring plausible common
human goals for knowledge capture purposes. An advan-
tage of acquiring broad knowledge about plausible goals is
that it can put constraints on the space of all goals, which
plays a role in, for example, goal recognition [6] or query
disambiguation [1].
“Queries containing explicit goals” according to our defini-
tion thus can be related to what other researchers have cha-
racterized as “better queries”, or queries that have “more
precise goals” (R. Baeza-Yates at the “Future of Web
Search” Workshop 2006, Barcelona). A query does not
contain an explicit goal when it is difficult or extremely
hard to elicit some specific goal from the query. Examples
include blank queries, or queries such as “car” or “travel”,
which embody goals on a very general, ambiguous and
mostly implicit level.