Evaluating the Effectiveness of Personal Cognitive Augmentation: Utterance/Intent Relationships, Brittleness and Personal Cognitive Agents

Abstract

The popularity of applying intelligent agents is moving business into a new age of actionable information production. Managing the introduction and operation of such entities in an enterprise is a critical factor board rooms will face as the trend continues. Thus, it is important now to develop tools that measure their effectiveness. This paper seeks to understand the efficacy of these agents I call Personal Cognitive Agents (PCA). At their infancy, PCAs are subject to a disconnect between what the human operator intends and what the PCA understands as operator intent. A relationship exists between what is uttered by the operator and the operator’s intent. I establish a metric called utterance intent relationship (UIR) for this purpose and seek to determine UIR’s viability as a universal tool to measure an agent’s effectiveness in human/agent symbiosis.