Living Documents: Designing for User Agency over Automated Text Summarization

Abstract

Text summarization is an example of a complex algorithm with a single direct input to output user operation. As different types of users e.g. students, journalists, etc. adopt these algorithms, it is imperative that user experience with text summarization AI systems is modified to allow agency. To explore this, we designed an interactive multi-document summarization system, called Living Documents, that allows agency over the summarization process. We evaluated this system in a preliminary study with 25 students identified from Prolific. The students perform summarization tasks using the control functions in our system. Results from this study show that these functions contributed to the user's understanding of how summarization works and this understanding slightly impacts user trust in our system positively. We discuss our experience designing for user control of a normally automated, complex process that produces immutable output and the future implications for other applications.