Automating detection of framing agency in design team talk

Abstract

Those who teach design contend with issues of authenticity and engagement. A problem that is too narrow or open can be challenging for students, yet finding that Goldilocks middle ground is complicated by many factors. Framing agency-making decisions that are consequential to framing and solving design problems-appears to provide clarity about student engagement with different types of design problems. However, detecting framing agency in design team talk is a labor intensive process. The purpose of this study was to evaluate a variety of text mining approaches for suitability in detecting framing agency in transcribed talk. We found no correlation between human-coding and sentiment analysis. However, interestingly, polarity derived from sentiment analysis did differentiate between a team that displayed almost no framing agency and those that did, with the former showing a high level of positivity. This reflects the lack of struggle, high certainty and agreeability the team displayed as they quickly agreed they all had similar and therefore correct answers. We also trained a Regularized Support Vector Machine Classifier to predict levels of framing agency, with the human-coded data as training data. The model showed 89% accuracy in detecting high framing agency. Given the recent increase in quality of auto-transcription tools, such approaches may lead to in-situ detectors in future.