Effort is All You Need: The Possibilities of Writing Analytics

Abstract

• Identification of Innovation: This article describes the browser-based word processor and learning management system the Forge, a faculty-led open source project hosted by LibreTexts, an open education nonprofit. A guiding principle of the Forge system is that writing analytics, particularly those focused on student time and effort when composing, offer a potential solution to the challenges to academic integrity in the generative AI era. In addition to incentivizing students to honor the writing process by engaging in effort during the drafting and revision stages, the Forge provides instructors with previously unavailable insights into how students write assignments in real contexts, both individually and for classes as a whole. • Exposition of Innovation: The Forge includes numerous different visualizations of student writing processes, including EssayMaps that present the editing history of the document; HeatMaps, which show the hours and days the student spent on the assignment; and WordChange graphs, which show the change in word count of the document every minute while the student is writing, highlighting spikes in the word count, and writing composed quickly. The system also provides advisory effort and revision scores to advise the instructor whether a deeper investigation of the assignment’s analytics may be warranted. • Application of Innovation: In its present form, the Forge can be used for a variety of research studies looking at how students write assignments in real contexts, including developing profiles of student writing processes, understanding the role that time-coded phenomena such as draft deadlines play in the composition process, understanding the differing strategies and processes students use to compose different types of assignments, assessing the effects of various pedagogical strategies and interventions, and understanding the ways students use generative AI writing tools. • Directions for Further Research: The possibilities of writing analytics for writing instruction are still largely untapped by technologists and scholars. When applied across writing programs, tools like the Forge could provide unprecedented amounts of rich data about students’ writing and writing processes for instructional, institutional, and research purposes. The development team is continuing to expand the platform’s capabilities, including potentially and eventually adding optional functionalities that use limited AI to provide students with dynamic “teacherly” feedback on their writing.