Interactive Digital Twins: Integrating XR and Motion Capture for Timber Assembly

Abstract

As digital technologies become increasingly embedded in design and construction processes, creating responsive, data-rich environments has great potential to advance architectural, engineering, and fabrication workflows. This paper presents a novel methodology that integrates motion capture-based object tracking with extended reality interfaces to create responsive digital twins in timber assembly workflows. We developed a software infrastructure for streaming, receiving, visualizing, and storing real-time spatial data, along with specialized hardware that enables the rapid attachment and removal of markers on timber construction elements. By establishing a direct feedback loop between the physical construction site and its virtual counterpart, our system allows real-time adjustments to element positioning, orientation, and sequencing. Applied to a complex timber project, our integrated approach reduced assembly time and errors while also paving the way for more adjustable and flexible workflows. The article details the system's implementation, case study insights, and strategies for scaling adaptive, data-driven architecture, engineering, and construction workflows.