Sneaky spatial segmentation: Reading architectural drawings with deep neural networks and without labeling data

Abstract

Currently, it is nearly impossible for an artificial neural network to generalize a task from very few examples. Humans, however, excel at this. For instance, it is not necessary for a designer to see thousands or millions of unique examples of how to place a given drawing symbol in a way that meets the economic, aesthetic, and performative goals of the project. In fact, the goals can be (and usually are) communicated abstractly in natural language. Machine learning (ML) models, however, do need numerous examples. The methods that we explore here are an attempt to circumvent this in order to make ML models more immediately useful. In this work, we present progress on the application of contemporary ML techniques to the design process in the architecture, engineering, and construction (AEC) industry. We introduce a technique to partially circumvent the data hungriness of neural networks, which is a significant impediment to their application outside of the ML research community. We also show results on the applicability of this technique to real-world drawings and present research that addresses how some fundamental attributes of drawings as images affect the way they are interpreted in deep neural networks. Our primary contribution is a technique to train a neural network to segment real-world architectural drawings after using only generated pseudodrawings.