AI-BEWITCHED ARCHITECTURE OF HANSEL AND GRETEL: Food-to-Architecture in 2D & 3D with GANs and Diffusion Models

Abstract

Architects such as Le Corbusier, Frank Gehry, Aldo Rossi, and Greg Lynn have implicitly turned culinary formalism into architectural formalism during their careers. How might AI assist in a similar act of bisociation (or conceptual blending)? The paper is the first to explore this food2architecture bisociation explicitly, and specifically with generative adversarial networks (GANs) such as CycleGAN and VQGAN-CLIP, and diffusion models such as OpenAI’s DALL-E 2, Midjourney and DreamFusion (using Stable Diffusion). Instead of using textual input prompts to generate images of architecture only with the discipline’s own vocabulary, the research merges them with the vocabulary of food, thus exploiting their potential in blending their respective conceptual and formal characteristics. While these diffusion models have recently been used by the general public to generate 2D imagery posts on various social media platforms, no existing work has conducted a detailed and systematic analysis on their exclusive capacity in bisociating food and architecture. Imagery outputs generated during two workshops involving 150 designers and non-designers are included here as illustrations. Beginning and ending the paper with the allfamiliar fairy tale of the gingerbread house, the research explores the creative design bisociative affordance of today's text-to-image and textto-3D models by turning culinary inputs into architectural outputs -- envisioning an explicitly computational version of the implicit 'food2architecture' mental models plausibly used by some of the most creative architects.