AIGC 2D/3D technology development and creative industry applications

Abstract

The growth of data and model size has led to the emergence of new generative models，such as large language models and diffusion models，which produce increasingly high-quality and diverse results. These large generative models are driving the rapid development of artificial intelligence（AI）-generated content（AIGC）. This article focuses on the core needs of the creative industry and reviews the technological and industrial developments in the AIGC 2D/3D field from 2023 to 2024. First，this article summarizes the development background of generative technologies and their market value. Sec-ond，given the technical development in the AIGC 2D/3D field，the pace of technological evolution is evidently fast. The focus has transitioned from primarily being on“generative adversarial networks”to focusing on“denoising diffusion models and Transformer structures”. The new structure has stronger expressive power，greater diversity，and more flexible control capabilities. In the AIGC 2D review section，the article categorizes image and video generation technologies into three types：“high-quality generation foundation”，“controllable generation technology”，and“editable generation technology”. AIGC 2D technology began around 2014 with models such as GANs and VAEs，which made breakthroughs in image genera-tion. Researchers have focused on generating high-quality 2D images，improving resolution，and enhancing details. By 2018，the focus shifted to better image quality，with StyleGAN advancing realism and diversity，thereby driving AIGC’s application in image processing and artistic creation. In 2020，diffusion models revolutionized 2D image generation，improving image quality and offering better control，such as text-to-image generation. By 2022，AIGC 2D technology evolved and expanded into advertising，content generation，and entertainment，with multimodal models such as CLIP and DALL-E enabling image creation from text descriptions. These innovations lowered the barrier for creators and improved generation quality. By 2024，AIGC 2D technology advanced to video generation，further expanding its applications. In the AIGC 3D review section，the article categorizes technologies based on“input data types”，“output data types”，and“gen-eration methods”. AIGC 3D development began in 2022，shifting from combining 2D generative models with implicit 3D representations to 3D diffusion models and mixed/explicit 3D data. Early research focused on traditional 3D models，such as voxels，meshes，and point clouds. With the success of generative adversarial network（GAN）and variational autoen-coder（VAE）in 2D generation，these techniques were adapted to 3D. In 2023，breakthroughs in denoising diffusion mod-els led to their application in 3D generation，showing potential for creating geometric structures and textures. However，the limitations of 2D diffusion models raised concerns about their ability to handle 3D data. By 2024，generative 3D technology matured，integrating with real-world applications and evolving from a 2D-based approach to a 3D-driven one. The use of multimodal models and natural language processing allowed users to generate complex 3D data from simple text or 2D images. This marked a shift from research to practical applications，with technological advancements and market applica-tions progressing in tandem. Next，the article summarizes the current technical challenges and industry application issues faced by both types of technologies. The main emphasis for the future development of AIGC 2D/3D technologies will be how to provide new technologies that better meet industry creation standards and needs. For AIGC 2D，precision in controlling the structure，details，and style of generated images or videos to meet user expectations remains a bottleneck. This is par-ticularly challenging in video generation，where camera movements，object motion，and scene composition are harder to control. Moreover，the lack of flexibility for fine-tuning and editing generated content remains an issue，and achieving a high match between generated results and multimodal inputs continues to be a challenge. Furthermore，content quality such as realism，accuracy，and aesthetic appeal needs significant improvement，especially in areas like hand and eye details and object motion continuity. AIGC 3D faces three key challenges：the gap between generated results and industry standards，differences in application requirements across industries，and the need to balance the divergent needs of indus-try experts and the general public. Industry-specific standards for 3D data，such as geometry features and texture resolu-tion，demand high-quality outputs from AIGC 3D. Different industries require different technical adaptations of AIGC 3D technology. Furthermore，the differing needs of professional creators，who seek high precision，and general users，who pri-oritize ease of use and affordability，add complexity to AIGC 3D’s real-world applications. Finally，the article provides a summary of the past 20 years，showing how the creative industry has experienced a“spiral upward development”driven by technological progress. It also offers some thoughts and perspectives on the future trends of technological development.