Hierarchical abstraction drives human-like 3-D shape processing in deep learning models

Abstract

Both humans and deep learning models can recognize objects from 3D shapes depicted with sparse visual information, such as a set of points randomly sampled from the surfaces of 3D objects (termed a point cloud). Although deep learning models achieve human-like performance in recognizing objects from 3D shapes, it remains unclear whether these models develop 3D shape representations similar to those used by human vision for object recognition. Evidence suggests that training with approximately 10,000 object instances enables models to acquire representations of local geometric structures in 3D shapes. We hypothesize, however, that their representations of 3D global shapes are still limited. To test this hypothesis, we conducted three human experiments systematically manipulating point density and object orientation (Experiment 1), local geometric structure (Experiment 2), and part configuration (Experiment 3). Human performance was stable across conditions in the first two experiments, but declined significantly in the part-scrambled condition of the final experiment. We compared human performance with two types of deep learning architectures: convolution-based models (e.g., DGCNN) and transformer-based models (e.g., Point Transformer). The transformer-based models more closely captured human performance patterns across experimental conditions. Ablation simulations revealed that this advantage is largely driven by progressive downsampling operations that enable hierarchical abstraction of 3D shapes.