Distributed Modularization of Thought: Lets Small Rival Large LMs

Abstract

We present the Distributed Modularization of Thought (DMoT) approach for code generation using several small language models, fine-tuned for distinct phases of the code generation process. Instead of the conventional single-model approach in direct code generation, our DMoT method decomposes the task into a hierarchy of subtasks using hierarchical Multi-Level Reasoning (MLR) graphs. Each phase, from solving simple subproblems to verifying syntax and compliance with initial requirements, is handled by a dedicated fine-tuned model. Models were trained through knowledge distillation from a larger model (ChatGPT-4.1) using a quantized LoRA technique, enabling execution on accessible hardware. Experimental evaluation is conducted on the BigCodeBench benchmark, which includes realistic and complex programming tasks. These results show that a system composed of several specialized small models outperforms single-model approaches in terms of accuracy (pass@1) and structural quality (CodeBLEU). So, small fine-tuned language models can rival and even outperform larger models, offering a resource-efficient path in the code generation processes.