A Critical Review and Evaluation of LLMs for RTL Generation

Abstract

Large language models (LLMs) are emerging as powerful tools for hardware design, with recent work exploring their ability to generate register-transfer level (RTL) code directly from natural-language specifications. This paper presents a critical review and empirical evaluation of LLM-based RTL generation. We examine thirty-one published efforts, classifying their use of techniques such as fine-tuning, reinforcement learning, retrieval-augmented prompting, and multi-agent orchestration across eight methodological dimensions including debugging support, post-RTL metrics, and benchmark development. Building on this synthesis, we experimentally evaluate frontier commercial and open-weight models—GPT-4.1, GPT-4.1-mini, Claude Sonnet 4, and Llama 4 Maverick —on the VerilogEvalV2 and RTLLM-v2.0 benchmarks under both single-shot generation and a lightweight ReAct-style reflection loop, with compilation and simulation performed through Icarus Verilog interfaced via the Model Context Protocol (MCP). Results show that these models achieve up to 89.74% on VerilogEval and 96.08% on RTLLM, matching or exceeding prior domain-specific pipelines without specialized fine-tuning. Detailed failure analysis reveals systematic error modes, including FSM mis-sequencing, handshake drift, blocking vs. non-blocking misuse, and state-space oversimplification. Finally, we outline a forward-looking research roadmap toward natural-language-to-System-on-Chip (SoC) design, emphasizing realistic benchmarks and open flows, richer specification formalisms, AI-human collaborative design environments, and system-level feedback that spans physical design, firmware, and design space exploration. Together, this work provides a synthesis of recent advances and a baseline evaluation of frontier LLMs, highlighting opportunities and challenges in moving toward AI-native electronic design automation.