Are Large Language Models Smart Coders? Evaluating Correctness and Efficiency of LLM-generated Algorithms

Abstract

Skilled software developers play a crucial role in creating energy-efficient code, contributing to the reduction of carbon emissions and promoting sustainable computing practices. In parallel, Large Language Models (LLMs) have recently emerged as powerful programming assistants, providing support for developers in complex coding tasks. In this context, this study investigates the ability of LLMs to generate computationally efficient implementations of classic algorithms-such as Heap Sort and Binary Search-across different programming languages, with a primary focus on optimizing energy consumption, memory usage, and execution time. Our results show that, in most cases, different LLMs are able to generate these algorithm implementations correctly. Furthermore, in the majority of the evaluated scenarios, LLM-generated code demonstrates superior performance over human-written algorithm code with regard to energy consumption, memory efficiency, and execution time.