Leveraging LLMs for Smart Cities Qualitative Data Analysis

Abstract

Public authorities frequently conduct surveys and analyze data from citizens, a process that is often labor-intensive when performed manually. This paper explores how generative artificial intelligence (GenAI) can assist in automating data analysis for public authorities. In this respect, we investigate the potential of large language models (LLMs) to perform sentiment analysis and summarization of unstructured data as smart services. Using data from the East Bristol livable neighborhood (EBLN) as a case study, we assess the accuracy and precision of these models and validate the results against ground truth data and expert evaluations. Our findings indicate that sentiment classification achieved over 90% accuracy. Additionally, incorporating retrieval-augmented generation (RAG) further enhances the results obtained. Comparative analysis revealed that LLMs achieved superior performance over traditional summarization techniques based on domain expert evaluation. These results suggest that LLMs offer a promising and impactful approach to improving the efficiency of qualitative analysis, though further research is required to enhance their accuracy and usefulness.