Comparative evaluation of large language models in assessing off-label drug use and generating patient consent forms: A Cross-sectional study

Abstract

Background: Off-label drug use (OLDU) involves the prescription of medications beyond their approved indications, populations, or dosages, and is a common practice in areas such as pediatrics, geriatrics, psychiatry, and oncology. Ethical OLDU necessitates sound clinical justification and transparent, patient-centered consent. With the rise of large language models (LLMs) in healthcare, their utility in supporting OLDU through evidence synthesis and consent documentation remains underexplored. Methods: A cross-sectional study was conducted evaluating the performance of two LLMs, ChatGPT-4.0 and DeepSeek-V3, across seven OLDU scenarios encompassing vulnerable populations and emerging indications. Each LLM was prompted to assess OLDU using the BRAvO decision-making framework based on the PrOACT-URL structure and to generate corresponding patient consent forms. Fifteen standardized prompts were used, and responses were evaluated using rubric-based scoring systems for clinical reasoning (maximum score = 27) and consent form quality (maximum score = 33 when reproductive concern was applicable and 30 for others). Results: Both LLMs provided structured and complete responses across all scenarios. ChatGPT demonstrated superior readability, empathetic tone, and patient-focused consent forms, consistently scoring in the excellent range. DeepSeek provided higher clinical detail and included extensive guideline references, with strong medico-legal structuring in its documentation. Scenario-specific variations were noted in how each LLM addressed uncertainties, risk mitigation, and reproductive considerations. Overall, both models scored within the excellent range for OLDU assessment and consent generation. Conclusion: LLMs such as ChatGPT and DeepSeek can effectively support OLDU decision-making and informed consent processes. Their integration into clinical workflows offers promise, though expert oversight remains essential to ensure accuracy, ethical compliance, and patient-centered care.