Audit-as-code: a policy-as-code framework for continuous AI assurance

Abstract

Introduction: Existing AI assurance and governance frameworks rely heavily on documented written policies and manual reviews of the implementation. The primary challenge is not the length of these documents, but to operationalize the gap from transforming qualitative requirements into verifiable controls. This approach makes ensuring continuous compliance through the development life cycle hard to enforce, scale, and reproduce. Methods: This study presents a continuous assurance framework called Audit-as-Code that maps governance requirements to technically-auditable rules, that can be a combination of versioned policy specification and executable checks for evidence artifacts, linked to structured evidence regarding data, models, provenance, performance, decisions, and explanations regarding the decisions being made. While the framework addresses the governance and regulatory mapping requirements, the primary focus of this study is MLOps/CI-CD (continuous integration/continuous delivery) operationalization, and turning these requirements into deterministic checks and gate decisions integrated in operational workflows. In this study, we introduce an assured readiness score that integrates the governance risk with other key responsible AI principles, such as traceability and explainability. This approach helps in aligning deployment decisions with predefined risk tiers, and the framework automates decisions on whether a system can proceed, requires remediation and fixes, or should be blocked. It also provides targeted suggestions for improvement and compliance for the lags identified. Results: We evaluated this framework on representative AI systems and demonstrated how a single evidence bundle can be used to support assessment across different governance regulations. Discussion: In doing so, Audit-as-Code ensures AI assurance transforms from a documentation-driven policy module to a quantitative, auditable, reproducible, and operationally practical module to ensure compliance.