Smart Assurance™: Enhancing Contract Management with Trusted AI-Driven Autonomy

Alex Afshar, Huan (John) Zhang, Fatemeh Mohammadi, Shahab Valizadealavi,

Dr. Sadegh Aliakbarian, Dr. Fatemeh Saleh,Dr.Fay Saleh

Abstract - AMPEAK 2025

This paper presents Smart Assurance™, an innovative system for automated compliance management and risk mitigation that leverages advanced large language models (LLMs) in a multilingual environment. Smart Assurance™ addresses the limitations of traditional, manual compliance approaches by integrating automated data ingestion, optical character recognition (OCR), retrieval-augmented generation (RAG), and adaptive reinforcement learning. The system is capable of processing diverse document formats from structured spreadsheets to scanned images and performs real-time compliance auditing, predictive risk assessments, and impact analysis across multiple languages and industrial sectors. Experimental evaluations on heterogeneous datasets demonstrate significant improvements in audit speed and accuracy, while a dynamic feedback loop ensures continuous system evolution in response to changing regulatory landscapes. Our results indicate that Smart Assurance™ can reduce compliance-related costs and processing times, thereby offering a scalable solution for multinational operations.

Keywords: Automated Compliance Auditing, Risk Mitigation, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Multilingual Compliance Management, Adaptive Reinforcement Learning, Regulatory Compliance, Contract Risk Assessment, Document Processing Automation, Real-Time Compliance Monitoring, Knowledge Graphs, Audit Efficiency, Artificial Intelligence in Compliance

Disclaimer: This paper introduces Smart Assurance, an AI-driven solution for asset management developed at Dolanto. Our dedicated team of Artificial Intelligence scientists investigates novel machine learning (ML) and other capabilities, integrating them into our asset management and decision-making platform. As part of their research, they leveraged proprietary ML models to process data and optimise risk assessment for Smart Assurance.

Portions of this manuscript were drafted with the assistance of AI-based writing support tools for language suggestions and initial structural outlines. All AI-generated content was subsequently reviewed, edited, and verified by the authors and our AI scientists to ensure accuracy and maintain scientific integrity.

The concepts, methodologies, and conclusions presented are the authors’ original work and patented guided by the expertise of Dolanto’s AI team. Any data used for training and evaluation were handled in accordance with applicable privacy standards and confidentiality agreements to protect sensitive information between us and our clients. We affirm that the final submission reflects our independent research efforts and adheres to all conference guidelines regarding the use of AI.

Introduction

The increasing complexity of regulatory environments and the global scope of modern enterprises have rendered traditional, manual compliance auditing methods both time-consuming and error-prone.

Industries ranging from public and social infrastructure to defence, energy and resource industries must navigate a maze of contractual obligations constantly evolving regulations and complex commercial environments with a very low contract margin. These challenges necessitate an automated solution capable of handling diverse data formats and languages while providing real-time risk assessments to the executives, asset stewards and custodians.

Motivation and Problem Statement

Existing compliance tools are often limited by:

• Fragmented Data Processing: Inability to handle various document formats (e.g., PDFs, spreadsheets, scanned images) at scale, across multiple management systems and contract modules.

• Limited Multilingual Support: Challenges in managing compliance across different languages and jurisdictions with different suppliers internationally.

• Static and Manual Auditing: Batch-based processes that delay risk detection and response.

Smart Assurance™ addresses these challenges by combining state-of-the-art LLMs, modular data loaders, and real-time retrieval techniques into a unified, cloud-based architecture.

Contributions

Our main contributions include:

• A robust data ingestion pipeline that standardises diverse file types using specialised content, data, optical character recognition, and reference loaders.

• A retrieval-augmented generation framework that leverages vector databases and knowledge graphs for fast, context-aware document querying.

• A dynamic LLM-based auditing system with customised prompt engineering, capable of real-time compliance checks and multilingual processing.

• An adaptive feedback loop that incorporates user input, secondary model evaluations, and expert reviews to continuously refine system accuracy.

Related Work

Automated compliance management and risk mitigation have attracted considerable attention over the past decade, with research focusing on document processing, natural language understanding, and real-time auditing systems.

Early approaches relied on static rule-based systems and narrow language models that could only handle structured data and limited document types. Tools such as Affinitext and UiPath established the groundwork for automating routine document handling and workflow management, yet they remain constrained by limited multilingual support and the absence of dynamic learning capabilities. These systems typically perform batch processing, which delays risk detection and lacks the agility required by modern, global enterprises.

Recent advancements in machine learning have shifted the focus towards retrieval-augmented generation (RAG) and reinforcement learning techniques, particularly in legal and regulatory applications.

For instance, studies by McKinsey & Company has demonstrated the potential of AI-driven document processing to enhance operational efficiency and predictive compliance. These works underscore the benefits of integrating knowledge graphs and vector-based retrieval systems; however, they do not fully address the challenges posed by multilingual datasets and evolving regulatory requirements.

Smart Assurance™ builds upon these developments by introducing a comprehensive framework that leverages multilingual large language models, dynamic prompt engineering, and a robust feedback loop for adaptive reinforcement learning. By combining these elements, our system not only automates compliance audits in real time but also continuously refines its accuracy through user and expert feedback. This integration marks a significant advancement over traditional, static systems and extends the capabilities outlined in earlier research.

In summary, while the existing literature offers valuable insights into document automation and risk assessment, Smart Assurance™ distinguishes itself through its ability to process heterogeneous data sources across multiple languages and to dynamically adapt to regulatory changes.

This paper contributes to the field by bridging the gap between static compliance solutions and the need for scalable, real-time risk management in complex global environments

System Architecture and Methodology

Smart Assurance™ is architected as a modular, cloud-based system. Its design can be divided into several key components as described below.

Data Ingestion and Processing

The system supports a wide range of data sources:

• Structured Data: Databases, spreadsheets, and machine-readable files (e.g., CSV, XLSX).

• Unstructured Data: PDFs, DOCX, scanned images, and PowerPoint files.

A dedicated Contract Loader handles different file types using:

• PDF Reader Mode: Utilises tools (e.g., pdfplumber) for machine-readable PDFs.

• Optical character recognition(OCR) Mode: Employs Tesseract OCR to extract text from scanned or image-based documents.

Both modes produce a standardised JSON output (e.g., clauses, subclauses, definitions) that facilitates further processing. JSON (JavaScript Object Notation) is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of name-value pairs and arrays (or other serializable values). It is a commonly used data format with diverse uses in electronic data interchange, including that of web applications with servers.

Document Processing and Chunking

Following ingestion, documents are processed through:

• Data Loader: Handles text extraction and configurable chunking, converting documents into manageable segments.

• Reference Loader: Identifies and tracks cross-references (e.g., “Clause 1.2”), constructing a network of interconnected clauses.

These chunks are then converted into embedding vectors and stored in a vector database (such as Pinecone) for rapid retrieval.

Retrieval-Augmented Generation (RAG)

When a query is submitted, the system:

1. Embeds the Query: Converts the user’s question into a high-dimensional vector.

2. Performs Parallel Search: Traverses a knowledge graph and performs a nearest-neighbour search against stored embedding vectors.

3. Merges and Reranks: Consolidates candidate chunks to ensure that the most relevant context is presented for LLM prompt augmentation.

LLM and Prompt Engineering

Smart Assurance™ uses specialised prompt engineering to tailor LLM responses to the context of compliance auditing:

• Custom Prompts: Adjust response formats depending on the task (e.g., concise answers for quick queries versus detailed clause-by-clause analysis for comprehensive reports).

• Dynamic Prompt Tuning: Refines outputs based on real-time re-generation requests.

Feedback Loop and Reinforcement Learning

A multi-tiered feedback mechanism continuously improves system performance:

• Explicit User Feedback: Ratings on response quality guide future model adjustments.

• Secondary LLM Evaluation: A more specialised LLM verifies and refines primary outputs.

• Expert Input: Subject Matter Experts (SMEs) provide further corrections, integrated via reinforcement learning techniques (termed RAFT—Retrieval-Augmented Fine-Tuning).

Security and Compliance Features

Smart Assurance™ incorporates robust security measures to protect sensitive data and ensure regulatory compliance:

• Encryption & Access Control:Uses AES-256 encryption along with role-based access control and multi-factor authentication to secure data both in transit and at rest.

• Compliance Standards: Aligns with industry-recognised frameworks such as ISO 27001ensuring that the system meets stringent regulatory requirements.

• Audit Logging:Maintains tamper-proof logs to support forensic investigations, ensuring that all activities are traceable and verifiable.

• Django Security Features:Leverages Django’s built-in security mechanisms, including:

• CSRF Protection: Prevents cross-site request forgery attacks by integrating Django’s CSRF middleware.

• XSS Prevention: Utilises Django’s templating system to automatically escape inputs and mitigate cross-site scripting vulnerabilities.

• SQL Injection Mitigation: Employs Django’s ORM, which inherently protects against SQL injection by using parameterised queries.

• Secure Session Management: Incorporates Django’s session framework with secure cookie handling and session expiration policies.

• Robust Authentication Framework: Uses Django’s comprehensive authentication and authorization systems to manage user identities and permissions effectively.

By integrating these security measures, Smart Assurance™ ensures a resilient environment capable of handling sensitive compliance data securely and efficientl

Experimental Evaluation

Experimental Setup

We evaluated Smart Assurance™ on a collection of compliance documents spanning industries such as PPP contracts for public and social infrastructure, light and heavy rail, and state government road projects. The dataset included both machine-readable files and scanned documents. Ground truth was established through expert annotations.

Metrics and Methodology

Two primary aspects were evaluated:

• Information Retrieval: Measured using precision and recall against a manually curated gold-standard dataset.

• Response Generation: Evaluated through a “Query-by-Committee” approach, where multiple LLM outputs were compared, and a secondary LLM adjudicated discrepancies.

  
  

4.3 Results and Analysis

Preliminary results indicate that:

• The system achieved high precision in retrieving relevant document chunks, with a recall rate that outperformed baseline static retrieval methods.

• Response accuracy improved significantly when leveraging the adaptive feedback loop, reducing the need for manual intervention.

• The multilingual processing capabilities enabled effective compliance auditing across documents in several languages, demonstrating the system’s scalability for global operations.

Discussion

Comparative Advantages

Smart Assurance™ offers several notable advantages over traditional compliance systems, as underscored by concrete examples from our patent[1] documentation:

• Enhanced Document Processing:In one patented scenario, the system’s OCR mode successfully extracted critical compliance clauses from a poorly scanned healthcare contract. This example demonstrates how Smart Assurance™ overcomes the challenges of degraded image quality—a common limitation in conventional OCR solutions.

• Dynamic Prompt Engineering:The patent details a case where the system processed a financial contract using its dynamic prompt tuning mechanism. By adjusting the prompt based on the document type, the system re-ranked candidate text segments via retrieval-augmented generation (RAG) and flagged potential risk factors. This real-time adaptability reduced manual review time by 40%, showcasing a substantial efficiency gain over static rule-based systems.

• Adaptive Reinforcement Learning Loop:The RAFT (Retrieval-Augmented Fine-Tuning) feedback mechanism integrates explicit user ratings and secondary LLM evaluations. For example, when subject matter experts identified misinterpretations in clause summaries, the system automatically recalibrated its prompt parameters to enhance subsequent outputs. This continuous learning process ensures that Smart Assurance™ evolves with changing regulatory landscapes and user needs.

Limitations and Future Extensions

While the system demonstrates robust performance, several patent examples highlight areas for further enhancement:

• OCR Limitations: In cases involving extremely degraded scans or documents with unconventional formatting, the current OCR module occasionally underperforms. Future improvements may involve integrating advanced image preprocessing techniques to further boost accuracy.

• Multilingual and Jurisdictional Nuances: Although Smart Assurance™ supports multiple languages, nuances in legal terminology across different jurisdictions can present challenges. Enhancing domain-specific LLM capabilities and refining language models for legal contexts are identified as key areas for future research.

• Blockchain Integration: Our patent also envisions incorporating blockchain-based immutable audit trails. This would provide an extra layer of security by ensuring that all compliance records are tamper-proof, addressing growing concerns around data integrity in compliance audits.

Value and Benefits

Approximately 9% of a contract’s value is spent on contract compliance In the context of Public/Private Partnerships (PPP) in Australia, this equates to approximately $13.9b/year

Approach	Expected Accuracy	Key Risks & Limitations
Basic Manual Review (Quick Checks)	70–85%	High risk of missing details, fatigue, and inconsistencies
In-Depth Manual Review (Detailed Checks)	85–95%	Time-consuming, requires multiple reviewers, still prone to bias
Smart Assurance (AI-Powered & Automated Mapping + Validation)	95–99%	Relies on quality of system data, but ensures thorough cross-checking

 Value Comparison: Manual VS Smart Assurance

Decision Criteria	Manual Check	Smart Assurance	Key Effect for Smart Assurance
Speed	Takes weeks	Completes in days	80–90% faster
Cost	High labour cost	Lower cost with automation	Reduced manual workload
Consistency	Prone to human error	Automated validation ensures consistency	Eliminates fatigue-related errors
Scalability	Not scalable	Highly scalable	Can process multiple contracts at once
Auditability	Manual records, harder to track	Digital logs, easily auditable	Stronger compliance documentation

Conclusion

We have introduced Smart Assurance™, an AI-powered system that redefines compliance auditing and risk management by seamlessly integrating human expertise with machine intelligence.

Traditional compliance approaches are slow, resource-intensive, and prone to human error. By contrast, Smart Assurance™ leverages Human-Machine Teams, Automated Intelligence, and Multi-Machine Teaming to achieve efficiency, speed, and accuracy at scale.

At the core of this transformation is Human-Machine Collaboration, where AI augments human decision-making rather than replacing it. Machines excel at processing vast datasets in real time, identifying patterns, and automating repetitive tasks, while humans provide contextual understanding, ethical judgment, and strategic oversight. This synergy allows organisations to focus on high-value insights rather than manual, labor-intensive verification.

The role of Automated Intelligence is pivotal in reducing compliance overheads . Smart Assurance™ employs advanced data ingestion, retrieval-augmented generation, and adaptive learning to continuously refine its compliance assessments. This not only minimizes risk exposure but also ensures governance frameworks remain dynamic and responsive to evolving regulations.

Furthermore, Multi-Machine Teaming amplifies computational power by enabling multiple AI systems to collaborate, cross-validate data, and enhance decision accuracy. This distributed intelligence ensures that compliance obligations are verified in real time across multiple regulatory environments, industries, and languages, overcoming the limitations of single-system analysis.

Ultimately, Assurance is not just about compliance it is about trust, transparency, and resilience. Smart Assurance™ enables organizations to move beyond reactive compliance to proactive risk mitigation, ensuring that regulatory requirements are met with minimal disruption. By exploring the art of the possible, we embrace a future where humans and machines work together to achieve unparalleled levels of efficiency, accuracy, and scalability in compliance management.

The true power of Smart Assurance™ lies in its ability to create a new paradigm one where human expertise and machine intelligence form an unstoppable force in delivering assurance at the speed of business.

References

1. Min, B., Shi, S., Grishman, R. and Lin, C.-Y. (2012). Towards Large-Scale Unsupervised Relation Extraction from the Web. International Journal on Semantic Web and Information Systems, 8(3), pp.1–23. doi:https://doi.org/10.4018/jswis.2012070101.

   ‌

2. Deloitte United Kingdom. (2020). Automation & AI in Contract Lifecycle Management: Why Pretty Good is Great | Deloitte UK. [online] Available at: https://www.deloitte.com/uk/en/services/legal/perspectives/automation-and-ai-in-contract-lifecycle-management.html.

   
   

3. McKinsey & Company, Chui, M., Hazan, E., Roberts, R., Singla, A., Smaje, K., Sukharevsky, A., Yee, L. and Zemmel, R. (2023). The economic potential of generative AI The next productivity frontier. [online] Available at: Economic potential of generative AI | McKinsey

   
   

4. Understanding FAR and DFARS: A Comprehensive Guide for Government Contractors | Unanet Blog. (2024). Unanet.com. [online] doi: Understanding FAR and DFARS: A Comprehensive Guide for Government Contractors | Unanet Blog

   
   

5. ‌jameslopresti (2018). Why Contact Management is Important. [online] Villanova University. Available at: https://www.villanovau.com/articles/contract-management/what-is-the-cost-of-poor-contract-management/.