Introduction

AI has been revolutionizing the way business is being done since its emergence, as a game changer in the domains of accounts, audit, taxation, and assurance. Auditor has no option in the digital era without getting familiar with the incipient AI auditing frameworks, software solutions and tools to competently perform the audit and assurance assignments. AI technology cannot substitute auditor’ discretion, wisdom, and judgement. AI audit services include auditing and assurance, taxation, and financial advisory. AI-enabled technology platform can provide high-quality auditing services, customized services in AI governance, risk assessment, and controls, risk management and compliance with AI ethics and regulations.

What AI enables is helping to face massive structured and unstructured big data mines by leveraging AI’s big data analytics and machine learning capability to systematize and arrange data fields, and conduct data analytics to answer probing audit questions, highlighting potential risks and vulnerabilities for detailed audit scrutiny for risk assessment and management. Risk of material misstatement is critical in financial statements’ attestation audit, requiring intense data analysis of connected complete data sets and transactions. AI empowers auditor to increase audit quality in every gamut of audit profession. Auditors use risk assessment to determine material misstatement in financial statements.

“AI helps the auditor to focus on more risky domains and helps using professional judgment to conduct detailed intense audit scrutiny on targeted financial statement transactions.”

Auditor can analyze complete groups of data and transactions rather than relying on sampling, making audit checks complete to identify anomalies and red flags for additional scrutiny. AI helps to automate tasks hitherto were performed manually and to converge financial statement auditing with forensic auditing wherever fraud is suspected based on complete big data analytics of connected data fields. AI helps to suggest better controls for accounts payable, accounts receivable, enterprise risk management, and financial planning and analysis functions by continuous risk monitoring and assessment procedures. CAs can perform more economic, effective, and efficient tax advisory, and audit advisory services using appropriate AI apps and software to scan documents and import relevant information and data for conducting intense audit procedures and checks. When data is kept in a cloud, AI can connect that data and extract relevant data sets using AI auditing platform to conduct real-time audit probing and analysis.

This article introduces AI enabled auditing landscape and cutting edge technologies that can be effectively deployed by Chartered Accountants while performing various audit, attestation, assurance and advisory functions, focusing particularly on Financial audit and Internal audits.

AI Audit Technologies

Auditors must be mindful using AI solutions. Auditors should be sensitive with corporate/client information. The audit programs with AI solutions should be reviewed and adjusted according to scope and objectives of the engagement. AI models must be re-trained with datasets to obtain desire output. AI can support internal audit functions by improving efficiency, resource optimization, knowledge, and skill development.

ChatGPT a chatbot, developed by OpenAI ‘Generative Pre-training Transformer’, is a language model that can generate human link text in a conversational context. ChatGPT includes AI/ML technologies, Chatbots, Robotic Process Automation (RPA), Deep Q&A questions and probable answers from ChatGPT, Text to speech conversion, image and voice recognition, translation, automate speech, writing, unsupervised deep learning, predictive analytics, forecasts, actions, performing repetitive actions, and bigdata analytics.

ChatGPT can help in audit program in access controls ensuring that only authorized persons access IT systems and data; Change management helps the auditors to review the policies and procedures in place to ensure that IT systems and data are properly authorized, tested, and implemented; system development and maintenance in a secure and controlled manner; IT systems and data are protected from disasters by putting needed physical and environmental controls; proper back up and restoration are ensured by disaster management controls, ensure network security form cyber threats and hacking, IT incident response is adequately geared up to take preventive and corrective actions.

“AI audit management software facilitates in audit planning, scheduling, audit program, fieldwork, evidence gathering, reporting, and follow-up reviewing.”

AI audit software must provide corrective and preventive action (CAPA) features enable companies to implement audit suggestions regarding safety and compliance to improve audit results. AI audit management software facilitates in audit planning, scheduling, audit program, fieldwork, evidence gathering, reporting, and follow-up reviewing. Central document and data storage helps auditor to streamline data collection, analyze financial statements, and reference documents, making inquiries, and prepare reports. Actionable digital checklists help auditing teams to collaborate with organizations to have better control over auditing projects. Many auditing apps help to generate audit reports. The auditing management systems leave audit trail ensuring accountability and compliance. Many audit software solutions are available for easing the audit tasks.

AI regulations and Auditing Frameworks

The European Parliament has recently approved rules for artificial intelligence, known as the EU AI Act. European Parliament decided to bring generative AI tools like ChatGPT under greater restrictions. Generative AI developers Microsoft-backed OpenAI’s ChatGPT and Google’s Bard are required to submit their systems for review and approval before releasing them commercially.

The General Data Protection Regulation (GDPR) in the EU regulates the organizations while using personal data with seven principles: Lawfulness, Fairness, and Transparency regulates that personal data processing must abide by the law. Data must be used only for a specific purpose. Personal data usage must be adequate, limited, and restricted. Data used should be accurate and up to date. Personal data must be stored for the specified purpose and for the period required. Personal data used must be processed securely. Data must be processed responsibly complying with the regulations.

Benchmarked AI Auditing Frameworks:

1. COBIT Framework (Control Objectives for Information and related Technology): for IT governance and management of an enterprise.
2. IIA’s (Institute of Internal Auditors) AI Auditing Framework: to assess the design, development, and working of AI systems and their alignment with the organization’s objectives, focusing on ‘Strategy, Governance, and Human Factor’ with the following seven elements:
   • I. Cyber Resilience
   • II. AI Competencies
   • III. Data Quality
   • IV. Data Architecture & Infrastructure
   • V. Measuring Performance
   • VI. Ethics
   • VII. The Black Box
3. COSO ERM Framework: for assessing the risks for AI systems in an organization with five components for internal auditing:
   • I. Internal Environment: Ensuring that Organization’s governance and management are managing AI risks,
   • II. Objective Setting: Collaborating with stakeholders to make risk strategy,
   • III. Event Identification: Identifying risks in the AI systems such as unintended biases, data breaching,
   • IV. Risk Assessment: looking at the impact of the risks,
   • V. Risk Response: focusing on entity’s risk situations, such as sub-optimal data quality.

Risk management

Currently, there are few precedents for handling AI audits. Issues and concerns are innumerable: lack of AI audit standards, varied AI systems/platforms, existing and evolving complex, innovative frameworks/software solutions and tools, dearth of competent data scientists, AI-specific regulations. Till auditing standards and regulations specific to AI are in place, auditors must adopt and adapt existing frameworks and regulations and communicate proactively with the stakeholders about AI systems, risks, and concerns. As there are varied AI definitions/taxonomies/systems/solutions/designs/architecture, and complexity of AI technologies, auditor must ensure transparency through iterative process, evaluation of controls, manage risks and governance issues.

The COBIT 2019 framework provides process descriptions, desired outcomes, benchmarked practices, and work products across IT spheres. While auditing cloud computing and cybersecurity, auditor must assess whether the Open Systems Interconnection (OSI) layer implementation was functioning effectively and focus on the controls and governance structures to determine whether they are operating effectively to provide some assurance on the business and IT governance. To identify risks, IT auditor must use risk and control matrix (RCM) and list out all the existing risking and how to control and manage them. ISACA lists of some of the AI audit risks:¹

• Lack of alignment between IT plans and business needs
• IT plans that are inconsistent with the organization’s expectations/requirements
• Improper translation of IT tactical plans from the IT strategic plans
• Ineffective governance structures that fail to ensure accountability and responsibility for IT processes related to the AI function

“Auditor must be vigilant about supplier risks in AI outsourcing like clouds to third parties, and document for cross-team transparency.”

Auditor must be vigilant about supplier risks in AI outsourcing like clouds to third parties, and document for cross-team transparency. AI encompasses diverse technologies, people, and processes and requires evaluation of risks and controls, policies, and governance. AI architecture generally combines ‘programming, data warehousing, stream processing platforms, machine learning tool kits, algorithms, cloud computing, cloud storage, computing clusters, compute kernels, application software testing and debugging, data process and modeling, and commercial off-the-shelf (COTS) software’. From a skills perspective, AI audits may require engagement of data scientists, data engineers, data architects and programmers. Auditor must adopt and adapt existing IT frameworks such as COBIT 2019 and regulations including the United States Health Insurance Portability and Accountability Act (HIPAA), and Fair Lending Act and the European Union’s General Data Protection Regulation (GDPR) until more specific AI standards are put in place. Algorithms require multiple rounds of tuning by data scientists and data engineers and enterprise-based commercial off-the-shelf solutions may contain components of machine learning.

A) Financial Audits

Financial audit requires mapping of all enterprise data fields connected to financial statements and collect audit evidence to form an audit opinion on whether the financial statements represent true and fair state of financial health of the entity in all material respects. AI technologies enhance efficiency, effectiveness and accuracy in the audit process but cannot replace the audit discretion, judgment, and professional opinion. Leveraging AI tools such as machine learning (ML) and natural language processing (NLP) to relevant big data sources, AI highlights potential risky fraudulent patterns to the auditor to execute intense audit probing. ML and NLP enable software to learn without being programmed to perform specific tasks to facilitate auditor to produce consistent, reliable audit outcome by identifying patterns, classifying risks and help taking mitigating and preventive controls and predicting risks.

Compliance issues

Audit must focus on compliance issues of AI applications. AI auditor must assess risk related to the rights and freedoms of data subjects, understand the data privacy and data protection principles and the impact of AI applications on the rights and freedoms of data subjects. The UK’s Information Commissioner’s Office (ICO) formulated some guidelines that serve as a baseline for auditors auditing AI applications under the EU General Data Protection Regulation (GDPR). Data Protection Impact Assessments (DPIAs) is legally mandated if entities’ AI systems process personal data, to comprehend how and why organizations’ AI systems process personal data, and what are any risks. AI systems involve trade-offs between privacy and other competing rights and interests, and therefore, auditor must assess what these trade-offs are and how to manage them.

Fair, lawful, and transparent processing

Auditor must guard the risk of failure to comply with the legally valid data protection principles. Auditors must use personal data with appropriate levels of security against its unauthorized/ unlawful processing, accidental loss, destruction/damage and must verify that all movement and storage of personal data from different locations are secure and documented to monitor security risk controls. Under the extant data protection law and regulations, individuals have rights to their personal data. Auditors must respect individual rights of information, access, rectification, erasure, and to restriction of processing, data portability when deploying AI. Big data requires appropriate controls to ensure its proper use. Auditing in the AI landscape involves evaluation of algorithms, models, and data streams, analysis of operations, results, and even unexpected outcomes, technical and ethical aspects of AI systems and adherence to principles such as equality and privacy.

Use of ML, NLP, RPA for Auditing

Auditor faces massive data mines, making it cumbersome to select audit samples. AI technologies RPA, ML and NLP help to provide insights from vast sea of data. AI technologies help in reduction in data processing cycle time, reduction in oversight errors, reducing time required for evidence verification. Data checks help to detect money laundering, fraudulent transactions, and ease accounts reconciliation. Auditor will be able to make intelligent predictions and insights. While using AI tools, auditor must be aware of ethical issues and data bias while using data. Moreover, inadequate testing of AI outcomes can lead to questionable audit outcomes. Further, human logic errors might corrupt AI algorithms used for auditing. AI technologies help in document classification, text summarization, data analytics and topic analysis, statistical analysis, data aggregation, interpretation, search, and retrieval and sentiment analysis with inferences to understand author’s sentiment.

The Chartered Institute of Internal Auditors (IIA) reports that data analytics enables internal auditors ‘to deliver faster and more incisive insights into fast-moving risks’ to help the board of directors to take appropriate action swiftly. As computers will not be capable to give a ‘nuanced control design opinion, internal audit will always require a human touch.

B) Internal Audit (IA)

IA has a critical role in maintaining the integrity of entity operations. AI assists IA in several ways: Internal Auditor is empowered to evaluate thousands of documents and contracts. ML increases audit productivity, accuracy, and timely audit execution with speed by automating audit checks by AI algorithms. AI helps in identifying relevant and reliable data aggregation and extraction and analysis, making audit reports, with useful insights, error free complete audit coverage rather than relying on samples, expediting the audit process. AI helps IA in developing audit plans, defining the audit objectives, scope, and methodology, to conduct big data analytics to analyze oceans of structured and unstructured data including sources relevant to the audit assignment including from social media to identify patterns of risky domains needing in-depth audit scrutiny and predict forecasts and trends. AI enables to save time and effort in developing audit plans, audit programs, field tests, evidence gathering, validating, testing bringing in new insights, testing procedures, and reporting techniques and methodologies.

AI auditing is around the clock, with continuous real time risk assessment and monitoring, saving money, efforts, enhancing audit focus on priority areas to improve audit quality with intense assurance reviews, exception management, process development, and interpersonal interactions bringing in value addition and operational efficiency and effectiveness. AI software solutions can help smooth execution of audit, sustained upkeep, and risk reduction and better integration rather than functioning in silos between departments.

Entity administration must establish a responsibility framework determining responsible persons for varied tasks and processes for automation. Standardized documentation and design review processes should also be part of a governance framework. There must be proper procedures put in place for keeping tabs on and fixing any alterations made to automated tests and processes and dealing with aberrations and impacts. Dynamic competitive corporate operations require regular around the clock risk assessment, cybersecurity checks and quality control testing. The operational and technological exceptions necessitate a testing framework and procedures to fix issues as and when emerge. Programme managers must be vigilant to regularly evaluating staff competencies to fill knowledge and expertise gaps focusing on recruitment of right personnel and role-specific training. AI helps to adapt change management process and procedures.

Internal Auditors can provide assurance and add value to the company by conducting AI enabled risk assessment and management and perform more proficient internal audit advisory and assurance works. They must identify systems and processes to develop right audit process. AI apps, whether developed in-house, or procured, must ensure realization of the intended purpose, meet design specifications, and assumptions on specific monitoring to be performed, based on company’s legal compliance requirements. As Internal audit is an operational audit to evaluate risks and internal controls of operational systems for departments, units, and business functions, its goal is broader than an external audit in the achievement of organizational objectives and determines ways to improve the operations. As compliance audit is an independent evaluation to determine whether the entity is complying with the requisite regulatory standards such as corporate bylaws, controls, policies, and procedures.

AI ML capabilities empower the auditor to do more audit advisory assignments such as understanding the entirety of ledgers and reporting on risks to executives and clients while enhancing quality of audit service. Use of ML improves the testing of ledger data by analyzing the entire dataset in a short time frame to identify material misstatements based on complete risk analysis. AI-based tools can flag transactional data based on variance from the standard set. AI-powered tools can help businesses to detect duplicates, out-of-policy spendings, incorrect amounts, suspicious merchants/attendees, and excessive spendings. Audit benefits include reduction in workload, cost reduction, enhanced audit quality. Many AI tools and services such as AI Consultant, AI/ML Development Services, Audit Software, Data Science/ML/AI Platform are in the market to ramp up audit productivity and efficiencies. Internal audit can effectively discharge responsibilities of regulatory compliance, monitoring risks and controls, and corporate governance. RPA and use process automation tools help even retailers, banks, global payment solution providers, manufacturers, media companies, movie advertising networks to handle seas of data from text/audio/video/social media sources. AI, ML and RPA will allow businesses and internal audit functions to rise much faster. The CACS (Commitment, Access, Capability, Skilling) Framework is comprehensive step in this direction. Adoption of proper CACS framework can be the inflection point for internal audit paving the way for AI to review volumes of unstructured data. CACS framework leads to next gen internal auditing. Internal audit teams must step up the audit delivery time in providing assurance, advice, and insights by adoption of AI enabled technologies.

“AI-powered tools help zero in fraudulent transactions, waste, excessive spending, suspicious merchants, duplicates and enhance audit delivery time by proper risk assessments.”

AI-powered auditing platforms can analyze the entirety of the financial transactions, identify gaps and can help recognize areas of highest risk of material misstatements with accuracy and speed. AI-powered tools help zero in fraudulent transactions, waste, excessive spending, suspicious merchants, duplicates and enhance audit delivery time by proper risk assessments. IA must embrace AI for better project scoping and Return on Investment taking care of risks including cyber threats. As corporate is outsourcing key business functions, internal audit process will extend far beyond business premises. As businesses tend to have less oversight over their contractor’s practices, IA faces a complex environment of fourth and fifth parties in completing audits. Multiple external threats add complications to the audit process including shortages of IT experts, climate change uncertainties, cyber threats, disrupted supply chains, and political/social/environmental issues. AI algorithms are suited to explore correlations between these factors and the potential risks they cause for a business by review, evaluation, and impact.

AI powered audit tools e.g. AuditMap.ai for reviewing internal reports and other documents to develop a risk profile by identifying trends before they can pose a risk, allowing the organization to develop effective mitigation strategy. AuditMap works as a cloud-based internal audit function tool or in a hybrid version. AI algorithms based on state-of-the-art natural language processing is highly effective in the assessment of risks and controls. AI-powered auditing platforms help in uploading audit reports and documents to enable better work ecosystem.

Overcoming AI audit challenges relies on improving the understanding of AI technology landscape and how it can transform audit profession with its inherent risks. Auditors are not data scientists. Their understanding of AI is limited. Appointing audit committees to liaise with internal audit teams is valuable step to bridge this knowledge gap.

Benefits of AI in Internal Audit

AI poses both risks and opportunities for internal auditors. On the one hand, the function must provide assurance their business is using these technologies appropriately, but auditors can also leverage AI and AI-adjacent systems to their advantage. AI alleviates the burden of laborious manual processes. AI can free up audit professionals for more value-added tasks. AI can offer significant risk and governance insights and strategic suggestions to the board of directors of enterprises. In the process, auditor’s assurance-based role extends to a key business advisor for the company’s future growth. Working with cutting-edge AI technologies will attract the industry’s leading professionals. AI adds audit efficiency and productivity. Generative AI tools empower IA to enhance their efficiency and productivity.

AI tools help in knowledge and expertise augmentation by leveraging AI generative tools to provide insights and forecasts. AI helps in providing better consistency and standardization, helps in data-based decision-making rather than reliance on professional Judgment, though AI tools should not replace the professional judgment and expertise of internal auditors. AI platforms and software provide better security and confidentiality. Generative AI may likely to produce risk of false information, which must be safeguarded by associating with subject matter experts. Regulations relating to intellectual property rights may create reputational risk. IA should compare the tool’s responses against reliable sources, consult subject matter experts. IA should support development of policies that govern the use of AI tools. Continuously monitoring and evaluating the risks and performance help enhancing effectiveness, reliability, and value. IA professionals must receive adequate training in handling effectively AI software and tools with associated risks, and ethical considerations. While external audits are independent to certify risks and controls, IA is proactive rather than reactive to anticipate potential bugs/risks for taking corrective and preventive actions to improve objectivity, transparency, efficiency, and productivity.

AI Audit Checklist

AI adoption requires clarity on business objectives and how AI is used to achieve those. AI strategy must be aligned to enterprise achieve and justifies their AI expenditures. AI audit data sources include internal, third-party, and public data sources, and auditor must assess data validity of data resources used. AI audit cannot be based on unreliable data sources because of ‘garbage in garbage out’ will be the audit outcome. Data privacy is a key concern for the implementation of privacy standards, protection of consumer rights, and legal aspects around data usage. Changes in algorithms and data might impact the accuracy of AI systems. Continuous monitoring, algorithm assessment, and checking possibilities of potential vulnerabilities constitute the core of AI audit.

AI systems are prone to security attacks by hackers and therefore must be backed by appropriate security controls and standards. Therefore, responsibility of the AI auditor is to ensure data integrity, completeness, accuracy, confidentiality, and availability of data, privacy issues, legal issues like copy rights infringements of data used in the audit. Identifying and vetting the data sources, checking for data quality and cross validation of data fields are critical before beginning the audit. If AI systems use personal data, auditor must valuate that cloud services meet the information security requirements such as OWASP (Open Web Application Security Project) guidelines. AI Auditing is a continuous iterating process. AI system and risk strategy should be modified based on the feedback, usage, consequences, influence, and impact.

Key Audit Checklist Questions:

1. What is the data source?
2. Is the source reliable and objective and the data integrity ensured?
3. Are the data fields easily available, accessible, complete?
4. Are the data fields relevant and complete for audit evidence to form the audit opinion?
5. Is the ‘single version of the truth (SVOT) agreed upon’?
6. Is the governance, risk, and compliance (GRC) properly evaluated and controlled?
7. Are AI components available help to solve data sampling issues and related internal audit problems to derive meaningful insights?
8. Is the audit output (the end results of the audit function) desirable and satisfactory?
9. Is the audit quality ensured?
10. Do the audit results justify costs?
11. Is the audit impact as expected?
12. How far the predictive analysis help predicting future trends?
13. Does robotic process automation (RPA) automate auditing steps and data extraction from the data fields into Word/Excel?
14. Does NLP, that automates repetitive tasks via voice commands validate audit checks?
15. Do Natural language generation and ingestion/NLP-based Chatbots help in reconciliations based on checklists?

Benefits of Auditing AI Systems

• Auditing prevents or mitigates risks associated with AI systems.
• Auditing ensures that AI applications are free from inherent data bias and discrimination.
• Auditing AI applications ensures that the system follows legal, regulatory, ethical, and social considerations.
• Technology risk assessment evaluates technology capabilities, including ML, security standards, cyber security, and performance.

Challenges for Auditing AI Systems

• AI systems can amplify the data biases which might result in unfair decisions. Auditor must be able to prohibit discrimination in AI systems.
• AI systems, that employs ML, deep learning, neural networks are complex to interpret.

Organizations, regulatory authorities, and auditors should keep in touch with AI advancements, realize its potential threats, and frequently revise the regulations, frameworks, and strategies to ensure fair, risk-free, and ethical use. AI and other emerging technologies can save valuable time and resources by offloading repetitive and mundane tasks from auditors. This can help auditors focus more on areas where creativity and critical thinking are important. AI solutions assists auditors help to save time, cost, and enhance knowledge, skillsets on emerging technologies.

---

Footnotes & References

¹ European Commission Auditing Artificial Intelligence Guidelines: https://ec.europa.eu/futurium/en/system/files/ged/auditing-artificial-intelligence.pdf

---

Author may be reached at: kps.ps2013@gmail.com and eboard@icai.in