By: Priscilla Campos Díaz

Acknowledgement Note:
This research was carried out as part of the Bluedot AI Safety Fundamentals course "AI Alignment". 

Abstract

This research examines global and regional practices in AI auditing, focusing on frameworks from the European Union and Latin America to identify strategies applicable to Costa Rica. It evaluates the country's legislative initiatives and explores its potential to lead ethical AI governance in the region. Additionally, the study addresses critical challenges, such as limited access to reliable data, insufficient technical expertise, and resource constraints, which hinder effective AI oversight. By proposing tailored recommendations for an AI auditing framework, this research aims to foster a responsible and transparent AI ecosystem that upholds Costa Rican values while aligning with international standards.

Introduction

Artificial Intelligence (AI) is experiencing exponential growth worldwide. The software and information services industry, renowned for its dynamic nature, has positioned itself at the forefront of this technological revolution. By 2024, global spending on AI technologies reached $33 billion, highlighting the sector's pivotal role in transforming how services are delivered and consumed (Massey, 2024). Costa Rica has embraced this phenomenon, recognizing AI as a powerful driver of innovation and economic development. However, implementing these advanced technologies brings significant challenges, particularly in ensuring ethical and equitable practices.

In response to these challenges, Costa Rica has made strides toward regulating AI. The country currently has three legislative proposals under consideration in the Legislative Assembly, along with a National Artificial Intelligence Strategy (ENIA). This research focuses on analyzing how these emerging regulations address the critical issue of AI auditing. The study is based on information available as of January 25, 2025.

AI auditing refers to the systematic evaluation of algorithms and models to ensure fairness, transparency, and compliance with applicable regulations. This process is crucial for identifying and mitigating algorithmic biases, safeguarding data privacy, and ensuring that automated decisions do not disproportionately harm vulnerable populations (Mökander, 2023). For audits to be effective, a clear and well-defined framework is essential. In its absence, audits risk becoming inconsistent and unreliable, eroding public trust in AI systems.

Currently, Costa Rica lacks a publicly accessible, AI-specific auditing framework, underscoring the urgent need to develop one that aligns with the country's unique context. Addressing this gap requires analyzing and drawing lessons from the experiences of other nations that have advanced in AI regulation and auditing practices.

This research aims to examine international practices in AI auditing and, based on these insights, propose recommendations tailored to Costa Rica's context. Through a comparative approach, the study seeks to provide a clearer understanding of the state of AI auditing in the country, informed by both local legislation and global best practices.And aspires to contribute to the broader conversation on ensuring that AI development is safe, inclusive, and aligned with the values and needs of Costa Rican society.

Context of AI audit on Costa Rica

Costa Rica currently has three distinct legislative bills addressing artificial intelligence (AI), none of which have been approved by the Legislative Assembly. In addition to these proposals, the country has introduced the National Artificial Intelligence Strategy (ENIA), a comprehensive public policy framework aimed at guiding the ethical and responsible development of AI.

According to the Ministry of Science, Innovation, Technology, and Telecommunications (MICITT), efforts to date have concentrated on reviewing and providing feedback on the proposed bills. The Ministry has also emphasized its intention to closely monitor developments in European AI legislation to inform local regulatory approaches (Murillo, October 2024).

Notably, Costa Rica distinguishes itself as the first Central American nation to launch a national strategy dedicated to AI, underscoring its commitment to becoming a regional leader in the field (Campos,  2024).

The following sections detail how AI auditing is addressed in the legislative bills and the ENIA:

1. Law for the Regulation of Artificial Intelligence in Costa Rica (Bill No. 23771)

   
   This legislative bill, presented to the Legislative Assembly on May 30, 2023, was drafted with the assistance of ChatGPT-4 by Congresswoman Vanessa Castro. Using a sequential prompt, the proposal was tailored to align with Costa Rica's legal framework, including its 1949 Constitution. Key aspects of AI auditing outlined in the bill include:

   • Article 6: Supervision and Auditing
     Establishes a competent authority responsible for supervising and auditing AI systems. This authority is empowered to verify legal compliance, investigate complaints, and impose sanctions when necessary.
   • Article 8: Bias and Discrimination
     Requires AI developers to implement technical and organizational measures to mitigate algorithmic biases and prevent discriminatory outcomes. Regular audits and the use of representative, diverse datasets are mandated to ensure equitable and unbiased results.

   • Article 15: Respect for Human Rights
     Stresses adherence to human rights principles as outlined in Costa Rica's Constitution and international treaties. Independent entities are proposed to oversee and audit AI systems’ compliance with these principles, with the authority to investigate and penalize violations.

      

2. Law for the Responsible Promotion of Artificial Intelligence in Costa Rica (Bill No. 23919)

   
   Introduced by Congressman Oscar Izquierdo, this bill does not include a specific section or article addressing AI auditing.

    

3. Law for the Implementation of Artificial Intelligence Systems (AI)

   
   Drafted by Congresswoman Johanna Obando, this bill highlights the importance of AI auditing under:

   • Article 3: Principles and Good Faith in AI
     Calls for due diligence and auditability throughout the entire lifecycle of AI systems, taking into account the associated risks and the current state of technological advancements.

      

In addition to these legislative initiatives, Costa Rica launched its National Artificial Intelligence Strategy (ENIA) in October 2024, led by MICITT. ENIA includes several provisions emphasizing the importance of AI auditing:

• Bias Audits
  ENIA highlights the necessity of regular bias audits to promote fairness and prevent algorithmic discrimination. Specific measures, such as gender audits, are recommended during both the development and deployment phases of AI systems.
• Human Oversight
  The strategy advocates for inclusive and transparent mechanisms of human oversight, enabling effective intervention when needed. Continuous testing and auditing are emphasized to correct potential biases and enhance accountability.
• Risk Management for High-Impact Systems
  High-risk AI systems, particularly in critical sectors like healthcare, education, and justice, require rigorous pre-deployment evaluations. Ongoing human oversight and regular audits are mandated to ensure adherence to ethical and legal standards.
• Transparency and Accountability
  ENIA proposes comprehensive mechanisms for supervision, impact assessment, and auditing across the AI lifecycle. Both internal and external audits are recommended to evaluate compliance with established regulations, assess the effectiveness of mitigation strategies, and review automated decision-making processes.

LATAM Context 

After analyzing Costa Rica's approach to AI auditing, table 1 from the International Panel on the Information Environment. (2024), highlights how auditing principles are being integrated into national policies in other Latin American countries such as Brazil, Chile, Mexico, and Uruguay. AI audits, which involve evaluating the fairness, accountability, and compliance of AI systems, have become a cornerstone of responsible AI adoption.

Table 1. Policy documents related to AI governance that incorporate audits.

Source: OECD’s live repository of AI strategies and policies and IPIE analysis based on literature review [127]

To gain a more comprehensive understanding of the context of AI in Latin America, and to identify the region's current position in various dimensions such as governance, research, and professional training in AI, the following three charts are presented. These charts are sourced from ILIA, the Latin American Artificial Intelligence Index, that provides quantitative and qualitative data on the progress of Artificial Intelligence in 19 countries across Latin America and the Caribbean, enabling the identification of achievements, gaps, and opportunities for improvement in AI ecosystems (ILIA, 2024).

Figure 1: Total Score for Governance Dimension 

Source: ILIA 2024

Figure 2: Score for Research Subdimension

Source:  ILIA 2024

Figure 3: Score for Professional Training in AI Indicator

Source: ILIA 2024

International AI auditing recommended practices

The global landscape of artificial intelligence (AI) auditing has seen the development of numerous frameworks and guidelines, reflecting efforts to ensure transparency, fairness, and compliance with regulatory standards. These frameworks are particularly valuable for countries like Costa Rica, where AI auditing practices are still in nascent stages.

European Data Protection Board's (EDPB)

The European Data Protection Board (EDPB) has created an AI Auditing Checklist aimed at evaluating AI systems within their operational and regulatory contexts. The checklist provides a dynamic and adaptable methodology for assessing whether AI systems comply with data protection principles, such as those outlined in the General Data Protection Regulation (GDPR) (Galdon, 2024).

The EDPB checklist emphasizes the importance of evaluating AI systems against a range of criteria:

• Data protection compliance: Ensuring personal data is processed lawfully, transparently, and for legitimate purposes.
• Impact assessments: Mandating Data Protection Impact Assessments (DPIAs) for high-risk AI applications to identify and mitigate potential risks.
• Accountability: Requiring organizations to demonstrate adherence to data protection standards through robust documentation and auditable practices.

For Costa Rica, which is observing European developments closely (Murillo, 2024), the EDPB’s approach provides a strong foundation for incorporating data protection principles into AI auditing. Adapting such a checklist could help address privacy concerns and establish trust in emerging AI systems.

International Panel on AI Auditing Standards (IPIE)

The International Panel on AI Auditing Standards focuses on defining essential characteristics for AI audits to ensure their reliability and global applicability. This initiative underscores the need for comprehensive, standardized auditing practices that can transcend jurisdictional differences and create a universal baseline for evaluating AI systems (IPIE, 2024).

Key recommendations from this panel include:

• Transparency and explainability: Audits should prioritize understanding how AI models make decisions, ensuring they are interpretable by both technical and non-technical stakeholders.
• Fairness metrics: The panel emphasizes the need for audits to measure and address biases in AI systems, particularly those impacting marginalized groups.
• Global applicability: Creating audit standards that are adaptable across diverse legal and cultural environments, enabling international collaboration.

COBIT Framework

The COBIT (Control Objectives for Information and Related Technologies) framework, although traditionally applied to IT governance, offers valuable insights for AI auditing. COBIT emphasizes the integration of governance and management practices to ensure that technology aligns with organizational goals and stakeholder needs (ISACA, 2018).

In the context of AI auditing, COBIT’s principles can be adapted to focus on:

• Risk management: Identifying and mitigating risks associated with AI systems, particularly in areas like bias, privacy, and security.
• Performance monitoring: Establishing KPIs to evaluate the effectiveness and ethical performance of AI systems.
• Stakeholder alignment: Ensuring that AI systems address the needs of all stakeholders, from end-users to regulatory bodies.

Costa Rica could leverage COBIT’s adaptable structure to create a governance model for AI systems that integrates technical, ethical, and societal considerations.

Checklist for AI Audit

As Costa Rica and other Latin American countries navigate the early stages of developing and implementing AI regulations, establishing robust frameworks for AI auditing is critical. This checklist by Sajid (2023) provides a comprehensive approach to assessing AI systems, ensuring fairness, accountability, and security, for countries in LATAM that are still in the process of shaping their AI policies, this can serve as a foundational guide for evaluating AI systems, promoting transparency, and safeguarding against potential risks.

• Data Sources
  The first step in auditing AI systems is identifying and validating the data sources. Auditors focus on ensuring the quality of the data and assessing whether the company has proper rights to use it.
• Cross Validation
  One key element auditors check is the appropriate cross-validation of the model. Validation data should be separate from training data, and the validation methods must ensure that the model can generalize well across different scenarios.
• Secure Hosting
  When AI systems use personal data, it is vital to verify that hosting or cloud services comply with necessary information security standards, such as those outlined by OWASP (Open Web Application Security Project).
• Explainable AI
  Explainable AI refers to making the system’s decisions interpretable and understandable. Auditors verify that the models are explainable using methods like LIME and SHAP.
• Model Outputs
  Auditors prioritize fairness in the model's outputs, ensuring consistency in results even when factors like gender, race, or religion are altered. They also evaluate the quality of the predictions by using appropriate scoring methods.
• Social Feedback
  AI auditing is an ongoing process. Once the AI system is deployed, auditors should monitor its social impact. The AI system, along with its risk strategy, should be adjusted and re-audited based on feedback, usage, and the resulting consequences, whether positive or negative.

Challenges

• Limited access to reliable data: Accessing accurate, up-to-date information on AI development and auditing practices in Costa Rica is a significant barrier to informed decision-making.
• Legislative delays: The absence of approved AI-specific laws hinders the implementation of systematic audits.
• Resource constraints: Developing technical expertise and securing resources for AI audits remain challenging for a small country with limited AI infrastructure.
• Public awareness and trust: Lack of awareness about AI risks and the benefits of auditing undermines public trust and adoption of auditing practices.
• Adapting international models: While global frameworks provide valuable insights, adapting them to Costa Rica’s socio-economic and regulatory context requires substantial effort.

Conclusions

• Costa Rica requires a tailored AI auditing framework to address the unique regulatory, ethical, and technical challenges posed by AI.
• Adopting elements from established frameworks like the EDPB and COBIT can provide a foundation for effective auditing mechanisms.
• By advancing AI governance and auditing, Costa Rica can position itself as a leader in Latin America, setting a precedent for ethical and responsible AI use.
• AI audits must prioritize fairness, data protection, and explainability to foster public trust and compliance with international standards.
• Stakeholders, including public institutions and private entities, need to collaborate to develop resources and expertise for AI auditing.

Bibliography

Campos, A. (2024, 28 octubre). Costa Rica lanza la Estrategia Nacional de Inteligencia Artificial, la primera en Centroamérica. El Sol de Occidente. https://elsoldeoccidente.com/nacionales/costa-rica-lanza-la-estrategia-nacional-de-inteligencia-artificial-la-primera-en-centroamerica/ 

Erick Murillo (October 2024).  Esto dice el MICITT sobre proyectos de ley para regular la inteligencia artificial. CRHoy.  https://www.crhoy.com/tecnologia/esto-dice-el-micitt-sobre-proyectos-de-ley-para-regular-inteligencia-artificial/ 

Galdon. (2024). AI Auditing Checklist for AI Auditing . https://www.edpb.europa.eu/our-work-tools/our-documents/support-pool-experts-projects/ai-auditing_en 

International Panel on the Information Environment. IPIE. (2024). Global Approaches to Auditing Artificial Intelligence: A Literature Review. https://www.ipie.info/research/global-approaches-to-auditing-artificial-intelligence 

Índice Latinoamericano de Inteligencia Artificial. ILIA. (2024). Índice Latinoamericano de Inteligencia Artificial. https://indicelatam.cl/ 

International Panel on the Information Environment. IPIE. (2024). Recommendations for a Global AI Auditing Framework: Summary of Standards and Features. https://www.ipie.info/research/sfp2024-2 

ISACA. (2018). AUDITING ARTIFICIAL INTELLIGENCE. https://ec.europa.eu/futurium/en/system/files/ged/auditing-artificial-intelligence.pdf 

Massey (2024). IDC’s Worldwide AI and Generative AI Spending – Industry Outlook. IDC Blog. https://blogs.idc.com/2024/08/21/idcs-worldwide-ai-and-generative-ai-spending-industry-outlook/ 

Mökander (2023). Auditing of AI: Legal, Ethical and Technical Approaches. DISO 2, 49.  https://doi.org/10.1007/s44206-023-00074-y

Instituto de auditores internos del Ecuador (2024). Nuevo marco de auditoría de inteligencia Artificial – Instituto de Auditores Internos de Ecuador. https://iaiecuador.org/2024/11/nuevo-marco-de-auditoria-de-inteligencia-artificial/ 

Sajid, H. (2023). How to perform an AI Audit in 2023. Unite.AI. https://www.unite.ai/how-to-perform-an-ai-audit-in-2023/