BIAS AND DISCRIMINATION IN ARTIFICIAL INTELLIGENCE SYSTEMS, CAUSES AND LEGAL RESPONSES IN THE EUROPEAN UNION

1. What is artificial intelligence?[Up]

In the current legal and technological landscape, it is essential to turn to authoritative sources in order to understand the scope, nature and definition of artificial intelligence. One of the main reference sources in this context comes from legal reports, such as those prepared by the European Union Agency for Fundamental Rights (‍2020), they define AI as systems that exhibit intelligent behaviour, capable of analysing their environment and taking action —with a certain degree of autonomy— in order to achieve specific objectives. This definition highlights the autonomous nature of AI systems, highlighting their ability to make decisions based on information they receive from the environment, making them key tools for efficiently addressing complex problems. The European Commission’s High-Level Expert Group on Artificial Intelligence (‍2019: 4) underscores the transformative potential of artificial intelligence, considering it not only an end in itself but also acts as a means to promote human prosperity. In this sense, AI is presented as a driver of progress, innovation, and improvement of both individual and social well-being, in addition to contributing to the common good through its implementation in diverse areas such as health, education, security, and the economy.

In turn, the UNESCO Recommendation on the Ethics of Artificial Intelligence (2021: 4) offers a complementary perspective, understanding AI as “systems capable of processing data and information in a manner that emulates intelligent behaviour”, which involves key aspects such as reasoning, learning, perception, prediction, planning, and control. This definition broadens the horizon of AI, emphasizing its practical applications and its ability to perform tasks that, at one time, could only be performed by humans.

In more technical terms, AI can be defined as the ability of a computer system or machine to imitate human cognitive functions, such as reasoning, problem-solving, language comprehension, and learning from data (‍Russel and Norvig, 2016: 19-‍23). Stuart Russell and Peter Norvig, two of the most renowned experts in the field, approach AI from a perspective centred on “intelligent agents”. In their work Artificial Intelligence: A Modern Approach, they describe AI as the study of agents that receive perceptions from the environment and execute actions, emphasizing the importance of the rationality of these systems when making decisions based on the data they receive (‍Russel and Norvig, 2016: 54-‍57). This definition highlights the ability of AI systems to act autonomously, in order to achieve a specific goal, without direct human intervention.

This section does not seek to provide a single definition of artificial intelligence, as such a definition must necessarily evolve over time as technological advances continue to shape the field. AI, being a dynamic area of ​​research, requires a flexible approach that allows for the incorporation of new perspectives and characteristics that emerge as new developments emerge (‍UNESCO, 2021:4).

2. How does it work?[Up]

Understanding artificial intelligence requires not only a conceptual definition but also a clear explanation of its functioning. In its contemporary form, AI operates primarily through algorithms designed to process vast amounts of data, identify patterns, make decisions, and, in many cases, learn from experience (‍Gentile, 2024: 53-‍57). One of its core elements is machine learning, a branch of AI that enables computers and machines to imitate human learning processes in order to perform tasks autonomously and improve their performance and accuracy as they are exposed to more data (‍Baughman et al., 2021). Within this field, deep learning represents a more advanced approach that relies on multilayered artificial neural networks inspired by the human brain; these networks progressively adjust their internal connections while processing information, which allows them to undertake highly complex tasks such as image recognition or language translation (‍Gentile, 2024: 53-‍57). Underpinning both of these techniques are algorithms, logical and mathematical sets of instructions that allow AI systems to analyze data, generate predictions, classify information, group similar content, and refine their outcomes (European Union Agency for Fundamental Rights, 2019: 1-‍2). Closely connected to these mechanisms is the field of big data, which encompasses the technological developments in data collection, storage, analysis, and application, and is generally characterized by unprecedented increases in the volume, velocity, and variety of data produced, often but not exclusively from online sources (‍European Union Agency for Fundamental Rights, 2018: 2-‍3). Finally, an essential component of modern AI is natural language processing (NLP), a specialized branch that allows machines to understand and generate human language, thus enabling applications such as virtual assistants, chatbots, or automatic translation (‍Jones, 2025). Taken together, these elements demonstrate that artificial intelligence is a dynamic and evolving technology capable of autonomously analyzing data, learning from experience, and making decisions, with the potential to perform increasingly complex tasks that positively impact individuals and society.

1. Data used by the AI System[Up]

When AI learns from data, it may contain pre-existing biases that are reflected in its decisions. These occur when the data used to train the AI is incomplete, incorrect, or not representative. This causes the system to make incorrect or discriminatory decisions (ibid.: 102). Low data quality can result from human intervention in its generation or classification. In the workplace, this is especially common, as seen in employment through digital platforms. (‍Aragüez, 2022: 8-‍10) A clear example of this can be seen in platforms like Uber, where customer-sourced ratings can embed users’ biases —often along racial or gender lines— in ways that algorithmic systems then enact, including through automatic driver deactivation at thresholds around a 4.6 average ^[1] (‍Levy et al., 2016). These systems amplify biased ratings, so that non-White drivers receive approximately 80 % lower ratings and earn 28 % less than their White counterparts (‍Teng et al., 2023). Similarly, field studies in Seattle and Boston show that Black and female riders face higher cancellation rates and longer wait times, highlighting racial and gender biases in service provision —NBER, National Bureau of Economic Research study (‍Ge et al., 2016: 3-‍18).

2. Biases derived from the design of the AI System itself[Up]

Biases can arise not only from how the algorithm processes data but also from how the system is presented and interacted with by users. These design-related biases include both the internal functioning of the algorithm and the structure of the user interface. For instance, interface design can influence how users’ access or interpret information. A clear example is a voice assistant that recognizes male voices more accurately than female voices, which can lead to reduced accessibility for women and a user experience shaped by gender bias. Moreover, the default use of female-sounding voices in many virtual assistants has been shown to reinforce harmful gender stereotypes —portraying women as submissive and overly polite— highlighted by the UNESCO and EQUAL skills Coalition Report called I’d Blush If I Could (‍West et al: 2019: 9 and 87-‍88).^[2] In addition, an experimental study by Mahmood and Huang (‍2024: 4-‍11)^[3] found that participants perceived feminine-voiced assistants as warmer and were more likely to interrupt them during errors, whereas gender-ambiguous voices reduced these biases.

3. Human biases[Up]

These biases do not stem directly from the data or the AI’s design but rather arise from human behaviors that influence how the system is used or interpreted. Specifically, biases derived from human actions occur when users, developers, or supervisors of AI unintentionally or deliberately introduce prejudice into its functioning; for example, developers with ideological biases may program a content moderation AI on social media to disproportionately censor certain topics (‍Lousada, 2024: 112). Additionally, there are risks of invisibility, where AI systems reinforce social inequalities by giving greater visibility to certain groups while marginalizing others. For instance, a search algorithm that prioritizes information in English may render knowledge or cultural perspectives in other languages invisible. As Helm et al. (‍2023: 8-‍13) demonstrate, “techno-linguistic bias” in AI language technologies systematically excludes underresourced languages and perpetuates epistemic injustice by limiting representation of marginalized linguistic communities.

1. Direct and indirect discrimination in the context of AI[Up]

The right to non-discrimination constitutes a fundamental guarantee in the application of artificial intelligence. As highlighted in the previous chapter, the improper use of data and algorithms risks generating decisions tainted by discriminatory biases, with direct consequences for individuals’ daily lives. Within the European Union, both direct and indirect discrimination are expressly prohibited under Union law, particularly through anti-discrimination directives such as Directive 2000/43/EC on racial equality and Directive 2000/78/EC on equality in employment.

Direct discrimination, as clarified in A. v. B. (Judgment of 11 November 2019, Case C-177/18, ECLI:EU:C:2020:26), occurs when a person is treated less favourably than another in a comparable situation due to a protected characteristic, such as sex, race or ethnic origin, religion or belief, disability, age, or sexual orientation. This form of discrimination is explicit and objective, requiring no proof of discriminatory intent, but only evidence of unjustified unequal treatment.

Indirect discrimination, by contrast, as defined in Bartsch v. Bosch (Judgment of 16 April 2008, Case C-427/06, ECLI:EU:C:2008:517), arises where an apparently neutral provision, criterion, or practice places members of a protected group at a particular disadvantage in comparison with others. While such measures may be justified, this is only permissible if they pursue a legitimate objective and employ means that are both appropriate and necessary in accordance with the principle of proportionality. The Court of Justice of the European Union has repeatedly affirmed that equal treatment requires not merely the absence of discriminatory intent but also the elimination of discriminatory effects in practice, as made explicit in CHEZ Razpredelenie Bulgaria (Case C-83/14, Judgment of 16 July 2015).

Beyond case law, this jurisprudence is embedded in a broader legal framework. Article 19 TFEU empowers the Council, acting unanimously on a proposal from the Commission, to adopt measures addressing discrimination based on sex, racial or ethnic origin, religion or belief, disability, age, or sexual orientation, forming the foundation for sectoral legislation in fields such as employment, education, healthcare, and access to goods and services. This is complemented by the Charter of Fundamental Rights, which, since acquiring binding force with the Lisbon Treaty, has reinforced the Union’s commitment to equality and non-discrimination. In particular, Article 21 of the Charter prohibits discrimination on an even wider range of grounds —including social origin, genetic features, language, political opinion, membership of a national minority, and property— and applies both to EU institutions and to Member States when implementing Union law. At the regional level, the European Convention on Human Rights adds further protection: Article 14 ensures that Convention rights are secured without discrimination, while Protocol No. 12 establishes a general prohibition of discrimination, thereby extending the protective scope beyond Convention rights.

Notwithstanding this robust legal framework, emerging technological developments —particularly those linked to algorithmic decision-making— pose significant challenges for enforcement. Among these, indirect discrimination proves especially difficult to address in the context of AI, where biases often manifest in systemic and opaque ways. As ‍Zuiderveen Borgesius (2019: 409–411) has noted, algorithmic bias typically operates subtly, complicating its detection, proof, and legal redress through traditional mechanisms. This reality underscores the pressing need to reconsider whether the classic categories of direct and indirect discrimination remain sufficient in the digital age, and to explore possible adaptations of EU anti-discrimination law so as to safeguard fundamental rights in environments increasingly shaped by automated decision-making.

To better illustrate the legal challenges discussed above, particularly the distinction between direct and indirect discrimination in algorithmic decision-making, the following two examples demonstrate how artificial intelligence systems may generate discriminatory outcomes —either explicitly or through seemingly neutral criteria that disproportionately affect certain protected groups.

An example of direct discrimination in the use of AI: A recruitment company uses an AI system to filter resumes and select candidates for interviews. The algorithm is designed to analyze keywords in the resumes and rate candidates based on their work experience and specific skills. Due to its programming or the historical data used to train it, the AI system systematically favors male candidates in traditionally male-dominated sectors like engineering, while disqualifying or ranking women with the same qualifications lower. This happens because the system interprets that men are more likely to be suitable for the role, as historically more men have held similar positions. This is a clear case of direct discrimination based on sex, as the AI system treats women less favorably simply because they are women, without any objective or proportional justification for the differential treatment.

An example of indirect discrimination: An AI credit scoring system is used to evaluate the creditworthiness of loan applicants, based on various factors such as credit history, employment, and geographic location. The algorithm prioritizes applicants who live in high-income areas, without considering other factors that may affect an individual’s ability to repay. Although the AI system does not explicitly discriminate based on race, it ends up disadvantaging racial minorities who live in low-income areas. People from lower-income communities are less likely to receive high scores in the system, even though their economic situation may not fairly reflect their repayment ability. This happens because geographic location and historical income in the area disproportionately influence the evaluation. This is an example of indirect discrimination based on racial origin or ethnic group. While people are not directly discriminated against because of their race, the AI system has a disproportionate impact on a protected group (people from low-income communities, often minorities), creating an indirect disadvantage.

2. Is it necessary to redefine the concept of discrimination in the digital age?[Up]

To protect citizens from potential discrimination caused by artificial intelligence, it is essential to ensure that the legal framework in place is capable of safeguarding users’ fundamental rights. A central point of debate, as highlighted by Lousada (‍2024: 117–120), concerns whether algorithmic discrimination can be effectively addressed through existing legal concepts —such as direct or indirect discrimination— or whether the emergence of AI-driven decision-making requires the development of a new legal category: algorithmic discrimination. This discussion has sparked divergent positions in the academic literature. While some authors argue that the current anti-discrimination laws are sufficient, provided they are properly interpreted and enforced, others contend that the unique features of algorithmic systems —such as opacity, lack of intentionality, and statistical correlation— demand a new normative framework. The following section will explore these differing viewpoints in greater depth, presenting the main arguments from both sides of the debate.

One strand of the academic literature holds that existing legal frameworks are sufficiently robust to address the challenges posed by algorithmic discrimination, without the need to develop a new legal category. Preciado (‍2021: 6) is one of the most prominent voices in this camp, arguing that algorithmic discrimination represents old forms of discrimination under new appearances and that the legal system already contains the necessary tools to respond to them. According to this view, the use of artificial intelligence does not alter the fundamental nature of discrimination, but merely the context in which it arises. Therefore, the focus should be on the correct application and interpretation of existing norms —particularly those related to indirect discrimination— rather than the creation of entirely new legal definitions.

Similarly, Malgieri (‍2019; 3-‍4 and 22-‍23) takes a nuanced position that emphasizes the adaptability of current EU legal instruments, such as the General Data Protection Regulation^[4] (GDPR) and the EU Charter of Fundamental Rights, to situations involving algorithmic decision-making. While acknowledging the complexity and opacity of automated systems, Malgieri contends that the legal notion of discrimination is technically and conceptually capable of encompassing algorithmic harms, provided that existing doctrines are applied flexibly and with sensitivity to the technological context. He argues that the lack of transparency or explainability in AI does not necessarily warrant a new concept, but rather calls for enhanced procedural safeguards and accountability mechanisms within the current legal framework.

Together, these authors suggest that the perceived novelty of algorithmic discrimination may be overstated, and that the core legal principles governing equality and non-discrimination remain applicable —even in the face of technological transformation. In this sense, the challenge lies less in redefining discrimination, and more in ensuring effective enforcement and evidentiary adaptation within the structures already in place.

In contrast, several scholars argue that the existing legal framework is insufficient to adequately address the unique characteristics of algorithmic discrimination, and therefore advocate for the development of a new legal concept tailored to these technological realities. Among the most influential voices in this debate are Sandra Wachter, Brent Mittelstadt, and Chris Russell, who in their work Why fairness cannot be automated: Bridging the gap between EU Non-Discrimination Law and AI argue that algorithmic discrimination is fundamentally “more abstract, subtle, and intangible” than traditional human forms of discrimination.

According to these authors, algorithms can produce discriminatory outcomes through complex statistical patterns and correlations that do not always align with legally protected categories (‍Wachter et al., 2020: 11-‍12). As a result, discriminatory effects may occur without clear intent, and victims may not even be aware they have been disadvantaged. This structural opacity and lack of transparency create significant challenges for detection, investigation, and legal redress, making the classical legal frameworks —based on identifiable actors, direct causality, and intent— less effective in these cases.

To address these gaps, Wachter and colleagues propose incorporating statistical fairness metrics into the legal analysis of discrimination. One such proposal is the use of Conditional Demographic Disparity (CDD) —a quantitative tool designed to evaluate whether and to what extent certain groups are disadvantaged by algorithmic systems. Introduced in the later sections of their study (‍Wachter et al., 2020: 54, 57, 62), CDD allows courts and regulators to assess disparities in treatment between demographic groups, while still respecting the contextual and flexible nature of EU non-discrimination law. This approach does not seek to replace legal reasoning with statistical metrics, but rather to provide an evidentiary bridge between technical outputs and legal concepts.

Their broader conclusion is that without adapting the legal framework to accommodate these new forms of discrimination, many harms caused by AI systems will remain legally invisible. Thus, the authors call for a rethinking of the current legal doctrine to develop a concept of algorithmic discrimination that is compatible with the structure and logic of automated decision-making.

Occupying a more nuanced position in the ongoing debate, Lousada (‍2024: 117-‍120) suggests that while the current anti-discrimination framework —particularly the concepts of direct and indirect discrimination— can be extended to cover algorithmic harms, significant adjustments and interpretative refinements are needed to ensure its effectiveness in the digital context. Rather than proposing a wholly new legal category, Lousada advocates for a series of reforms that bridge the gap between traditional legal norms and the challenges posed by AI-driven decision-making.

First, Lousada emphasizes the importance of adapting existing legal concepts. Although direct and indirect discrimination remain valid categories, their interpretation must evolve to account for discrimination originating not from human intent, but from biased datasets, opaque algorithms, or discriminatory interface designs. This requires courts and regulators to develop a more technically informed understanding of how discrimination may manifest in algorithmic systems.

Second, Lousada highlights the critical role of transparency. The current difficulty in accessing algorithmic logic, source code, or data flows severely impedes the ability of victims to prove discrimination. In this regard, he proposes that non-compliance with transparency obligations —particularly those set out in the forthcoming European Artificial Intelligence Act (AI Act)— could itself constitute a form of discriminatory harm, insofar as it obstructs legal remedies.

Third, the author proposes greater flexibility in evidentiary standards, particularly for cases of indirect discrimination. This could include modifying the traditional test to accommodate the specificities of algorithmic decision-making, or even establishing a quasi-objective presumption of discrimination when a protected group is systematically disadvantaged by an automated system.

Fourth, Lousada argues for a harmonized interpretation between anti-discrimination law and the AI Act. In this view, the preventive mechanisms and risk classifications contained in the AI Act should be seen as complementary to existing non-discrimination norms, creating a layered model of protection that combines ex ante regulation with ex post legal enforcement.

Finally, while he does not definitively call for a new legal concept, Lousada remains open to the potential development of new categories, such as discrimination by omission or refined exceptions based on good faith. These innovations, he suggests, could be explored in future jurisprudence or legislative reform, particularly as courts begin to grapple with the practical consequences of AI deployment in sensitive areas such as employment, credit, and public services.

In sum, the academic debate reflects a tension between continuity and innovation: while some scholars argue that existing anti-discrimination law is sufficiently adaptable to capture algorithmic harms, others maintain that the distinctive features of AI demand a new legal category of algorithmic discrimination. A middle ground, exemplified by Lousada, points to the need for interpretative refinements and procedural reforms within the current framework, in close coordination with the forthcoming AI Act. Ultimately, the challenge lies in ensuring that fundamental rights remain effectively protected in the face of rapidly evolving technological realities.

1. Traceability and explainability of algorithms[Up]

In the digital age, algorithmic systems and artificial intelligence play an increasingly significant role in decision-making that affects people’s lives. In this context, the principles of transparency and explainability have become essential for ensuring the protection of human rights. In this new chapter, we will analyze them in depth, observing why they are so crucial for protecting users’ human rights and also for enabling the challenge of decisions made through AI.

Traceability: Traceability refers to the ability to chronologically interrelate uniquely identifiable entities in a way that is verifiable. (‍National Institute of Standards and Technology, 2023: 15-‍16). Therefore, traceability refers to the ability to understand how an automated system works, what data it uses, and under which criteria it makes decisions. It involves the ability to track the entire process that the AI system follows to reach a decision. This includes knowing what data was used, how the model was trained, which algorithms were applied, and what changes were made to the system (‍Baatout, 2023). As UNESCO points out in its Recommendation on the Ethics of Artificial Intelligence (‍2021:12), “people should be clearly and understandably informed when a decision affecting them has been made by an automated system”.

When this traceability is absent, we face what is known as “algorithmic opacity.” As Felzmann et al. (‍2020: 3339) point out, this opacity involves a lack of transparency caused by the existence of a “black box,” i.e., systems that lack explanatory capability and whose internal logic is inaccessible or incomprehensible to users and, in many cases, even to the developers themselves. It is not only that we do not know what decision is made, but that we are completely unaware of how and why.

This lack of traceability creates worrying situations, especially when algorithms make decisions that directly affect people’s rights and opportunities. For example, when a bank loan, social aid, or job position is denied, and the algorithm does not allow us to understand the criteria used or offer the possibility to challenge the decision, it becomes a form of an unappealable verdict, potentially based on partial, erroneous, or biased data, with no right for the affected person to an explanation (‍Blázquez, 2022: 268).

Beyond the individual case, the greater risk, according to Blázquez Ruiz (ibid.), is the consolidation of a hyper-technician society, where even the elites do not fully understand the functioning of the tools that govern key decisions. This scenario fosters structural inequality and a loss of democratic and citizen control over the systems that increasingly mediate our daily lives (ibid.: 268). For all these reasons, traceability is not just a technical issue, but an essential element to guarantee accountability, justice, and respect for fundamental rights in digital environments

Explainability: means ensuring that algorithmic decisions, as well as any data driving those decisions, can be explained to end-users and other stakeholders in non-technical terms. This includes information that allows an explanation of the general functioning of the system, the specific use of data within the system, and individual decisions taken by the system. (‍Felzmann et al., 2020: 3347)

In the current context of the rapid development of artificial intelligence, explainability emerges as a fundamental condition to ensure the ethical and just application of automated systems, particularly those based on deep learning. This requirement has been emphasized by authors such as López de Mántaras (‍2021), who argues that algorithms must incorporate explanatory modules that allow understanding and interpreting the decisions they generate. Explainability should not be conceived as an optional addition, but as an indispensable feature of any intelligent system, especially when its decisions directly impact individuals’ social or legal rights (‍Turri, 2022).

This concern has also been reflected at the institutional level. The European Commission, through the High-Level Expert Group on Artificial Intelligence, included explainability as one of the key Principles in its guidelines for trustworthy AI (‍2019: 9). According to this body, AI systems must be explainable so that users can understand and, if necessary, challenge the results, especially when these affect fundamental rights. In a constantly evolving social and technological context, where artificial intelligence systems are actively involved in decisions that affect fundamental rights, explainability emerges as an irreplaceable principle for the effective protection of human rights.

Traceability and explainability are not simply technical features, but essential guarantees of transparency, accountability, and respect for fundamental rights. Both academic voices and institutional initiatives —such as the European Commission’s Guidelines for trustworthy AI (2019)— stress their indispensable role in enabling individuals to understand, question, and contest algorithmic decisions. Advancing toward more transparent and explainable AI is therefore not only a technical challenge, but also a democratic imperative: without it, artificial intelligence risks consolidating opaque power structures that undermine justice, equality, and citizen control.

2. Preventive approaches: methods to avoid bias before implementing AI[Up]

Before an artificial intelligence system is put into operation, it is essential to establish mechanisms that ensure its impartiality and respect for fundamental rights. Preventing discriminatory biases is not only a technical issue but also an ethical and legal one, especially when AI is applied in sensitive areas such as employment, education, healthcare, or justice. Many of the a priori prevention mechanisms aimed at avoiding discrimination produced by AI are found in the European Union Agency for Fundamental Rights (FRA) document on data quality and artificial intelligence (2019: 2-‍4, 13).

To prevent discrimination in AI systems, the first crucial aspect, according to the FRA, is data quality. While large volumes of data may give the impression of accuracy, this is misleading if the quality of the data is not assessed. Statistical accuracy and the ability to reflect the real world depend not only on the amount of data but also on its quality. A key element of data quality is whether the data are fit for purpose —meaning they are suitable for the specific task they are intended to support. Evaluating whether data serve their intended purpose helps to identify errors and potential risks in data-driven systems.

According to the FRA, data must be of high quality to prevent discriminatory bias in AI systems. This means data should be accurate, complete, consistent, up-to-date, valid, non-duplicated, available, and traceable to a reliable source (provenance). These characteristics ensure that the data are fit for their intended purpose and reduce the risk of bias during model training and decision-making.

Another important concept introduced is measurement error, which refers to the extent to which the data accurately represent what they are supposed to measure. For example: Is income a reliable indicator of creditworthiness? What defines a “good employee”? Being punctual? Performing well? Understanding these dimensions of data quality and measurement is essential to identify and mitigate potential sources of bias, especially in training data used in machine learning models.

Another key aspect of data quality, according to the FRA, is labeling. Human labeling of outcome data (e.g., assigning categories to images) is essential for evaluating dataset bias and algorithm performance, but it can introduce measurement errors if quality control is lacking. Poor or biased labeling leads to unfair and inaccurate models.

The FRA warns of the risk of representation error, which arises when training data fail to reflect the target population or phenomenon; for instance, using nationality as a proxy for origin may exclude naturalized individuals and distort results. Such shortcomings directly undermine two core pillars of data quality: reliability, understood as consistency across time and conditions, and validity, referring to whether data genuinely measure the phenomenon they are intended to capture.

In machine learning, high data volume alone does not guarantee quality. If data lack validity or representativeness, bias is not eliminated but automated at scale. The FRA warns that large datasets can amplify systematic errors, creating a false sense of accuracy. Understanding and addressing these risks is essential for responsible AI development.

Another highly relevant aspect that deserves to be highlighted in this work is the HUDERIA (Human Rights, Democracy and the Rule of Law Impact Assessment) document (Conseil of Europe, 2024), which establishes a methodology for evaluating the risks and impacts of artificial intelligence (AI) systems from the perspective of human rights, democracy, and the rule of law. This document was adopted by the Artificial Intelligence Committee (CAI) of the Council of Europe in November 2024 and builds on the previous work of CAHAI and the Alan Turing Institute.

HUDERIA establishes a preventive risk mechanism designed to anticipate and manage the potential negative effects of AI use, structured in four phases. The first phase, COBRA, helps understand the system’s context and map potential risks throughout its lifecycle. The second phase, SEP, ensures that the voices of those who could be affected are heard by engaging stakeholders to understand risks from their perspective. The third stage, RIA, conducts a detailed assessment of risks and impacts, focusing particularly on the protection of democracy, the rule of law, and human rights. Finally, the MP phase defines strategies to mitigate identified risks and monitor the system’s operation throughout its lifecycle, functioning as a structured risk audit. We will not delve further into this topic, though its importance and relevance are undeniable. For more detailed information, the following link provides an expanded explanation.^[5]

To prevent discrimination caused by AI, the report of European Union Agency for Fundamental Rights called Getting the Future Right: Artificial Intelligent and Fundamental Rights (‍2020: 7, 10, 64, 66, 72, 96) emphasizes the importance of conducting impact assessments that go beyond technical performance and include a fundamental rights perspective. While traditional assessments often focus on factors like accuracy or cybersecurity, this is not enough if the social and human effects of AI systems are ignored. Impact assessments serve as preventive tools, helping organizations and governments identify potential risks to rights such as privacy and non-discrimination. Although some legal frameworks (like the General Data Protection Regulation) already require such evaluations, they often overlook whether AI systems may infringe on human rights. We cannot forget another fundamental tool, the HUDERIA document, which we consider contains excellent recommendations and practical guidelines to ensure the safety in the operation of an AI system. For AI to be used safely and ethically, it is essential to ensure effective oversight and to establish clear responsibilities for those who design, deploy, and regulate these systems.

3. Corrective approaches: ex post solutions to mitigate negative impacts[Up]

Even with strong preventive measures, harmful impacts from AI may still occur. This section addresses corrective (ex post) approaches, focusing on how to respond when fundamental rights are violated. It highlights the need for effective systems of accountability, reparation, and access to legal or administrative remedies, along with the importance of retroactive transparency and the roles of both public and private actors in ensuring restitution.

According to Arenós (‍2025), specific legal frameworks are needed to assign responsibility for harm caused by AI systems, allowing victims access to judicial or administrative remedies. The creation of civil liability insurance for AI-related damages is also proposed to encourage safer and more ethical AI practices.

The European Union Agency for Fundamental Rights (‍FRA, 2020: 13, 67 and 81) highlights that effective access to justice for AI decisions is essential to protect fundamental rights. Citizens must be informed about AI use, understand its workings, and know how to file complaints. Access to remedies is not only a right in itself, as stated in Article 47 of the Charter of Fundamental Rights of the EU (2000), but also crucial to exercising other rights, like the right to a fair trial. States must ensure access to judicial or alternative routes for reparation.

Challenges such as AI’s technical complexity and intellectual property constraints hinder transparency. Some entities adopt simpler AI models in sensitive areas and promote accountability and citizen oversight initiatives.

Furthermore, Mantelero and Esposito (‍2021:55-56) stress the importance of conducting human rights impact assessments before and after AI deployment. These assessments, performed by independent bodies, help identify and mitigate risks while ensuring compliance with ethical and legal standards.

In 2025, Meritxell Borràs, president of the Catalan Data Protection Authority (APDCAT), introduced an innovative methodology for supervising and humanizing artificial intelligence, aimed at assessing its impact on fundamental rights such as privacy, equality, and non-discrimination (‍Autoridad Catalana de Protección de Datos, 2025a, ‍2025b). The proposal is based on a risk matrix that evaluates both the probability of rights violations occurring and the severity of their potential consequences, depending on the characteristics of the AI system and its context of use. This approach is designed to help identify high-risk applications —such as those used in justice, security, or healthcare— before harm occurs, allowing public and private decision-makers to proactively adapt systems to prevent violations. Beyond identifying risks, the methodology promotes the adoption of corrective measures through continuous review and auditing protocols, ensuring compliance with ethical and legal standards. Overall, this initiative represents a significant step toward establishing more transparent and robust regulatory frameworks that guarantee human oversight and protect citizens’ rights in the development and use of AI.

Additionally, ‍Aritz Obregón Fernández and Guillermo Lazcoz Moratinos (2021: 2-‍3, and 20-‍27) highlight the critical need for meaningful human control (MHC) over high-risk AI systems, especially in sensitive areas like justice, public safety, and health. MHC requires human supervision at all stages of AI decision-making to protect fundamental rights, ensuring that AI outputs can be reviewed, validated, or modified when necessary. They argue this principle must be explicitly integrated into the design, development, and deployment of AI, supported by both International Law and European Union law.

The authors discuss how EU regulations, such as the 2021 proposed AI Regulation, adopt a risk-based approach demanding human oversight for high-risk AI but call for clearer emphasis on MHC due to AI’s disruptive potential. They also note practical challenges, particularly the need to train and certify human operators with technical skills to effectively supervise complex AI systems and intervene appropriately.

1. Case C-311/18, Schrems II[Up]

The C-311/18 case, known as Schrems II, represents a landmark in the evolution of European data protection law and the regulation of international transfers of personal information (‍Mildebrath, 2020). The dispute arose from the transfer of personal data by Facebook Ireland Ltd. to its parent company in the United States, carried out under the Privacy Shield framework and the Standard Contractual Clauses (SCCs) approved by the European Commission (EPRS, 2020). Austrian lawyer and privacy activist Maximilian Schrems challenged the legality of these transfers before the Irish Data Protection Commission, arguing that U.S. law did not provide adequate safeguards against the extensive access of personal data by agencies such as the NSA, thereby breaching the standards of the GDPR and the rights enshrined in the Charter of Fundamental Rights of the European Union, particularly those concerning privacy (Art. 7), data protection (Art. 8), and effective judicial remedy (Art. 47).

In its judgment of 16 July 2020, the Court of Justice of the European Union invalidated the Privacy Shield, concluding that the U.S. regime failed to provide a level of protection essentially equivalent to that required by the EU and lacked effective judicial redress mechanisms. By contrast, the SCCs were upheld as valid, but subject to strict conditions: their use requires that the recipient country can ensure —either through domestic legislation or supplementary measures adopted by the parties— a level of protection essentially equivalent to the GDPR. This entails a prior assessment of the legal framework and suspension of transfers when adequate safeguards cannot be guaranteed.

Although Schrems II does not directly address artificial intelligence, its reasoning on data protection and fundamental rights is highly relevant for the governance of automated decision-making. The judgment highlights the necessity of transparency, oversight, and effective redress —principles that, as noted by Costello (‍2020), are central to ensuring that international data transfers comply with EU standards of privacy and security. While Costello’s analysis focuses on cross-border data flows, these same safeguards can be extrapolated to the context of opaque or “black box” algorithms capable of making decisions with significant social and legal consequences. Moreover, the principle of equivalence in international transfers established by the judgment implies that any personal data processing —including that conducted through AI— must meet standards comparable to those of the GDPR, thereby preventing bias or discriminatory practices. In this way, Schrems II reinforces accountability, responsibility, and the protection of fundamental rights, providing a normative framework which, although originally conceived for international data transfers, is fully applicable to the oversight of AI’s impact in digital society (Judgment of 16 July 2020).

2. Case C-154/21, UI v. Österreichische Post AG[Up]

The ruling of the Court of Justice of the European Union in UI v. Österreichische Post AG (C-154/21) addresses the creation of automated political profiles without the data subject’s consent and clarifies the scope of compensation for non-material damages under Article 82 of the GDPR (Court of Justice of the European Union, 2023). The Court established that compensation is not granted automatically in the event of a breach of the Regulation; rather, the claimant must demonstrate (i) the existence of a violation, (ii) actual damage —whether material or non-material— and (iii) a causal link between the violation and the harm suffered. Importantly, the Court emphasized that there is no minimum severity threshold for non-material damages, thereby strengthening the protection of fundamental rights and acknowledging the subjective impact of algorithmic processing (Court of Justice of the European Union, 2023).

From the perspective of artificial intelligence, the case is highly relevant. The processing carried out by Österreichische Post AG relied on automated statistical methods to infer estimated political profiles from demographic and socioeconomic characteristics. Although not formally classified as AI, these practices replicate core dynamics of algorithmic systems, including opacity in decision-making and the prediction of behavior without meaningful human oversight. The judgment illustrates that such automated processing can inflict non-material harm —such as distress, loss of trust, or a sense of surveillance— that is nonetheless legally protected under the GDPR (‍Rooney & Coll, 2024; ‍Brams et al., 2023).

According to Brams et al. (‍2023), organizations must provide data subjects with information on the specific recipients of personal data when requested, except in cases where this is impossible or when the request is manifestly unfounded or excessive. This obligation reinforces transparency and accountability in automated processing: individuals should not only be informed that their data is used in profiling but also be able to identify who receives, or may receive, that information.

From both a legal and ethical standpoint, the ruling strengthens the connection between fundamental rights and the operation of automated systems. It requires organizations to document data flows, ensure human oversight, and recognize that even intangible algorithmic harms can give rise to legal liability. This sets an important precedent for balancing technological innovation with individual rights and transparency in AI governance across Europe.

Although neither Schrems II (C-311/18) nor UI v. Österreichische Post AG (C-154/21) directly concern autonomous AI systems, both establish principles highly relevant for their regulation. Schrems II underscores the need for equivalence, transparency, and effective redress in data processing, while UI v. Österreichische Post AG highlights the risks of automated profiling and confirms liability for even non-material harms. Together, they reinforce a normative framework centered on accountability, human oversight, and the protection of fundamental rights, offering valuable guidance for addressing discrimination and other challenges posed by AI in Europe.