This document is an excerpt from the EUR-Lex website
Document 52021AR2682
Opinion of the European Committee of the Regions — European approach to artificial intelligence — Artificial Intelligence Act (revised opinion)
Opinion of the European Committee of the Regions — European approach to artificial intelligence — Artificial Intelligence Act (revised opinion)
Opinion of the European Committee of the Regions — European approach to artificial intelligence — Artificial Intelligence Act (revised opinion)
COR 2021/02682
OJ C 97, 28.2.2022, p. 60–85
(BG, ES, CS, DA, DE, ET, EL, EN, FR, GA, HR, IT, LV, LT, HU, MT, NL, PL, PT, RO, SK, SL, FI, SV)
28.2.2022 |
EN |
Official Journal of the European Union |
C 97/60 |
Opinion of the European Committee of the Regions — European approach to artificial intelligence — Artificial Intelligence Act
(revised opinion)
(2022/C 97/12)
|
I. RECOMMENDATIONS FOR AMENDMENTS
Amendment 1
Recital 1
Text proposed by the Commission |
CoR amendment |
The purpose of this Regulation is to improve the functioning of the internal market by laying down a uniform legal framework in particular for the development, marketing and use of artificial intelligence in conformity with Union values. This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, safety and fundamental rights, and it ensures the free movement of AI-based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation. |
The purpose of this Regulation is to improve the functioning of the internal market and protect the fundamental rights of citizens by laying down a uniform legal framework in particular for the development, marketing and use of artificial intelligence in conformity with Union values. This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, safety and fundamental rights, and it ensures the free movement of AI-based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation. |
Reason
The reference to fundamental rights is intended to emphasise the link with the EU Charter of Fundamental Rights.
Amendment 2
New recital after Recital 6
Text proposed by the Commission |
CoR amendment |
|
Defining AI systems is an ongoing process that should take into account the context in which AI operates, keep pace with societal developments in this field and not lose sight of the link between the ecosystem of excellence and the ecosystem of trust. |
Reason
Developments in AI and technology require an adaptive and evolving approach. This recital reflects the fact that the definition of AI should move with the times and state of development of AI systems and applications.
Amendment 3
Recital 20
Text proposed by the Commission |
CoR amendment |
In order to ensure that those systems are used in a responsible and proportionate manner, it is also important to establish that, in each of those three exhaustively listed and narrowly defined situations, certain elements should be taken into account, in particular as regards the nature of the situation giving rise to the request and the consequences of the use for the rights and freedoms of all persons concerned and the safeguards and conditions provided for with the use. In addition, the use of ‘real-time’ remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement should be subject to appropriate limits in time and space, having regard in particular to the evidence or indications regarding the threats, the victims or perpetrator. The reference database of persons should be appropriate for each use case in each of the three situations mentioned above. |
In order to ensure that those systems are used in a responsible and proportionate manner, it is also important to establish that, in each of those three exhaustively listed and narrowly defined situations, certain elements should be taken into account, in particular as regards the nature of the situation giving rise to the request and the consequences of the use for the rights and freedoms of all persons concerned and the safeguards and conditions provided for with the use. Consultation of the relevant local and regional authorities should take place prior to the exceptional use of those systems. In addition, the use of ‘real-time’ remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement should be subject to stringent limits in time and space, having regard in particular to the evidence or indications regarding the threats, the victims or perpetrator. The reference database of persons should be appropriate for each use case in each of the three situations mentioned above. |
Reason
‘Real-time’ remote biometric identification systems should not be used lightly.
Amendment 4
Recital 21
Text proposed by the Commission |
CoR amendment |
Each use of a ‘real-time’ remote biometric identification system in publicly accessible spaces for the purpose of law enforcement should be subject to an express and specific authorisation by a judicial authority or by an independent administrative authority of a Member State. Such authorisation should in principle be obtained prior to the use, except in duly justified situations of urgency, that is, situations where the need to use the systems in question is such as to make it effectively and objectively impossible to obtain an authorisation before commencing the use. In such situations of urgency , the use should be restricted to the absolute minimum necessary and be subject to appropriate safeguards and conditions, as determined in national law and specified in the context of each individual urgent use case by the law enforcement authority itself. In addition, the law enforcement authority should in such situations seek to obtain an authorisation as soon as possible, whilst providing the reasons for not having been able to request it earlier. |
Each use of a ‘real-time’ remote biometric identification system in publicly accessible spaces for the purpose of law enforcement should be subject to an express and specific authorisation by a judicial authority or by an independent administrative authority of a Member State. Such authorisation should be obtained prior to the use, except in duly justified situations of urgency, that is, situations where the need to use the systems in question is such as to make it effectively and objectively impossible to obtain an authorisation before commencing the use. In any case , the use should be restricted to the absolute minimum necessary and be subject to appropriate safeguards and conditions, as determined in national law. In addition, the law enforcement authority should immediately inform the relevant local and regional authorities and seek to obtain an authorisation from the competent authorities . |
Reason
The political and administrative responsibility for managing and monitoring public spaces lies with local and regional authorities. They should therefore be duly involved in the deployment of such systems in public spaces. In urgent situations where it would not be reasonable to await prior consultation, the local or regional authority concerned should be immediately informed about the deployment of biometric systems in the public space.
Amendment 5
Recital 39
Text proposed by the Commission |
CoR amendment |
AI systems used in migration, asylum and border control management affect people who are often in particularly vulnerable position and who are dependent on the outcome of the actions of the competent public authorities. The accuracy, non-discriminatory nature and transparency of the AI systems used in those contexts are therefore particularly important to guarantee the respect of the fundamental rights of the affected persons, notably their rights to free movement, non-discrimination, protection of private life and personal data, international protection and good administration. It is therefore appropriate to classify as high-risk AI systems intended to be used by the competent public authorities charged with tasks in the fields of migration, asylum and border control management as polygraphs and similar tools or to detect the emotional state of a natural person; for assessing certain risks posed by natural persons entering the territory of a Member State or applying for visa or asylum; for verifying the authenticity of the relevant documents of natural persons; for assisting competent public authorities for the examination of applications for asylum, visa and residence permits and associated complaints with regard to the objective to establish the eligibility of the natural persons applying for a status. AI systems in the area of migration, asylum and border control management covered by this Regulation should comply with the relevant procedural requirements set by the Directive 2013/32/EU of the European Parliament and of the Council, the Regulation (EC) No 810/2009 of the European Parliament and of the Council and other relevant legislation. |
AI systems used in migration, asylum and border control management affect people who are often in particularly vulnerable position and who are dependent on the outcome of the actions of the competent public authorities. The accuracy, non-discriminatory nature and transparency of the AI systems used in those contexts are therefore particularly important to guarantee the respect of the fundamental rights of the affected persons, notably their rights to free movement, non-discrimination, protection of private life and personal data, international protection and good administration. It is therefore necessary to classify as high-risk AI systems intended to be used by the competent public authorities charged with tasks in the fields of migration, asylum and border control management as polygraphs and similar tools or to detect the emotional state of a natural person; for assessing certain risks posed by natural persons entering the territory of a Member State or applying for visa or asylum; for verifying the authenticity of the relevant documents of natural persons; for assisting competent public authorities for the examination of applications for asylum, visa and residence permits and associated complaints with regard to the objective to establish the eligibility of the natural persons applying for a status. AI systems in the area of migration, asylum and border control management covered by this Regulation should comply with the relevant procedural requirements set by the Directive 2013/32/EU of the European Parliament and of the Council, the Regulation (EC) No 810/2009 of the European Parliament and of the Council and other relevant legislation. |
Reason
This adaptation expresses the need to subject the AI systems concerned to the more robust regime for high-risk AI systems.
Amendment 6
Recital 43
Text proposed by the Commission |
CoR amendment |
Requirements should apply to high-risk AI systems as regards the quality of data sets used, technical documentation and record-keeping, transparency and the provision of information to users, human oversight, and robustness, accuracy and cybersecurity. Those requirements are necessary to effectively mitigate the risks for health, safety and fundamental rights, as applicable in the light of the intended purpose of the system, and no other less trade restrictive measures are reasonably available, thus avoiding unjustified restrictions to trade. |
Requirements should apply to high-risk AI systems as regards the quality of data sets used, technical documentation and record-keeping, transparency and the provision of information to users, human oversight, and robustness, accuracy and cybersecurity. Those requirements are necessary to effectively mitigate the risks for health, safety , data security, consumer rights and fundamental rights, as applicable in the light of the purpose of the system, and no other less trade restrictive measures are reasonably available, thus avoiding unjustified restrictions to trade. Natural persons or groups of persons affected by high-risk AI systems placed on the EU market or otherwise put into service should be informed in an appropriate, easily accessible and comprehensible manner, and have access to explicit, readily accessible and publicly available information explaining that they are subject to such systems. |
Reason
The transparency and information requirements applicable to providers and users should be extended to the persons or groups of persons potentially affected by the use of high-risk AI systems, as listed in Annex III to the Regulation. In a comprehensible manner also means ‘in a language that is comprehensible and accessible to the user, including oral-auditory and manual visual languages’.
Amendment 7
New recital after Recital 44
Text proposed by the Commission |
CoR amendment |
|
AI system providers shall refrain from any measure promoting unjustified discrimination based on sex, origin, religion or belief, disability, age, sexual orientation, or discrimination on any other grounds, in their quality management system. |
Reason
Unlawful discrimination originates in human action. AI system providers should refrain from any measures in their quality system that could promote discrimination.
Amendment 8
Recital 47
Text proposed by the Commission |
CoR amendment |
To address the opacity that may make certain AI systems incomprehensible to or too complex for natural persons, a certain degree of transparency should be required for high-risk AI systems. Users should be able to interpret the system output and use it appropriately. High-risk AI systems should therefore be accompanied by relevant documentation and instructions of use and include concise and clear information, including in relation to possible risks to fundamental rights and discrimination, where appropriate. |
To address the opacity that may make certain AI systems incomprehensible to or too complex for natural persons or public authorities at all levels of governance , a high level of transparency should be required for high-risk AI systems. Users should be able to interpret the system output and use it appropriately. High-risk AI systems should therefore be accompanied by relevant documentation and instructions of use and include concise and clear information, including in relation to possible risks to fundamental rights and discrimination, where appropriate. |
Reason
The accountability of those who design high-risk AI systems is weakened by the use of the words ‘a certain degree of transparency’.
Amendment 9
Recital 48
Text proposed by the Commission |
CoR amendment |
High-risk AI systems should be designed and developed in such a way that natural persons can oversee their functioning. For this purpose, appropriate human oversight measures should be identified by the provider of the system before its placing on the market or putting into service. In particular, where appropriate, such measures should guarantee that the system is subject to in-built operational constraints that cannot be overridden by the system itself and is responsive to the human operator, and that the natural persons to whom human oversight has been assigned have the necessary competence, training and authority to carry out that role. |
High-risk AI systems should be designed and developed in such a way that natural persons and public authorities at all levels of governance can oversee their functioning. For this purpose, appropriate human oversight measures should be identified by the provider of the system before its placing on the market or putting into service. In particular, where appropriate, such measures should guarantee that the system is subject to in-built operational constraints that cannot be overridden by the system itself and is responsive to the human operator, and that the natural persons to whom human oversight has been assigned have the necessary competence, training and authority to carry out that role. |
Reason
Self-explanatory.
Amendment 10
Recital 67
Text proposed by the Commission |
CoR amendment |
High-risk AI systems should bear the CE marking to indicate their conformity with this Regulation so that they can move freely within the internal market. Member States should not create unjustified obstacles to the placing on the market or putting into service of high-risk AI systems that comply with the requirements laid down in this Regulation and bear the CE marking. |
High-risk AI systems should bear the CE marking to indicate their conformity with this Regulation so that they can move freely within the internal market. Member States should not create obstacles to the placing on the market or putting into service of high-risk AI systems that comply with the requirements laid down in this Regulation and bear the CE marking. Member States shall have the power to regulate high-risk AI practices and AI systems solely on the basis of overriding and duly justified public and national security interests . |
Reason
While Member States should not obstruct the application of the Regulation, they should retain the right to regulate high-risk AI systems if public and national security interests are at stake.
Amendment 11
Recital 70
Text proposed by the Commission |
CoR amendment |
Certain AI systems intended to interact with natural persons or to generate content may pose specific risks of impersonation or deception irrespective of whether they qualify as high-risk or not. In certain circumstances, t he use of these systems should therefore be subject to specific transparency obligations without prejudice to the requirements and obligations for high-risk AI systems. In particular, natural persons should be notified that they are interacting with an AI system , unless this is obvious from the circumstances and the context of use . Moreover, natural persons should be notified when they are exposed to an emotion recognition system or a biometric categorisation system. Such information and notifications should be provided in accessible formats for persons with disabilities. Further, users, who use an AI system to generate or manipulate image, audio or video content that appreciably resembles existing persons, places or events and would falsely appear to a person to be authentic, should disclose that the content has been artificially created or manipulated by labelling the artificial intelligence output accordingly and disclosing its artificial origin. |
Certain AI systems intended to interact with natural persons or to generate content may pose specific risks of impersonation or deception irrespective of whether they qualify as high-risk or not. The use of these systems should therefore be subject to specific transparency obligations without prejudice to the requirements and obligations for high-risk AI systems. In particular, natural persons should be systematically notified that they are interacting with an AI system. Moreover, natural persons should be notified when they are exposed to an emotion recognition system or a biometric categorisation system. Such information and notifications should be provided in accessible formats for persons with disabilities. Further, users, who use an AI system to generate or manipulate image, audio or video content that appreciably resembles existing persons, places or events and would falsely appear to a person to be authentic, should disclose that the content has been artificially created or manipulated by labelling the artificial intelligence output accordingly and disclosing its artificial origin. |
Reason
No exceptions should be made to the transparency and notification requirement when natural persons interact with AI systems.
Amendment 12
Recital 76
Text proposed by the Commission |
CoR amendment |
In order to facilitate a smooth, effective and harmonised implementation of this Regulation a European Artificial Intelligence Board should be established. The Board should be responsible for a number of advisory tasks, including issuing opinions, recommendations, advice or guidance on matters related to the implementation of this Regulation, including on technical specifications or existing standards regarding the requirements established in this Regulation and providing advice to and assisting the Commission on specific questions related to artificial intelligence. |
In order to facilitate a smooth, effective and harmonised implementation of this Regulation a European Artificial Intelligence Board should be established. The Board should be responsible for a number of advisory tasks, including issuing opinions, recommendations, advice or guidance on matters related to the implementation of this Regulation, including on technical specifications or existing standards regarding the requirements established in this Regulation and providing advice to and assisting the Commission on specific questions related to artificial intelligence. The members of the European Artificial Intelligence Board should reflect the interests of European society. The Board should be gender-balanced. |
Reason
The European AI Board should properly reflect the broad interests of European society. These interests include human rights, climate and the energy-efficient use of AI systems, safety, social inclusion, health, etc. Gender balance is a precondition for diversity in issuing opinions, drafting guidelines, etc.
Amendment 13
Recital 77
Text proposed by the Commission |
CoR amendment |
Member States hold a key role in the application and enforcement of this Regulation. In this respect, each Member State should designate one or more national competent authorities for the purpose of supervising the application and implementation of this Regulation. In order to increase organisation efficiency on the side of Member States and to set an official point of contact vis-à-vis the public and other counterparts at Member State and Union levels, in each Member State one national authority should be designated as national supervisory authority. |
Member States hold a key role in the application and enforcement of this Regulation. In this respect, each Member State should designate one or more national competent authorities for the purpose of supervising the application and implementation of this Regulation. In order to increase organisation efficiency on the side of Member States and to set an official point of contact vis-à-vis the public and other counterparts at Member State and Union levels, in each Member State one national authority should be designated as national supervisory authority. Local and regional authorities shall be entrusted with supervisory or enforcement tasks where deemed appropriate by the Member State. |
Reason
In order to ensure the feasibility of the Regulation and its supervisory and enforcement framework, the Member State should be empowered to entrust, where necessary and where possible, local and regional authorities with carrying out supervisory or enforcement tasks.
Amendment 14
Recital 79
Text proposed by the Commission |
CoR amendment |
In order to ensure an appropriate and effective enforcement of the requirements and obligations set out by this Regulation, which is Union harmonisation legislation, the system of market surveillance and compliance of products established by Regulation (EU) 2019/1020 should apply in its entirety. Where necessary for their mandate, national public authorities or bodies, which supervise the application of Union law protecting fundamental rights, including equality bodies, should also have access to any documentation created under this Regulation. |
In order to ensure an appropriate and effective enforcement of the requirements and obligations set out by this Regulation, which is Union harmonisation legislation, the system of market surveillance and compliance of products established by Regulation (EU) 2019/1020 should apply in its entirety. Where necessary for their mandate, national public authorities and, where applicable, local or regional authorities, or bodies, which supervise the application of Union law protecting fundamental rights, including equality bodies, should also have access to any documentation created under this Regulation. |
Reason
This amendment takes into account the varying governance structures in the EU Member States.
Amendment 15
Recital 83
Text proposed by the Commission |
CoR amendment |
In order to ensure trustful and constructive cooperation of competent authorities on Union and national level, all parties involved in the application of this Regulation should respect the confidentiality of information and data obtained in carrying out their tasks. |
In order to ensure trustful and constructive cooperation of competent authorities on Union, national , regional and local level, all parties involved in the application of this Regulation should respect the confidentiality of information and data obtained in carrying out their tasks. |
Reason
This amendment takes into account the varying governance structures in the EU Member States.
Amendment 16
TITLE I, Article 3(1) — Definitions
Text proposed by the Commission |
CoR amendment |
‘artificial intelligence system’ (AI system) means software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with; |
‘artificial intelligence system’ (AI system) means software that is developed with one or more of the techniques and approaches listed (non-exhaustively) in Annex I , combined with social practices, identity and culture, and that can, for a given set of human-defined objectives, by observing its environment through collecting data, interpreting the collected structured or unstructured data, managing knowledge, or processing the information derived from these data, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with; |
Reason
An AI system consists of a combination of technical elements that link data, algorithms and processing power to social practices, society, identity and culture. The definition of such a dynamic socio-technical aggregate should therefore be future-proof and regularly updated to accurately reflect AI’s ever-growing societal impact, while identifying fast changing AI-related challenges and opportunities, including the link between knowledge management and AI. In this context, an algorithm developed by another algorithm should also be subject to the Regulation.
Amendment 17
Article 5(1)
Text proposed by the Commission |
CoR amendment |
||||||||||
The following artificial intelligence practices shall be prohibited: |
The following artificial intelligence practices shall be prohibited: |
||||||||||
|
|
||||||||||
|
|
||||||||||
|
|
||||||||||
|
|
||||||||||
|
|
Reason
Subliminal techniques can, in general, undermine freedom, human rights and thus the functioning of the democratic rule of law. At the same time, artificial intelligence may undermine consumer rights. The purpose of the additions is to make this clear.
Concerning social classification by public authorities or on their behalf, it should be banned if carried out for general purposes, given the dangers resulting from such practices, as explained in Recital 17. The generation or collection of data for specific purposes should only be allowed with human oversight and provided that it does not violate the right to dignity and non-discrimination and the values of equality and justice.
Amendment 18
Article 5(4)
Text proposed by the Commission |
CoR amendment |
A Member State may decide to provide for the possibility to fully or partially authorise the use of ‘real-time’ remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement within the limits and under the conditions listed in paragraphs 1, point (d), 2 and 3. That Member State shall lay down in its national law the necessary detailed rules for the request, issuance and exercise of, as well as supervision relating to, the authorisations referred to in paragraph 3. Those rules shall also specify in respect of which of the objectives listed in paragraph 1, point (d), including which of the criminal offences referred to in point (iii) thereof, the competent authorities may be authorised to use those systems for the purpose of law enforcement. |
A Member State may decide to provide for the possibility to fully or partially authorise the use of ‘real-time’ remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement within the limits and under the conditions listed in paragraphs 1, point (d), 2 and 3. That Member State shall lay down in its national law the necessary detailed rules for the request, issuance and exercise of, as well as supervision relating to, the authorisations referred to in paragraph 3. Those rules shall also specify in respect of which of the objectives listed in paragraph 1, point (d), including which of the criminal offences referred to in point (iii) thereof, the competent authorities may be authorised to use those systems for the purpose of law enforcement. Those rules shall lay down the arrangements for informing and consulting the local and regional authorities concerned. This consultation shall take place prior to the exceptional use of these systems in public spaces. In urgent situations where it would not be reasonable to await prior consultation, the local or regional authority concerned shall be immediately informed of the deployment of the relevant AI practice. |
Reason
The political and administrative responsibility for managing and monitoring public spaces lies with local and regional authorities. They should therefore be put in a position to provide input prior to the deployment of such AI practices and be duly informed of the exceptional use of AI systems for the purpose of law enforcement.
In urgent situations where it would not be reasonable to await prior consultation, the local or regional authority concerned shall be immediately informed.
Amendment 19
Article 13
Text proposed by the Commission |
CoR amendment |
||||||||||
Article 13 Transparency and provision of information to users |
Article 13a Transparency and provision of information to users |
||||||||||
1. High-risk AI systems shall be designed and developed in such a way to ensure that their operation is sufficiently transparent to enable users to interpret the system’s output and use it appropriately. An appropriate type and degree of transparency shall be ensured, with a view to achieving compliance with the relevant obligations of the user and of the provider set out in Chapter 3 of this Title. |
1. High-risk AI systems shall be designed and developed in such a way to ensure that their operation is sufficiently transparent to enable users to interpret the system’s output and use it appropriately. An appropriate type and degree of transparency and a comprehensible explanation shall be ensured, with a view to achieving compliance with the relevant obligations of the user and of the provider set out in Chapter 3 of this Title. The explanation shall be provided at least in the language of the country where the AI system is deployed. |
||||||||||
2. High-risk AI systems shall be accompanied by instructions for use in an appropriate digital format or otherwise that include concise, complete, correct and clear information that is relevant, accessible and comprehensible to users. |
2. High-risk AI systems shall be accompanied by publicly accessible and comprehensible instructions for use in an appropriate digital format or otherwise that include concise, complete, correct and clear information that is relevant, accessible and comprehensible to users. |
||||||||||
3. The information referred to in paragraph 2 shall specify: |
3. The information referred to in paragraph 2 shall specify: |
||||||||||
|
|
||||||||||
|
|
||||||||||
|
|
||||||||||
|
|
||||||||||
|
|
||||||||||
|
|
||||||||||
|
|
||||||||||
|
|
||||||||||
|
|
||||||||||
|
|
||||||||||
|
13b Transparency and information to persons affected Persons or groups of persons for whom a high-risk AI system is intended to be used shall be informed in an appropriate, easily accessible and comprehensible manner, and have access to explicit, readily accessible and publicly available information of such use. |
Reason
In order to strengthen the ecosystem of trust, the instructions for use for high-risk AI systems should be made publicly accessible. These instructions should be written in a language of the country where the AI system is deployed, and should be comprehensible to the reader.
With a view to the transparency and ‘explainability’ of algorithms, it should be possible to explain with which parameters the model has been tuned and what measures have been taken to prevent overfitting and underfitting.
Article 13b lays down an obligation for transparency and information vis-à-vis persons who interact with the AI system or who could be affected by the AI system.
Amendment 20
Article 14(4)
Text proposed by the Commission |
CoR amendment |
||||
The measures referred to in paragraph 3 shall enable the individuals to whom human oversight is assigned to do the following, as appropriate to the circumstances: |
The measures referred to in paragraph 3 shall enable the individuals to whom human oversight is assigned to do the following, as appropriate to the circumstances: |
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
Reason
There are several forms of bias that may be problematic. Examples include the designer’s or user’s own bias (social bias), bias as to whether the AI system deployed is a suitable solution to the problem (technical bias) and statistical forms of bias.
Amendment 21
Article 14, new paragraph after paragraph 5
Text proposed by the Commission |
CoR amendment |
|
Any decision taken by AI systems as referred to in Annex III(5) (a) and (b) shall be subject to human intervention and shall be based on a diligent decision-making process. Human involvement in these decisions shall be guaranteed. |
Reason
Article 14 deals only with human oversight of high-risk AI systems. For government decisions, it is important to stress that human intervention, contact and due process are to be ensured.
Amendment 22
Article 17(1) new subsections after m
Text proposed by the Commission |
CoR amendment |
||||
Providers of high-risk AI systems shall put a quality management system in place that ensures compliance with this Regulation. That system shall be documented in a systematic and orderly manner in the form of written policies, procedures and instructions, and shall include at least the following aspects: |
Providers of high-risk AI systems shall put a quality management system in place that ensures compliance with this Regulation. That system shall be documented in a systematic and orderly manner in the form of written policies, procedures and instructions, and shall include at least the following aspects: |
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
Reason
This addition stresses that inclusiveness and the fight against unjustified discrimination should be important elements of the quality system.
The system should comply with the ethical values that a user of the AI system wishes to establish for that system or that the provider may reasonably expect to be established for a high-risk AI system. The provider must be able to explain how it has taken this into account.
Amendment 23
Article 19(1)
Text proposed by the Commission |
CoR amendment |
Providers of high-risk AI systems shall ensure that their systems undergo the relevant conformity assessment procedure in accordance with Article 43, prior to their placing on the market or putting into service. Where the compliance of the AI systems with the requirements set out in Chapter 2 of this Title has been demonstrated following that conformity assessment, the providers shall draw up an EU declaration of conformity in accordance with Article 48 and affix the CE marking of conformity in accordance with Article 49. |
Providers of high-risk AI systems shall ensure that their systems undergo the relevant conformity assessment procedure in accordance with Article 43, prior to their placing on the market or putting into service. Where the compliance of the AI systems with the requirements set out in Chapter 2 of this Title has been demonstrated following that conformity assessment, the providers shall draw up an EU declaration of conformity in accordance with Article 48 and affix the CE marking of conformity in accordance with Article 49. The providers of high-risk AI systems shall publish the EU declaration of conformity and a summary of the conformity assessment in a publicly accessible place. |
Reason
In order to strengthen the ecosystem of trust in AI systems, providers of high-risk AI systems must be open. The public should therefore be able to check that conformity assessment has been properly established in accordance with the rules of the Regulation.
Amendment 24
Article 29, new paragraph after paragraph 6
Text proposed by the Commission |
CoR amendment |
|
Users of high-risk AI systems shall be responsible for making an ethical assessment before putting the system into use. They shall be able to explain the possible impact of the deployment of the technology on people and society. They shall specify their intended purpose in deploying the AI system, the overarching values, how those values have been weighted and whether or not they have been implemented in the system. They shall assess the actual impact of the system on people and society throughout the life cycle of the AI system. |
Reason
Ethics is a broad concept. There are many ways of practising technology ethics, both in terms of theoretical underpinnings and practical methodologies, tools and design values. Values are matters that are considered important by certain (groups of) people; they may be more specific or more conceptual. It is important to keep open the range of possible moral values to be implemented and to continue evaluating the life cycle of the AI system.
Amendment 25
Article 52(1)
Text proposed by the Commission |
CoR amendment |
Providers shall ensure that AI systems intended to interact with natural persons are designed and developed in such a way that natural persons are informed that they are interacting with an AI system, unless this is obvious from the circumstances and the context of use . This obligation shall not apply to AI systems authorised by law to detect, prevent, investigate and prosecute criminal offences, unless those systems are available for the public to report a criminal offence. |
Providers shall ensure that AI systems intended to interact with natural persons are designed and developed in such a way that natural persons are informed that they are interacting with an AI system. This obligation shall not apply to AI systems authorised by law to detect, prevent, investigate and prosecute criminal offences, unless those systems are available for the public to report a criminal offence. The scope of options and legal position of natural persons interacting with AI systems shall not be limited by this interaction. |
Reason
Where technological tools are used as a medium for interaction with natural persons, there may be a risk of limiting the choices made by natural persons interacting with them. Natural persons should always be duly informed whenever they encounter AI systems and this should not be subject to interpretation of a given situation. Their rights should be guaranteed at all times in interactions with AI systems.
Amendment 26
Article 57(1)
Text proposed by the Commission |
CoR amendment |
The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, and the European Data Protection Supervisor. Other national authorities may be invited to the meetings, where the issues discussed are of relevance for them. |
The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, and the European Data Protection Supervisor. Other national , regional and local authorities may be invited to the meetings, where the issues discussed are of relevance for them. |
Reason
Local and regional authorities should be able to participate in the monitoring of AI systems and to report on their implementation on the ground.
Amendment 27
Article 58
Text proposed by the Commission |
CoR amendment |
||||||||||||||||
When providing advice and assistance to the Commission in the context of Article 56(2), the Board shall in particular: |
When providing advice and assistance to the Commission in the context of Article 56(2), the Board shall in particular: |
||||||||||||||||
|
|
||||||||||||||||
|
|
||||||||||||||||
|
|
||||||||||||||||
|
|
Reason
Local and regional authorities are closest to local residents and economies. They should explicitly feature when it comes to sharing their knowledge.
Amendment 28
Article 59(1)
Text proposed by the Commission |
CoR amendment |
National competent authorities shall be established or designated by each Member State for the purpose of ensuring the application and implementation of this Regulation. National competent authorities shall be organised so as to safeguard the objectivity and impartiality of their activities and tasks. |
National competent authorities shall be established or designated by each Member State for the purpose of ensuring the application and implementation of this Regulation. National competent authorities shall be organised so as to safeguard the objectivity and impartiality of their activities and tasks. Local and regional authorities shall be empowered to carry out supervisory or enforcement tasks where deemed appropriate by the Member State. |
Reason
In order to ensure the feasibility of the Regulation and the given monitoring and enforcement framework, the Member State should be able to entrust, where necessary and where possible, local and regional authorities with supervisory or enforcement tasks. In this context, local and regional authorities must receive support and training in order to be fully empowered to carry out supervisory or enforcement tasks.
Amendment 29
Article 69(3)
Text proposed by the Commission |
CoR amendment |
Codes of conduct may be drawn up by individual providers of AI systems or by organisations representing them or by both, including with the involvement of users and any interested stakeholders and their representative organisations. Codes of conduct may cover one or more AI systems taking into account the similarity of the intended purpose of the relevant systems. |
Codes of conduct may be drawn up by national, local or regional authorities, by individual providers of AI systems or by organisations representing them or by both, including with the involvement of users and any interested stakeholders and their representative organisations. Codes of conduct may cover one or more AI systems taking into account the similarity of the intended purpose of the relevant systems. |
Reason
National, local and regional authorities should be given the legal power to draw up codes of conduct for the AI systems they develop or use.
Amendment 30
ANNEX I — Artificial intelligence techniques and approaches referred to in Article 3, point 1
Text proposed by the Commission |
CoR amendment |
||||
|
Having regard to the current state of science, AI includes the following techniques and methods: |
||||
|
|
||||
|
|
||||
|
|
Reason
The definition and list of AI techniques should be future-proof. The list of specific techniques and approaches used for the development of AI systems should not be an exhaustive list and it must be clear that it is based on the current scientific state of play.
Amendment 31
Annex III, 1-5
Text proposed by the Commission |
CoR amendment |
||||
High-risk AI systems pursuant to Article 6(2) are the AI systems listed in any of the following areas: |
High-risk AI systems pursuant to Article 6(2) are the AI systems listed in any of the following areas: |
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
||||
|
|
Reason
Telecommunications, water and internet infrastructure are an integral part of critical infrastructure.
The classification of high-risk systems hinges on whether such systems could pose a real risk to citizens. The mere analytical and theoretical assessment of residents’ claims to public services does not entail a high risk. Complementing ‘evaluate’ with ‘decide on’ emphasises that this risk effectively translates into decision-making, particularly for residents.
II. POLICY RECOMMENDATIONS
THE EUROPEAN COMMITTEE OF THE REGIONS
Ecosystem of excellence
1. |
stresses that the Commission’s goal of making the EU a global leader in the responsible and human-centred development of AI can only be achieved if local and regional authorities have a significant role. Local and regional authorities are best placed to help create an environment propitious to boosting investment in AI in the coming years and fostering trust in AI; |
2. |
highlights that besides involving local and regional authorities, it is important to provide support and training in order to enhance their competencies in the field, especially as they may receive supervisory and enforcement roles; |
3. |
notes that EU funding is becoming available for the development of AI, but points to the fragmented approach here, due to the diverse range of programmes, which increases the risk of fragmentation and overlap; |
4. |
calls, therefore, on the Commission to develop and connect strong and pluralistic common data spaces in which societal use-cases can be resolved with the use of public and private data. This also requires alignment with legislative initiatives under the European Data Strategy. |
Ecosystem of trust
5. |
regrets that the proposal for a regulation does not refer to local and regional authorities, despite the fact that the legal framework will apply to both public and private players; |
6. |
notes, to this effect, that AI systems can play an important role in local and regional authorities’ interaction with citizens and service provision. Furthermore, AI systems have the potential, among other things, to improve public-sector efficiency and help local and regional authorities to respond to the adjustments that need to take place at local and regional level in the context of the green and digital transitions. It is therefore important that the experience gained by local and regional authorities is actively used in the ongoing revision of the Regulation; |
7. |
calls for further clarification of the definitions of ‘provider’ and ‘user’, in particular in situations wherein companies, research institutions, public authorities and residents jointly develop and test AI systems in living labs. Due consideration should be given also to citizens or consumers affected by AI-driven decisions of systems employed by professional users; |
8. |
stresses the need for prior consultation of the relevant local and regional authorities where AI systems are to be used for the real-time remote biometric identification of natural persons in publicly accessible spaces for law enforcement purposes; |
9. |
welcomes the European Commission public consultation on the adaptation of civil liability rules to the specific challenges of digital age and artificial intelligence (1) and expects that this will result in an updated framework aimed at ensuring consumer redress for damage caused by AI applications; |
10. |
wonders why AI systems used in democratic processes such as elections are not on the list of high-risk AI systems; |
11. |
calls for high-risk AI systems to be subject to the same transparency and information requirements for natural persons as currently apply to users; |
12. |
highlights the major human rights risks and implications associated with the use of social classification; |
13. |
is highly sceptical here of the two scenarios set out in the Regulation (2) as grounds for determining when a social classification leads to detrimental or unfavourable treatment of individuals or groups of people, as it is extremely difficult to establish the existence of such grounds. In this context, the Committee urges for the clear formulation of strong safeguards in order to ensure that the ban on social classification practices is not circumvented; |
14. |
notes the fact that the recitals of the Regulation address the risks to which individuals are exposed as a result of interacting with high-risk AI systems in the context of, inter alia, education, training, employment, human resource management, access to self-employment and access to and receipt of certain essential private and public services; |
15. |
calls on the Commission to consider in greater depth the high-risk classification of AI systems intended for use by public authorities (3); |
16. |
calls for an authority to provide substantial ex ante advice on the interpretation of provisions in the Regulation, also in relation to the General Data Protection Regulation. This will enhance legal certainty and reduce the costs of designing and implementing AI systems; |
17. |
underlines to this effect the importance of clarity in the formulation of the Regulation, which is instrumental in building an ecosystem of trust and lifting legal uncertainty surrounding the development and use of AI systems. This would avoid misinterpretations of the proposed requirements and minimise the risks of subsequent mismanagement of AI applications, thus maximising the regulation’s effectiveness and credibility of sanctions. At the same time, and in line with the European Commission’s better regulation agenda, early detection and elimination of potential overlaps and/or conflicts with existing rules is of key importance; |
18. |
notes that many local and regional authorities are using the same AI systems for similar tasks. The systems are designed by private companies in the vast majority of cases; |
19. |
notes that the proposal for a regulation does not stand alone when it comes to guaranteeing citizens' rights and that it must be seen in the context of existing legislation. Member States are therefore encouraged to ensure that, on an ongoing basis, they take the necessary administrative measures to enable them to deal with the opportunities and risks posed by the use of AI in the public sector; |
20. |
notes that this means that in conformity assessment, European and national rules are being interpreted by companies and notified bodies, and that this is having an impact on the practices of local and regional authorities using these AI systems. This makes it difficult to determine the extent to which local and regional policy is taken into account in these AI systems. Therefore, the Committee attention to the specific needs of local and regional authorities and to the fact that a ‘one-size-fits-all’ approach may undermine the effectiveness of AI systems in responding to those needs. Besides, the Committee suggests that Member States should be empowered to regulate high-risk AI systems in the face of overriding and justified reasons of public interest; |
21. |
calls, in this regard, for conformity assessments to be transparent and accessible to the public. Moreover, local and regional authorities should also be able to participate in the monitoring of AI systems, report on their implementation on the ground and make a formal contribution to the European Commission's evaluation of the application of the regulation; |
22. |
stresses that the application of the regulatory sandbox requires the right legal, methodological and ethical conditions to be created to enable the development of technology and legislation and the evaluation of legislation. Clear criteria for allowing companies to participate in the regulatory sandbox should be established. To ensure that consumer organisations can enforce the provisions of the Artificial Intelligence Act, the latter must be added to Annex I of the European Directive on representative actions for the protection of the collective interests of consumers ((EU) 2020/1828); |
Information campaigns
23. |
stresses the importance of public campaigns, so that the general public is informed about and familiarised with the existence and usefulness of AI systems as well as potential risks. Further underlines the urgent need for comprehensive information for consumers on Artificial Intelligence / machine-driven decision-making. Asks to this effect that the European Commission provide funding for such campaigns; |
Administrative burden
24. |
expresses its concern about the potential administrative burden of the proposed Regulation. The administrative burden can hinder small and medium-sized enterprises and local and regional authorities in promoting innovation and deploying AI systems (4); |
Proportionality and subsidiarity
25. |
considers that the draft regulation complies with the requirements of the proportionality and subsidiarity principles. The added value of EU action in this field and the appropriateness of the legal bases chosen by the Commission are clear and consistent. The impact assessment included a distinct section on subsidiarity. Moreover, no national parliament issued a reasoned opinion on non-compliance with the principle of subsidiarity by the deadline for submissions, set on 2 September 2021. |
Brussels, 2 December 2021.
The President of the European Committee of the Regions
Apostolos TZITZIKOSTAS
([62]) Council Framework Decision 2002/584/JHA of 13 June 2002 on the European arrest warrant and the surrender procedures between Member States (OJ L 190, 18.7.2002, p. 1).
([62]) Council Framework Decision 2002/584/JHA of 13 June 2002 on the European arrest warrant and the surrender procedures between Member States (OJ L 190, 18.7.2002, p. 1).
(1) https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/12979-Civil-liability-adapting-liability-rules-to-the-digital-age-and-artificial-intelligence/public-consultation_en
(2) Article 5(1)(c)
(3) Annex III(5)(a)
(4) A recent study (Study to Support an Impact Assessment of Regulatory Requirements for Artificial Intelligence in Europe, p. 12), supported by the European Commission, estimated that, on the basis of reasonable assumptions, obtaining certification for an AI system could cost on average between EUR 16 800 and EUR 23 000, roughly equivalent to 10-14 % of development costs.