In the context of the Legal subgroup of Living-in.EU and in collaboration with DG GROW, two Roundtables on Procurement Clauses of AI have just been concluded.  

Organised by Eurocities in the context of Living-in.EU and led by the Dutch Law firm Pels Rijcken, the exercise aimed at peer review common procurement clauses on AI for local authorities.  

Building on the good practice of the City of Amsterdam and considering the latest development of the European proposal for an Artificial Intelligence Act, the objective of the Roundtables has been to peer review the Procurement clauses developed by Amsterdam and draft a standard for public procurement that can be used by public authorities across the European Union.  

Peer Review Approach 

The Roundtables were held online and included discussions and breakout sessions using pre-defined formats and collaborative tooling. More than 40 experts from legal, industry, technical, procurement, and ethics domains participated in the exercise.

• Recordings and Presentations of Session 1
• Presentations of Session 2 

During the peer review exercise, experts provided valuable feedback to the Draft Procurement Clauses, which are available on the Public Buyers Platform here. 

Feedback 

The two Roundtables focused on Section B, C & D of the Full Version of the Procurement Clauses of AI.  

Section B of the Clauses on the “Essential requirements in relation to AI systems”, mostly focused on Art. 2 – 3- 6 – 7 & 8. Regarding the AI Risk Management System, several feedbacks focused on the importance of clarifying and sharing responsibilities between the contracting bodies and public authorities, ensuring easy access to information, clarifying requirements, methods and frameworks so to ensure a constant estimation and revaluation of risks. In this regard, some experts also mentioned the importance of engaging in discussions with SMEs, before starting procurement discussions, so to anticipate the potential risks and impact on the market of AI systems.  

Another key discussion has occurred on the issue of transparency and explainability. Art.6 and 7 of the current versions of the clauses indicate requirements to ensure transparency and explainability of AI systems. Various experts have provided suggestions to clarify such requirements, for instance specifying criteria on how to ensure transparency also for off-the-shelf AI solutions, indicating more in details the responsibilities of the public authorities and the suppliers to make sure that transparency and explainability is ensured, clarifying situations in which AI systems are purchased from various suppliers and it could be more difficult to identify obligations and responsibilities.  

Section D of the Clauses on “AI Register and audit” mostly focused on art.16-17-18 and 19 which discuss the rights to the Public Organisation Data Sets. On the issue of intellectual property rights, experts have suggested to including safety protections both for the AI systems but also for the datasets used, providing the bases for additional agreements between the contracting authority and suppliers to use the datasets also for other purposes, while ensuring GDPR and confidentiality rules. Experts have also underlined the importance of introducing measures to allow suppliers to enrich datasets after the conclusion of the agreement, introducing more specifities on which data could be used and in which conditions.  

In the case of handing over of datasets, experts have also provided valuable feedbacks, suggesting that AI clauses should also specify the purpose and the conditions in which datasets should be handed over, clarifying the responsibilities of suppliers and contracting authorities to make sure that datasets are free of IP rights.  

Next Steps 

The feedback which has been gathered during the two roundtables will serve to update the Draft procurement clauses of AI. After validation, the AI clauses will be translated into all EU languages, and these will be made publicly available through the Public Buyers Platform and the Living-in.EU website. 

For any further information, you can contact viviana.demonte@eurocities.eu, project officer at Eurocities or Gabriela.Ruseva@eurocities.eu, project coordinator of Living-in.EU at Eurocities.