Model procedure for lawful AI implementation

Process for the legally compliant introduction / use of AI in the company

by Prof. Dr. Thomas Wilmer

The use of AI should be carefully planned; transparency is particularly important, both with regard to the involvement of AI and the quality assurance of the training data and results, as well as the integration of API interfaces. An increased use in the fields of ECM / DMS, ERP and email management can be observed. An expansion to other fields - far beyond text generation functions - is imminent in many companies.

In addition to numerous data protection issues, there are a number of topics to be considered in the area of labour law that cannot be fully illustrated here. The following information will be updated regularly, especially with regard to the AI Act & Co.

A process for the legal integration of AI should be created which, depending on the type of tool and the purpose of use, contains clear milestones on the following points, among others:

1. Definition of the purpose of the use of AI / preliminary legal clarification

a.        Clarification of legally significant preliminary questions:

  1. Is it necessary to provide personally identifiable information in the prompts (data inputs)?

  2. Could the prompts allow conclusions to be drawn about business secrets (patent drafts, etc.)?

  3. Are the AI results also made available to third parties - possibly in an automated way? Can third parties (e.g. customers) also enter prompts?

  4. Does the intended use include high-risk systems according to Art. 5 of the AI Act (read in conjunction with its Annexes II and III)?

  5. Does the AI deployment have an impact on existing workplaces? Could the use of AI be suitable for monitoring the performance of employees?

  6. Are the interfaces to non-local AI IT security certified / KRITIS ready?                           
  7. Is all data processed by the AI system processed in Germany without parent companies of the AI or subcontractors of the AI provider being located in the US or otherwise outside the EU?

  8. If used in the HR sector: Can the system exclude indirect adverse effects pursuant to Art. 3 para. 2 AGG (the German General Equal Treatment Act)?

b.        Clarification dependent on the results

  1. whether the AI allows data leakage to the outside / a local limitation of prompt storage is possible;

  2. whether the AI use can be limited to areas that are not relevant to the Trade Secrets Act;

  3. whether mechanisms can be set up to limit a reaction to misuse by or to the detriment of third parties;

  4. whether the AI complies with the requirements of the AI Regulation;

  5. whether the works council has been involved at an early stage or whether it can be avoided that employees' performance is monitored;

  6. what IT security measures must be taken to prevent cyber security problems;              

  7. what measures must be taken to permit third country transfers of personal data.          

2. Involvement of works council / staff council

Pursuant to section 90(1) of the Works Council Constitution Act (BetrVG) No. 3, the employer must inform the works council in good time about the planning of work procedures and work processes, including the use of AI, and submit the necessary documents; the same applies in the case of operational changes with significant disadvantages for the workforce pursuant to section 111 of the Works Council Constitution Act (BetrVG).

Insofar as AI is used in the creation of guidelines on personnel selection for recruitment, transfers, regrouping and dismissals, this requires the consent of the works council according to section 95a subsection 2 BetrVG; if necessary, the conciliation board must be consulted. If necessary, the works council may consult an expert under section 80 (3) of the Works Council Constitution Act.

Where the use of AI may be suitable for monitoring the performance of employees, the works council must have a say on its introduction under section 87a(1)(6) of the Works Council Constitution Act (BetrVG). The same applies to the regulation of order in the workplace under section 87(1)(1) BetrVG. The works council should therefore be involved in the planning of privacy by design and privacy by default at an early stage and, depending on the purpose, prepare a works agreement. In addition, duties to inform employees and, depending on the size and organisation of the company, the supervisory board / economic committee (§ 106 BetrVG) must be taken into account.

 

3. Data protection

Insofar as prompts may contain personal data and/or the tool collects personal user data (esp. CRM/ERP: customer/supplier data, etc.):

a.        Involvement of data protection officer 

b.        Examination of the requirements of the GDPR, the German Telecommunications Data Protection Act (TTDSG) and the German Federal Data Protection Act (BDSG).

  1. Initiation of data protection impact assessment pursuant to Art. 35 GDPR: Clarification of the legal basis according to Art. 6 GDPR / § 26 German Data Protection Act (BDSG) (insofar as the latter is still current), also in the case of the integration of third parties and the transfer of data to the AI provider.

  2. Checking whether special personal data (Art. 9 (1) GDPR) are present with the consequence of checking special legal basis; checking whether minors' data are affected (parental consent may be required).

  3. Clarification of the legal basis according to Arts. 6 and, if applicable, 9 GDPR / Section 26 BDSG (the German Federal Data Protection Act, if the latter is still current), also in the case of the integration of third parties and the transfer of data to the AI provider.

  4. Clarification of the requirements of Article 5 of the GDPR (principles of data protection compliant processing, including data minimisation, purpose limitation) and Article 25 of the GDPR (privacy by design and privacy by default).

  5. Examination of whether scoring is inadmissible pursuant to Art. 22 of the GDPR.

  6. Clarification of the requirements for technical and organisational measures (Art. 32 GDPR).

  7. Preparation of the information of the affected employees, customers, suppliers, sales partners (coordination with any information required under labour law).

  8. Clarification of whether the technical structure of the tool (IP recording, connection of the software, processing of the entries) complies with data protection (if applicable, TTDSG for web connection).

  9. Conclusion of a controller-to-controller contract (usually not a commissioned processing contract, as the AI provider also pursues its own interests in the processing of the collected/transferred data), alternatively an Order Processing Contract (especially if the outflow of personal data is excluded (see above 1.a.i), alternatively joint processing according to Art. 26 GDPR).

  10. In the case of data transfers abroad, review of data protection-compliant use (depending on the country, adequacy decision (new: USA from 10.07.2023, see www.dataprivacyframework.gov/s/), SCCs etc., keep an eye on the respective status of NOYB lawsuits). 

  11. For individual data protection issues, please refer to the Checklist for the use of LLM-based chatbots of the Hamburg Data Protection Authority dated 13 November 2023, as well as the Checklist for GDPR-compliant AI from the Bavarian Data Protection Authority of 24 January 2024, and the comprehensive Guide on AI & Data Protection organized by the Data Protection Officer of the State of Baden Baden-Württemberg ("ONKIDA").

c.         Documentation of data protection compliance

 

4. IT security, licence management

  1. Involvement of IT security officer / corporate IT, IT suppliers

  2. Requirements regarding TOMs under Art. 32 DSGVO

  3. Conformity with technical requirements of the AI Regulation

  4. Integration into existing IT security concept

  5. Specifications IT-Sig. 2.0 to the extent applicable, exclusion of security incidents

  6. Clarification of the purposes of use with regard to existing software use: Does the use of AI have an effect on other contracts / rights of use? Will the AI use software that has not been licensed for this purpose (check licensing model)?

5. In the case of platform use of the AI system (access by third parties via large platform) and / or use in smart products

  1. Compliance with the requirements of the German Telemedia Act (TMG) and the German Telecommunications Data Protection Act (TTDSG) as well as liability for interference, where applicable the German Network Data Protection Act (NetzDG) / the German Product Liability Act (ProdHG), producer liability in accordance with § 823 Ab.1 BGB (German Civil Code);

  2. Compliance with the new regulations of the Digital Markets Act, the Digital Services Act, the Data Act;

  3. Compliance with transparency obligations;

  4. Clarification of the use of personal end-user data / right of access, consideration of the German Act on the Protection of Business Transactions (GeschGehG);

  5. Fulfilment of reporting system obligations;

  6. Consideration of the requirements of the new rules of the Product Liability Directive and the AI Liability Directive;

  7. Fulfilment of discovery obligations;

  8. Availability of potential evidence.

 

6. Review of the AI provider / AI Act / agreements.

  1. Submission and examination of certifications, conformity assessment procedures, conformity markings, technical documentation;

  2. Compliance with the AI Regulation depends on risk classification, consideration of further regulations depending on intended use (product safety, etc. and high-risk classification), etc:

  3. No violation of prohibition list Art. 5 of the AI Act

  4. Classification test according to Art. 6 of the AI Act / Annexes II and III

  5. Design and transparency obligations Art. 52 of the AI Act, Art. 4 II of the AI Liability Directive

  6. Art. 10 AI Act: Requirements for training, validation and testing datasets. Requirements for training, validation and test data sets. These must be relevant, sufficiently representative and, with regard to the intended purpose, as error-free and complete as possible. They must have the appropriate statistical properties, including, where applicable, in relation to the persons or groups of persons in relation to whom the high-risk AI system is to be used.

  7. Art. 16 AI Act: Obligations of providers in the case of high-risk systems (including quality management and documentation).

  8. Art. 26 AI: Can the obligations of the operators be fulfilled in time?

  9. Agree on training to optimise prompting / results;

  10. Clarify rights to input and output;

  11. Choice of a success-oriented performance description / clear description of the target state / definition of defects (especially compliance with Art. 6 of the Product Liability Directive!) / warranty issues / especially freedom from legal defects of the training data and results / liability;

  12. Safeguarding the freedom of AI results from third party rights;

  13. Clarification of data protection and data security (see 3 and 4 above), clarification of the status of subcontractors, clarification of data protection compliance at the AI provider, initial audit if necessary;

  14. Depending on the purpose of use, special requirements, e.g. compliance with archiving obligations according to the German Tax Code (AO) / German Commercial Code (HGB) when processing tax-relevant data / business letters.

  15. With powerful providers: check strategies for leveraging non-negotiable general terms and conditions in the ordering process (best practices topic).

7. Implementation phase / Start of the system

  1. Training of users: Information on the data protection situation, preservation of confidentiality in prompts, expectation management on the defectiveness of results;

  2. Final preparation of FAQ / user information / ethical guidelines (for sensitive applications) / privacy statements;

  3. Documentation of the existence of the legal basis for data collection, conclusion of company agreement, finalisation and collection of consent forms;

  4. Breathing deeply, Jacobson's progressive muscle relaxation, then: 1 Knoppers, 1 hot drink with milk foam and reusable tube (metal, not a self-dissolving and therefore urgent cellulose recyclate);
  5. Start of the system.

8. Ongoing compliance / constant update.

  1. Quality assurance prompt input, feedback rounds, incident analyses;

  2. Notice and takedown procedures for legal violations when third parties are involved;

  3. Review of compliance for controller-to-controller and controller-to-processor agreements, review of legal developments for foreign data transfers, ongoing audits;

  4. Examination of the development of legal bases, in particular with regard to employee data protection;

  5. Ongoing compliance with the purpose limitation pursuant to Art. 5 I b GDPR;

  6. Recording and implementation of revocations in the case of consent-based data processing in accordance with Art. 6 I a of the GDPR;

  7. Ongoing review of new legal developments / guidance from the supervisory authorities (Data Protection Authorities, the European Data Protection Board, the European Data Protection Supervisor).

Back to homepage ChatGPT, DALL-E-Recht & Co.