SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillski-monitoring-konzept

ki-monitoring-konzept

Deployment & operations

View repository

Updated 10-15 20:08

About this skill

/ki-governance:ki-monitoring-konzept

Purpose

The skill creates a monitoring concept for the ongoing monitoring of an AI system after deployment: which metrics, at what frequency, with which thresholds, by whom, with which escalation. Methodologically, the skill follows ISO/IEC 42001 (Ch. 8 Operation, Ch. 9 Performance Evaluation) and NIST AI RMF (MEASURE / MANAGE function). Statutory logging and oversight obligations of the AI Act (Art. 12 record-keeping obligations, Art. 14 human oversight, Art. 72 post-market monitoring) are referenced as points of reference; the assessment of obligations itself belongs in ki-vo-compliance.

Inputs

  • System ID and risk class (from the risk assessment)
  • Model type (classic ML / LLM / hybrid) and training data sources
  • Deployment context (critical / non-critical process; high-risk indications Annex III AI Act)
  • Existing telemetry and logging infrastructure
  • Affected protected interests (personal rights, trade secrets, professional secrecy)

Sub-agent architecture

Researcher provides AI Act anchors (Art. 12, 14, 72), GDPR anchors (Art. 32), standards (ISO/IEC 42001 Ch. 8/9, NIST AI RMF MEASURE/MANAGE, BSI AIC4 requirements on monitoring), BaFin BDAI / MaRisk AT 7.2. Drafter creates the monitoring concept with a KPI table, escalation matrix and audit plan. Reviewer checks for completeness of the metrics, clear thresholds, logging compliance and human-in-the-loop.

Process

1. Monitoring dimensions

Dimension Example metrics Reference
Performance Accuracy, precision, recall, F1, AUC; LLM: task success rate, hallucination rate NIST AI RMF MEASURE 2.3
Data drift Kolmogorov-Smirnov test, Population Stability Index (PSI), embedding drift ISO/IEC 5338, NIST MEASURE 2.4
Concept drift Performance degradation over time, label distribution shift NIST MEASURE 2.4
Fairness / bias Demographic parity, equal opportunity, disparate impact, group-specific error rates NIST MEASURE 2.11; AGG protected categories
Robustness / security Adversarial resistance, prompt injection tests, red-teaming results NIST MEASURE 2.7; BSI AIC4
Availability Uptime, latency, error rate classic SLO
Compliance / logging Completeness of logs pursuant to Art. 12 AI Act (for high-risk), retention period Art. 12 AI Act reference point
Human oversight Override rate, escalation rates, training evidence Art. 14 AI Act reference point; Art. 22 GDPR

2. Frequency per risk class

Risk class (AI Act trigger) Performance check Drift check Fairness audit Full audit
High risk (Annex I/III) daily / continuous weekly quarterly semi-annually
Limited risk (Art. 50) weekly monthly semi-annually annually
Minimal risk monthly quarterly annually annually

In the case of sector-specific supervision (BaFin, BAFA, BfArM, BNetzA), their frequency requirements take precedence.

3. Thresholds and escalation

Per KPI:

  • Green range: tolerance, no intervention
  • Yellow (warning): increased monitoring, root cause analysis, ticket
  • Red (alarm): escalation to AI owner + risk owner, if necessary decommissioning, incident report

Escalation matrix with roles and availability details. For high-risk systems: documented emergency plan.

4. Logging and records

  • For high-risk systems (reference point Art. 12 AI Act): automatic recording of lifecycle events; retention period in accordance with Art. 19 AI Act – detailed review in ki-vo-compliance.
  • Under data protection law (Art. 32 GDPR + § 9 BDSG considerations): logs themselves may contain personal data; access and deletion concept (necessity, purpose limitation, retention period).
  • Under professional law: review logs as a data flow involving “contributing persons” (§ 203 (4) StGB).

5. Human oversight (human-in-the-loop)

Concrete requirements (reference Art. 14 AI Act + Art. 22 GDPR):

  • Clear override option for the supervising person
  • Training of the supervising persons (Art. 4 AI Act)
  • Documentation of override decisions
  • Limit “automation bias”: no mere confirmation routine

6. Incident management

  • Incident definition: serious incident within the meaning of Art. 3 No. 49 AI Act (reference point) ↔ internal incident (below the reporting obligation)
  • Reporting channels: internal (Data Protection Officer, IT security, compliance); external (supervisory authorities – detailed review in ki-vo-compliance and datenschutzrecht)
  • Lessons learned: documented follow-up, risk register update

7. Audits

  • Internal audit in accordance with ISO/IEC 42001 Chap. 9 (at least annually)
  • For third-party providers: contractual audit right under Art. 28 III lit. h GDPR; BSI AIC4 as assessment standard
  • For sector-specific supervision: schedule their audits

Sources and citation style

Binding: ../../../references/zitierweise.md.

Union and national law

  • Regulation (EU) 2024/1689 (AI Act) – Reference points Art. 12 (record-keeping obligations), Art. 13 (transparency towards deployers), Art. 14 (human oversight), Art. 17 (quality management system for providers), Art. 26 (obligations of deployers), Art. 72 (post-market monitoring), Art. 73 (serious incidents). Obligation review in the plugin ki-vo-compliance.
  • Art. 22 GDPR
  • Art. 32 GDPR (security of processing)
  • § 1 AGG

Industry standards

  • ISO/IEC 42001:2023 – AIMS, in particular Ch. 8 (Operation), Ch. 9 (Performance Evaluation), Ch. 10 (Improvement)
  • ISO/IEC 5338:2023 – AI system life cycle processes
  • NIST AI RMF 1.0 (01/2023) – MEASURE and MANAGE functions
  • BSI AIC4 – requirement groups for monitoring, incident management, audit

Supervisory practice

  • BaFin "Supervisory requirements for the use of big data and artificial intelligence" (BDAI) (June 2018) [unverifiziert – prüfen auf bafin.de]
  • BaFin MaRisk AT 4.3 (outsourcing), AT 7.2 (technical and organizational equipment)
  • DSK resolutions on AI and automated decision-making [unverifiziert – prüfen auf datenschutzkonferenz-online.de]

Commentaries

  • Selmayr, in: Ehmann/Selmayr, DSGVO, 3rd ed. 2024, Art. 22 marginal no. 1 et seq. [unverifiziert – prüfen]
  • Jandt, in: Kühling/Buchner, DSGVO/BDSG, 4th ed. 2024, Art. 32 marginal no. 1 et seq. [unverifiziert – prüfen]
  • Spindler, in: Hartmann/Hilgendorf, KI-Recht, 2024, Ch. Monitoring and Post-Market Surveillance [unverifiziert – prüfen]

Case law ([unverifiziert – prüfen in Beck-Online / juris / curia.europa.eu])

  1. CJEU, judgment of 07.12.2023 – C-634/21, ECLI:EU:C:2023:957 (SCHUFA, Art. 22 GDPR)
  2. BAG on co-determination for monitoring devices § 87 I No. 6 BetrVG

Output format

KI-MONITORING-KONZEPT
System: <System-ID>  |  Risikoklasse: <KI-VO-Trigger>  |  Stand: <Datum>

I. Geltungsbereich und Schutzgüter
II. Monitoring-Dimensionen und KPIs
    <Tabelle: Dimension | Metrik | Schwellenwert Grün/Gelb/Rot | Frequenz | Verantwortlich>
III. Logging (Art. 12 KI-VO als Bezugspunkt)
     – Aufbewahrung, Zugriff, Löschkonzept
IV. Menschliche Aufsicht (Art. 14 KI-VO / Art. 22 DSGVO)
    – Override-Mechanik, Schulung Art. 4 KI-VO, Bias-Schutz
V. Eskalationsmatrix
   <Rolle | Erreichbarkeit | Auslöser | Reaktionszeit>
VI. Vorfallsmanagement
    – interner / externer Meldepfad (Detailprüfung /ki-vo-compliance)
VII. Audit-Plan
     – intern jährlich; Drittanbieter (Art. 28 III lit. h DSGVO, BSI AIC4)
VIII. Wiedervorlage und Konzept-Review
IX. Quellenverzeichnis

Risks / typical errors

  • No thresholds – monitoring without a trigger is theater.
  • Missing drift check – the model deteriorates silently.
  • Fairness audit on an overall view – group-specific error rates are overlooked.
  • Override mechanism only formal – automation bias remains unaddressed.
  • Framing logging data as "technical" – they are generally personal data, observe Art. 32 GDPR and retention periods.
  • Build AI Act obligations into the monitoring concept instead of referring to ki-vo-compliance – maintenance effort and inconsistency.
  • Ignoring sector-specific supervision (BaFin, BfArM).