L4 · 07 · Governance, Regulation & Responsible AI
At expert level you're not just building AI — you're accountable for it. Regulation is arriving fast (the EU AI Act, sector rules, internal audit), and "we didn't know the model would do that" isn't a defense. This page covers the regulatory landscape, model risk management, documentation duties, and how to build responsibly without killing velocity.
By the end of this page you can
- Describe the EU AI Act's risk tiers and what triggers them.
- Apply model risk management: inventory, classification, documentation, monitoring.
- Write the documentation regulators and auditors actually ask for.
- Reason about IP, data protection and employment implications.
- Design governance that keeps teams fast, not just compliant.
Why this is now your problem
When AI was an experiment, governance was a slide in a deck. Now LLM features touch customers, money and personal data, so they fall under real obligations: privacy law (GDPR and friends), emerging AI-specific regulation, and industry audit expectations. Senior engineers who build these systems get asked — and should be able to answer — "where does the data go, what's the worst it can do, and who signed off?"
The EU AI Act (the one to know)
The first comprehensive AI law, phased in from 2024. It sorts AI uses into risk tiers:
| Tier | Examples | Obligation |
|---|---|---|
| Unacceptable | Social scoring, manipulative dark patterns, some biometric surveillance | Banned |
| High risk | Hiring/credit/insurance decisions, critical infrastructure, law enforcement, education | Strict: risk management, data governance, documentation, human oversight, logging, conformity assessment |
| Limited | Chatbots, generated content | Transparency: disclose it's AI, label generated content |
| Minimal | Spam filters, most dev tools | Largely free |
Even if you're not in the EU, it's the template other jurisdictions copy — and it applies to providers and deployers. Your store assistant is probably "limited" (transparency duty: tell users it's AI); a resume-screening tool would be "high risk" with the full weight of obligations.
Model risk management (MRM)
Borrowed from banking, now spreading to all AI. Treat each model/LLM feature as a risk-managed asset:
- Inventory: a register of every model/feature in production — you can't govern what you don't list.
- Risk classification: what decision does it influence, and what if it's wrong? Auto-complete ≠ loan approval.
- Validation: evals and testing before deployment (L3·04) — documented, repeatable.
- Documentation: purpose, data, training/fine-tuning, evals, limitations, approval trail.
- Monitoring: drift and quality over time (L3·07) plus an incident path.
- Human oversight: someone accountable, and a way to override or shut it off (kill switch).
Documentation that gets asked for
| Artifact | Contents |
|---|---|
| Model card | What it does, training data (in general terms), evals, limitations, intended use, out-of-scope uses |
| System card / datasheet | The whole feature: model + prompts + tools + data + guardrails, not just the model |
| Data lineage | Where training/retrieval/user data comes from and where it goes |
| Decision log | Why this model, why this threshold, who approved |
| Incident log | What went wrong, when, how fixed — auditors love this |
The pattern: document enough that a competent outsider could understand and reproduce your risk assessment. If you can't explain why a decision was made six months later, the documentation failed.
IP, data and people
- Training data: is your fine-tune data yours to use? Scraped data has copyright questions still in court.
- Generated code/IP: model output can resemble licensed code; check policy on attribution and provenance tools.
- Personal data: don't feed PII to models without a lawful basis; anonymise, minimise, check provider data terms.
- Provider terms: does the API train on your inputs? Retention? Many offer zero-retention tiers for a reason.
- People: AI that makes employment, credit or access decisions about people is the highest-risk category — be careful.
Governance that doesn't kill velocity
Lightweight and effective
- A model register + a one-page risk template per feature
- Approved-model list maintained by the platform team (L4·06)
- Evals required in CI — compliance is the quality gate
- Kill switch + incident path, tested
Heavyweight and counterproductive
- Committee approval for every prompt change
- Bans that push people to shadow AI on personal devices
- Paperwork nobody reads, that blocks shipping but not risk
The best governance is paved paths: make the compliant way the easiest way (approved models behind the gateway, templates, built-in evals) so following it is faster than going around it.
- Write a one-page model card for the Store Assistant: purpose, data, evals, limitations, oversight.
- Classify it under the EU AI Act tiers. What would change if it set staff schedules or approved refunds autonomously?
- Draft the "model register" row: owner, risk tier, data flows, eval cadence, kill switch.
Key takeaways
- The EU AI Act's risk tiers (banned/high/limited/minimal) are the regulatory template to know.
- Model risk management: inventory, classify, validate, document, monitor, oversee.
- Document enough that an outsider can reproduce your risk reasoning.
- Good governance paves paths, not paperwork — make compliant the easy path.