Your startup uses AI to triage patient symptoms, score loan applications, or screen CVs. You thought GDPR was the only regulation you needed to worry about. Then the EU AI Act arrived.
At Metaluxo we advise HealthTech and FinTech companies on the security implications of the AI Act. The regulation is complex, but the security requirements are not exotic. They map closely to existing frameworks — with a few AI-specific additions.
Risk levels under the AI Act
Prohibited AI
Systems that manipulate behaviour, exploit vulnerabilities, or use biometric identification in public spaces. These are banned outright.
High-risk AI
Systems that affect safety, fundamental rights, or access to essential services. This includes:
- Medical devices and diagnostic tools
- Credit scoring and insurance risk assessment
- Recruitment and HR systems
- Educational scoring
- Law enforcement risk assessment
- Critical infrastructure management
Most HealthTech and FinTech AI products fall into this category.
Limited-risk AI
Systems that interact with humans (chatbots) or generate content. Transparency obligations apply.
Minimal-risk AI
Spam filters, recommendation engines, and other low-impact systems. Minimal obligations.
Security requirements for high-risk AI
The AI Act requires high-risk systems to meet standards across seven areas. The security-relevant ones are:
1. Risk management system
You must identify and mitigate risks throughout the AI system’s lifecycle. This is functionally identical to ISO 27001 risk assessment — just applied to AI-specific risks (bias, accuracy, robustness).
2. Data governance
Training data must be relevant, representative, and free of errors. For security, this means:
- Data provenance documentation
- Quality assurance processes
- Protection against data poisoning attacks
- Secure data storage and transfer
3. Technical documentation
You must document the system’s architecture, training process, performance metrics, and limitations. This is the AI equivalent of a software design document.
4. Record-keeping
Automatic logging of events during operation. For security, this means:
- Input and output logging
- Model version tracking
- Error and anomaly logging
- Access logs for the AI system
5. Transparency
Users must be informed that they are interacting with an AI system. For high-risk systems, you must provide clear instructions on capabilities, limitations, and expected performance.
6. Human oversight
High-risk AI systems must be subject to meaningful human oversight. The human must be able to:
- Understand the system’s capabilities and limitations
- Correctly interpret outputs
- Decide not to use the system in particular situations
- Intervene to override or reverse decisions
7. Accuracy, robustness, and cybersecurity
The system must achieve appropriate levels of accuracy, robustness, and security. Specifically:
- Defence against adversarial attacks
- Resilience to errors and inconsistencies
- Protection of model integrity
- Secure development lifecycle
What this means for your security programme
If you already have ISO 27001 or a mature security programme, the gap is small:
| AI Act requirement | Existing control | Gap |
|---|---|---|
| Risk management | ISO 27001 risk assessment | Extend to AI-specific risks |
| Data governance | Data classification and handling | Add data quality and provenance |
| Technical documentation | SDLC documentation | Add model cards and training logs |
| Record-keeping | System logging | Extend to AI inputs and outputs |
| Transparency | Privacy policy | Add AI disclosures |
| Human oversight | Change management | Add AI decision override procedures |
| Cybersecurity | Existing security controls | Add adversarial robustness testing |
Timeline
- 2025: Prohibited AI practices banned
- 2026: Governance obligations for general-purpose AI models
- 2027: High-risk AI system obligations fully in force
If your product is high-risk AI, you have time to prepare — but not much.
At Metaluxo we advise HealthTech and FinTech companies on AI Act readiness, typically delivering a gap assessment and remediation roadmap in four weeks. If your product uses AI and you are unsure whether the Act applies, book a free 30-minute consultation and we will assess your risk level.
Common questions
Does the AI Act apply to all AI systems?
No. It applies based on risk level. Minimal-risk AI (spam filters, recommendation engines) has minimal obligations. High-risk AI (medical devices, credit scoring, recruitment) has strict requirements.
When does the EU AI Act come into force?
The Act was adopted in 2024. Prohibitions took effect in February 2025. High-risk system obligations phase in between 2026 and 2027.
Do we need a new security framework for AI?
Not necessarily. ISO 27001, SOC 2, and existing risk management processes cover much of the AI Act security requirements. The gap is usually in AI-specific governance and documentation.
