Metaluxo
← Insights

AI security

The EU AI Act: What It Means for Your Security Programme

The EU AI Act classifies AI systems by risk. High-risk systems — including many used in healthcare and fintech — face new security obligations.

Under the EU AI Act — and most companies are high-risk.

Your startup uses AI to triage patient symptoms, score loan applications, or screen CVs. You thought GDPR was the only regulation you needed to worry about. Then the EU AI Act arrived.

At Metaluxo we advise HealthTech and FinTech companies on the security implications of the AI Act. The regulation is complex, but the security requirements are not exotic. They map closely to existing frameworks — with a few AI-specific additions.


Risk levels under the AI Act

Prohibited AI

Systems that manipulate behaviour, exploit vulnerabilities, or use biometric identification in public spaces. These are banned outright.

High-risk AI

Systems that affect safety, fundamental rights, or access to essential services. This includes:

  • Medical devices and diagnostic tools
  • Credit scoring and insurance risk assessment
  • Recruitment and HR systems
  • Educational scoring
  • Law enforcement risk assessment
  • Critical infrastructure management

Most HealthTech and FinTech AI products fall into this category.

Limited-risk AI

Systems that interact with humans (chatbots) or generate content. Transparency obligations apply.

Minimal-risk AI

Spam filters, recommendation engines, and other low-impact systems. Minimal obligations.


Security requirements for high-risk AI

The AI Act requires high-risk systems to meet standards across seven areas. The security-relevant ones are:

1. Risk management system

You must identify and mitigate risks throughout the AI system’s lifecycle. This is functionally identical to ISO 27001 risk assessment — just applied to AI-specific risks (bias, accuracy, robustness).

2. Data governance

Training data must be relevant, representative, and free of errors. For security, this means:

  • Data provenance documentation
  • Quality assurance processes
  • Protection against data poisoning attacks
  • Secure data storage and transfer

3. Technical documentation

You must document the system’s architecture, training process, performance metrics, and limitations. This is the AI equivalent of a software design document.

4. Record-keeping

Automatic logging of events during operation. For security, this means:

  • Input and output logging
  • Model version tracking
  • Error and anomaly logging
  • Access logs for the AI system

5. Transparency

Users must be informed that they are interacting with an AI system. For high-risk systems, you must provide clear instructions on capabilities, limitations, and expected performance.

6. Human oversight

High-risk AI systems must be subject to meaningful human oversight. The human must be able to:

  • Understand the system’s capabilities and limitations
  • Correctly interpret outputs
  • Decide not to use the system in particular situations
  • Intervene to override or reverse decisions

7. Accuracy, robustness, and cybersecurity

The system must achieve appropriate levels of accuracy, robustness, and security. Specifically:

  • Defence against adversarial attacks
  • Resilience to errors and inconsistencies
  • Protection of model integrity
  • Secure development lifecycle

What this means for your security programme

If you already have ISO 27001 or a mature security programme, the gap is small:

AI Act requirement Existing control Gap
Risk management ISO 27001 risk assessment Extend to AI-specific risks
Data governance Data classification and handling Add data quality and provenance
Technical documentation SDLC documentation Add model cards and training logs
Record-keeping System logging Extend to AI inputs and outputs
Transparency Privacy policy Add AI disclosures
Human oversight Change management Add AI decision override procedures
Cybersecurity Existing security controls Add adversarial robustness testing

Timeline

  • 2025: Prohibited AI practices banned
  • 2026: Governance obligations for general-purpose AI models
  • 2027: High-risk AI system obligations fully in force

If your product is high-risk AI, you have time to prepare — but not much.


At Metaluxo we advise HealthTech and FinTech companies on AI Act readiness, typically delivering a gap assessment and remediation roadmap in four weeks. If your product uses AI and you are unsure whether the Act applies, book a free 30-minute consultation and we will assess your risk level.

Common questions

Does the AI Act apply to all AI systems?

No. It applies based on risk level. Minimal-risk AI (spam filters, recommendation engines) has minimal obligations. High-risk AI (medical devices, credit scoring, recruitment) has strict requirements.

When does the EU AI Act come into force?

The Act was adopted in 2024. Prohibitions took effect in February 2025. High-risk system obligations phase in between 2026 and 2027.

Do we need a new security framework for AI?

Not necessarily. ISO 27001, SOC 2, and existing risk management processes cover much of the AI Act security requirements. The gap is usually in AI-specific governance and documentation.

Related reading

Designed graphic on a deep navy background showing the number 73 with the caption PERCENT — enterprises struggling with AI execution. AI SECURITY

AI Risk Management: Execution Beats Adoption

73% of enterprises struggle with AI execution not adoption — shadow AI, data leakage, model risk. The five-part framework that closes the gap. · 7 min read
Designed graphic on a deep navy background showing the number 4 with the caption LAYERS — defense in depth for customer-facing SaaS. VCISO

ASOS Breach: Why SaaS Needs Defense in Depth

The ASOS breach shows how one compromised identity leads to deep penetration. Four defense layers every customer-facing business needs now. · 6 min read
Send us a message
Message us Book now