AI Security and Governance

AI Security Landscape

As organisations increasingly adopt artificial intelligence, new security challenges emerge that differ significantly from traditional cybersecurity threats. AI systems introduce unique attack surfaces and require specialised governance frameworks.

Key AI Security Threats

Prompt Injection

Attackers craft malicious inputs to manipulate the behaviour of large language models, potentially causing the system to reveal sensitive information, generate harmful content, or perform unauthorised actions.

Data Poisoning

Introducing malicious or misleading data into training datasets to corrupt model behaviour. This can lead to biased outputs, reduced accuracy, or intentional backdoors in the model.

Model Theft

Unauthorised copying or reverse-engineering of AI models through API queries or other means. This represents both an intellectual property concern and a security risk if the model contains sensitive training data.

Adversarial Examples

Carefully crafted inputs designed to cause AI models to make specific errors. In security contexts, this could mean evading malware detection or misclassifying threats.

AI Governance Framework

Hong Kong’s AI Taskforce has proposed a governance framework built on five pillars:

1. Accountability: Clear responsibility for AI system outcomes
2. Transparency: Disclosure of AI usage and decision-making processes
3. Fairness: Prevention of bias and discrimination in AI outputs
4. Reliability: Ensuring AI systems perform as intended across scenarios
5. Security: Protecting AI systems from manipulation and abuse

Practical Recommendations

• Implement input validation and sanitisation for all AI system inputs
• Maintain comprehensive logs of AI system interactions
• Conduct regular security assessments of AI models and pipelines
• Establish human oversight for high-stakes AI decisions
• Develop incident response procedures specific to AI security events