What is prompt injection?

Prompt injection is an attack technique where an adversary crafts input — user text, documents, web pages, emails, or tool outputs — so that a large language model (LLM) or agent follows the attacker’s instructions instead of the developer’s system policy.

It is one of the defining AI security risks for Hong Kong organisations deploying chatbots, copilots, RAG knowledge bases, and agentic workflows. Unlike classical XSS or SQL injection, the “interpreter” is a probabilistic model: filters that work yesterday may fail tomorrow when the model, tools, or retrieval corpus change.

Why it matters in Hong Kong

  • Customer-facing bots may leak personal data regulated under the PDPO and PDPO glossary.
  • Internal copilots connected to email, tickets, or code can exfiltrate secrets or change configurations without a traditional malware foothold.
  • Event and marketing workflows that paste client lists into GenAI tools create both privacy and injection exposure — a recurring theme in Trust Reviews AI lenses.
  • Deepfake-enabled social engineering often pairs with AI assistants that “helpfully” process untrusted documents — see AI Security and Governance.

HKISG scores related controls under Methodology v2026.2 (AI security & model governance + evidence quality). Reading this page does not change any public score.

Common patterns

  1. Direct injection — a user message tries to override system rules (“ignore previous instructions and…”).
  2. Indirect injection — malicious content hides inside a retrieved document, webpage, ticket, or email that the model treats as trusted context.
  3. Tool / agent abuse — the model is steered to call privileged tools (refund, reset password, export CRM, change access).
  4. Multi-step chaining — benign-looking first turns build context that later unlocks a sensitive action.

Related terms: LLM, Deepfake, social engineering, HKCERT vs HKISG.

What good looks like

  • Separate untrusted content from system instructions; treat retrieved text as data, not commands.
  • Require human approval for high-impact tool actions (payments, access changes, bulk exports).
  • Log prompts and tool calls with retention aligned to your incident process.
  • Abuse-test assistants before go-live — include indirect injection via documents and URLs.
  • Keep a dated inventory of GenAI systems with named owners (HKISG Evidence quality domain).
  • Ban pasting personal data into public LLMs unless contractually controlled enterprise tools are used.
  • Document fallback behaviour when the model or API provider degrades.

Hong Kong operator checklist

  1. Inventory every production LLM, RAG app, and agent — named owner required.
  2. Map which tools each agent can call and which require human confirmation.
  3. Test for direct and indirect injection before customer launch; keep dated notes.
  4. Align personal-data flows with PDPO notes and Board Briefing Pack language.
  5. Pair with Security Bulletins when AI-assisted phishing or bot abuse campaigns appear.
  6. If an active incident is underway, coordinate via HKCERT — HKISG materials support framing and evidence, not CERT response.

Sources and further reading

Editorial note

Published by the Hong Kong Information Security Group (HKISG) for educational purposes. Not a substitute for legal advice, formal audit opinions, or national CERT coordination.