AI Security Projects for Practice: 10 Hands-On Labs for Prompt Injection, Data Poisoning, and Model Hardening

AI security projects for practice are essential for anyone building or deploying large language models and machine learning systems. Two threat families dominate real-world incidents: prompt injection, where malicious instructions hijack model behavior or leak data, and data poisoning, where corrupted training data degrades accuracy, inserts backdoors, or causes targeted failures. These risks appear across widely used guidance including OWASP LLM Top 10, MITRE ATLAS, NIST AI risk management practices, ISO/IEC 42001 governance expectations, and emerging regulatory enforcement such as the EU AI Act.

What makes these threats urgent is their practicality. Research and industry reporting indicate that prompt injection can succeed at high rates in everyday enterprise content such as emails, SharePoint files, and web metadata, with reported success rates reaching up to 86% in realistic settings. On the training side, poisoning can quietly erode model integrity over time. A widely discussed example involves tampered ImageNet subsets that reduced model accuracy and forced retraining with stricter governance controls.

This article outlines 10 hands-on labs you can run as AI security projects for practice. Each lab covers what you will learn, what to build, and how to harden the system. These labs also map well to structured learning paths in AI, cybersecurity, and DevSecOps for professionals building documented, assessable skills.

Why Prompt Injection and Data Poisoning Deserve Hands-On Practice

Prompt injection scales in ways that make it particularly difficult to contain: attackers can hide malicious instructions in data your model reads, and the model may comply even when safety guardrails appear to be in place. This includes direct injection, where the user types the attack, and indirect injection, where the attack lives inside retrieved documents, emails, or web pages your system ingests.

Data poisoning has a different failure mode: it changes what the model learns. Indiscriminate poisoning reduces overall accuracy, while targeted poisoning creates failures on specific classes - for example, a fraud detection model that begins missing real fraud cases. Backdoor poisoning allows a model to behave normally until it encounters a trigger phrase, pattern, or token, at which point it misclassifies on demand. Industry groups like the Cloud Security Alliance emphasize that poisoning is long-term and covert, making governance and data provenance critical controls.

How to Use These AI Security Projects for Practice

Run the labs in two passes:

1. Offense first: reproduce the vulnerability so you can measure it.

2. Defense second: implement mitigations and evaluate improvement using repeatable tests.

Most labs are scoped for 90 to 120 minutes, which is sufficient time to build, break, and harden a small system.

10 Hands-On Labs to Build Prompt Injection, Poisoning, and Hardening Skills

1) Direct Prompt Injection Lab (Basic)

Goal: demonstrate how a model can be coerced into ignoring system instructions.

• Build: a simple chatbot with system prompt rules and a small set of sensitive strings in a mock knowledge base.

• Attack: attempt instruction override, role-play jailbreaks, and variations of "ignore previous instructions."

• Harden: input normalization, instruction hierarchy enforcement, output validation, and safe response templates for restricted topics.

Measure: the percentage of attack prompts that trigger policy-violating outputs before versus after mitigation.

2) Excessive Agency and Arbitrary Tool Invocation Lab

Goal: test guardrails for agentic AI that can call tools such as web search, file access, code execution, or email.

• Build: an agent with two to three tools, for example search, calculator, and file read.

• Attack: prompt the model to call tools it should not use, or to exfiltrate data through tool outputs.

• Harden: least-privilege tool permissions, explicit allowlists, human-in-the-loop gates for risky actions, and structured tool schemas.

Key concept: many failures originate from authorization gaps, not just malformed prompts.

3) Rogue Reviewer Lab: Label-Flip Data Poisoning

Goal: simulate poisoning in a sentiment classifier by flipping training labels.

• Build: a basic text classifier trained on product or service reviews.

• Attack: flip a percentage of labels, for example labeling negative reviews as positive.

• Detect: use exploratory data analysis to identify anomalies, class imbalance shifts, and label-text inconsistencies.

Harden: data validation rules, sampling audits, and cross-annotator agreement checks.

4) Secure Data Preprocessing Lab with Semantic Validation

Goal: neutralize poisoning and label manipulation before training begins.

• Build: a preprocessing pipeline with text cleaning, deduplication, and schema validation.

• Attack: inject duplicated spam, near-duplicates, or mislabeled samples that pass naive checks.

• Harden: semantic similarity checks, outlier detection, and rules that verify label alignment with text features.

Deliverable: a data quality report artifact generated on every training run.

5) Model Integrity Defense Lab: Tamper-Evidence and Parameter Integrity

Goal: detect unexpected model changes and pipeline tampering.

• Build: a training pipeline that stores model artifacts - weights, configuration, and tokenizer - in versioned storage.

• Attack: simulate parameter tampering or swap a model file in the artifact store.

• Harden: model signing, hash verification at load time, immutable artifact registries, and gated promotion between environments.

Supply chain angle: incorporate SBOM-style inventories and SLSA-inspired build provenance for ML artifacts.

6) Poisoned Pipeline Lab: Backdoor Trigger Injection

Goal: create and detect a backdoor that activates on a specific trigger.

• Build: an image or text classifier.

• Attack: insert a small trigger pattern tied to a target label and retrain the model.

• Test: confirm that normal accuracy remains high while triggered inputs misclassify.

• Harden: trigger scanning, data provenance checks, robust training practices, and periodic retraining with clean, verified datasets.

Real-world mapping: this mirrors "looks fine until it matters" failures common in safety-critical and fraud detection contexts.

7) Prompt Injection CTF Lab: 10 Scenarios

Goal: practice diverse injection patterns at speed across varied scenarios.

• Scenarios: system prompt override, indirect injection via retrieved documents, jailbreak patterns, and instruction smuggling.

• Harden: layered controls including retrieval filtering, content sanitization, response constraints, and policy-aware post-processing.

Tip: track false positives and false negatives to ensure defenses do not break legitimate user workflows.

8) ML CTF Lab Set: Poisoning, Inversion, and Extraction

Goal: broaden skills beyond injection to cover core ML security threats.

• Attack modules: prompt injection, data poisoning, model inversion, and model extraction.

• Harden: rate limiting, output filtering, privacy risk tests, and differential privacy concepts where appropriate.

Why it matters: model inversion and extraction connect directly to data leakage concerns in regulated industries and compliance-sensitive environments.

9) Offensive AI Security CTF Lab (Team Red-Teaming)

Goal: run realistic red-team exercises against LLM applications and ML pipelines.

• Attack: prompt injection against tool-using agents, backdoor testing, and workflow exploitation.

• Defend: monitoring, incident response playbooks, and measurable policies for disclosure and remediation.

Practice outcome: build experience in detection and response, not just prevention.

10) OWASP-Oriented Adversarial Labs: Hardening in DevSecOps Pipelines

Goal: align hands-on work with a recognized risk taxonomy.

• Build: CI checks for prompt-risk tests, dataset integrity checks, and model artifact verification.

• Harden: automated gates that fail builds when risk thresholds are exceeded.

Bonus: incorporate attention-based or behavior-based detection experiments. Recent work suggests that attention-driven detectors can improve prompt injection detection performance by roughly 10% AUROC over simpler baselines in some evaluations, providing a useful benchmark for your own experiments.

Model Hardening Checklist You Can Reuse Across Labs

• Input validation and sanitization: normalize text, strip hidden instructions where possible, and segment user, system, and retrieved content as separate sources of trust.

• Guardrails and permissions: enforce least privilege for tools, maintain explicit allowlists, and require approvals for irreversible actions.

• Data provenance and governance: track dataset sources, maintain versioning, and verify integrity with hashes and access controls.

• Adversarial training: include synthetic injected prompts and poisoned samples to improve robustness against known attack patterns.

• Differential privacy concepts: limit the influence of individual samples to reduce poisoning leverage and privacy leakage risk.

• Red-teaming and monitoring: schedule continuous testing, maintain logging and anomaly detection, and define incident response procedures in advance.

How These Labs Align with Professional Upskilling

For teams standardizing capabilities, these AI security projects for practice map well to structured curricula covering AI, cybersecurity, and DevSecOps. The value of pairing lab work with formal certification is consistency: shared terminology, repeatable assessment criteria, and documented capability development that organizations can track and verify.

Conclusion

Prompt injection and data poisoning are active, documented threats. Prompt injection has demonstrated high success rates against common enterprise content channels, and poisoning incidents have forced costly retraining cycles and governance overhauls across the industry. The most reliable path to competence is hands-on: reproduce the failure, implement layered defenses, and measure the improvement.

Use these 10 labs as a practical sequence: start with direct prompt injection, progress to agent tool abuse, then build strong data hygiene and integrity controls, and validate everything through CTF-style red-teaming. A well-documented set of AI security projects for practice becomes a repeatable blueprint for building trustworthy AI systems ready for production deployment.