Skill Scanner

A best-effort security scanner for AI Agent Skills that detects prompt injection, data exfiltration, and malicious code patterns. It combines pattern-based detection (YAML + YARA-X), AST and dataflow analysis, an optional LLM-as-a-judge, and a bounded CEL decision layer over typed detector facts.

Important: This scanner provides best-effort detection, not comprehensive or complete coverage. A scan that returns no findings does not guarantee that a skill is free of all threats. See Scope and Limitations below.

Supports OpenAI Codex Skills and Cursor Agent Skills formats following the Agent Skills specification. With --lenient, also scans non-standard formats such as Claude Code .claude/commands/*.md and flat markdown skill repos.

---

Highlights

• Multi-Engine Detection - Static analysis, behavioral dataflow, LLM semantic analysis, and cloud-based scanning for layered, best-effort coverage
• Typed CEL Decisions - The core scanner uses the official cel-go v0.32.0 runtime to correlate bounded facts after deterministic detection and before optional LLM analysis
• Measured Presets - low-noise and quiet presets and LLM caps, with recall, false-positive rate and F1 published for each (Recommended Settings)
• CI/CD Ready - SARIF output for GitHub Code Scanning, reusable GitHub Actions workflow, exit codes for build failures
• Pre-commit Hook - Standard pre-commit framework integration to scan skills before every commit
• Extensible - Plugin architecture for custom analyzers

Join the Cisco AI Discord to discuss, share feedback, or connect with the team.

---

Scope and Limitations

Skill Scanner is a detection tool. It identifies known and probable risk patterns, but it does not certify security.

Key limitations:

• No findings ≠ no risk. A scan that returns "No findings" indicates that no known threat patterns were detected. It does not guarantee that a skill is secure, benign, or free of vulnerabilities.
• Coverage is inherently incomplete. The scanner combines signature-based detection, LLM-based semantic analysis, behavioral dataflow analysis, optional cloud services, and configurable rule packs. While this approach improves coverage, no automated tool can detect every technique, especially novel or zero-day attacks.
• False positives and false negatives can occur. Presets, the LLM judge and scoped suppressions reduce noise, but no configuration eliminates all incorrect classifications. Pick a measured setup from Recommended Settings and tune the scan policy to your risk tolerance.
• Human review remains essential. Automated scanning is one component of a defense-in-depth strategy. High-risk or production deployments should pair scanner results with manual code review and/or threat modeling.

Measured results

On held-out skills (MaliciousSkillBench's frozen test split, 839 malicious and 545 harmless):

• Rules alone catch 7.7% of malicious skills at HIGH, at a 4.0% false-positive rate.
• With the LLM judge (Gemma 4 26B, balanced), 66.7% reach review at MEDIUM+ (15.4% FPR), and 33.7% are blocked at HIGH (6.4% FPR).

Every figure, with its corpus and method, is in Measured Results and on the evaluation Space.

---

Documentation

The documentation website is cisco-ai-defense.github.io/docs/skill-scanner. Deep-dive pages live in docs/.

Guide	Description
Quick Start	Get started in 5 minutes
Recommended Settings	Pick a setup for the lowest FPR or the highest F1, with copy-paste configs
LLM Providers	Configure the LLM judge for any provider, gateway or local model
Results and Tuning	Read findings, build a review queue, lower false positives
Architecture	System design and components
CEL Decision Layer	Typed facts, safety bounds, rollout modes, and telemetry
Threat Taxonomy	Complete AITech threat taxonomy with examples
LLM Analyzer	LLM configuration and usage
System One Analyzer	Optional advisory screening tier, and why it cannot gate
Meta-Analyzer	Optional second-pass review (off by default; measured cost)
Behavioral Analyzer	Dataflow analysis details
Scan Policy	Custom policies, presets, and tuning guide
Policy Quick Reference	Compact reference for policy sections and knobs
Measured Results	Every published figure, with its corpus, population and model
Rule Authoring	How to add signature, YARA, and Python rules
GitHub Actions	Reusable workflow for CI/CD integrat