Detect and patch vulnerabilities in AI-generated Python code, directly from your editor.
Website · Marketplace · Issues · Discussions
Redlyne is a VS Code extension that scans Python code — especially code generated by AI assistants like Copilot, ChatGPT, or Claude — for known vulnerability patterns, and proposes one-click in-editor patches you can apply with a single confirmation.
It runs entirely on your machine. No code is sent to any server.
Built for AI. Built without AI. Redlyne is powered by a deterministic rule engine — no LLM, no probabilistic guesses, no hallucinated fixes. The detection rules are hand-crafted by security researchers, not auto-generated. Every flag and every remediation suggestion is reproducible and auditable.
AI coding assistants are fast, and they're also reliably insecure. In a 2025 study evaluating 609 Python snippets generated by GitHub Copilot, Claude 3.7 Sonnet, and DeepSeek V3, ~76% of the code contained security vulnerabilities — for GitHub Copilot specifically the rate climbs to ~84% (source). The patterns are well-known to security engineers: SQL injection from string concatenation, command injection from os.system, weak crypto defaults, hard-coded secrets, unsafe deserialization, path traversal, SSRF, and more.
There's also a structural problem alongside the security one. A 2025 study analyzing 500 Python snippets across four AI assistants found that ~13% of them are incomplete — missing imports or context. For GitHub Copilot specifically, that number rises to ~31% — almost one snippet in three (source). AST-based analyzers like Bandit, CodeQL, and PyT can't process those snippets at all: without a parseable program, they can't build the tree they rely on.
Redlyne sits in your editor and flags vulnerability patterns the moment you select the snippet, then offers a remediated version. Pattern-based detection works on any fragment, complete or not. Instead of relying on another LLM to find LLM bugs, it uses a curated set of deterministic rules built by people whose job is to find vulnerabilities.
- Built for AI, built without AI — deterministic rule engine, no LLM, no hallucinations, every flag is reproducible
- Expert-curated rule set — 459 detection patterns mapped to OWASP Top 10:2025 categories observed in AI-generated Python code, derived from the SecurityEval, Copilot CWE Scenarios, and PoisonPy benchmarks
- Static vulnerability detection on Python code selected in the editor
- Automated remediation suggestions, applied as in-place edits to the source file with your explicit confirmation
- Right-click integration in the editor context menu for any Python selection
- Local execution — your code never leaves your machine
- 459 deterministic detection rules mapped to OWASP Top 10:2025 categories — see the full list in COVERAGE.md
- 70+ rules with auto-remediation, including 14 multi-line templates that rewrite vulnerable blocks while preserving indentation and adding required imports
- ~70-100 ms per snippet on a typical laptop
- Rules derived from analysis of vulnerable Python samples in state-of-the-art security benchmark datasets
- Each detected vulnerability comes with an automated patch suggestion — not a comment, not an LLM guess, but a concrete code replacement
- Auto-fixes verified end-to-end: every patch is checked for syntax safety, regression-freedom, and that the targeted rule actually stops firing — 9 out of 10 patches on PoisonPy pass all three checks
Evaluated May 2026 across 1700+ vulnerable Python samples spanning five public benchmark datasets. Full cross-dataset numbers in Head-to-head with open-source baselines.
96.8% recall · 0.822 F1 · 459 rules in ~1 ms per file
| Metric | Value |
|---|---|
| Recall | 96.8% |
| F1 score | 0.822 |
| Accuracy | 79.0% |
| Precision | 71.4% |
| Analyzed | 100% (310/310 files, no parse failures) |
| Time on full dataset (310 files) | ~0.4s |
Recall on PoisonPy is higher than the baseline reported in the original paper (~91%) — Redlyne extends the rule set to 459 patterns and catches more issue classes.
9 out of 10 auto-fixes verified safe
When Redlyne emits a patch, the fix is checked against three independent correctness properties:
| Check | What it means | Pass rate on PoisonPy applied patches |
|---|---|---|
| Syntax-safe | Patched code compiles as valid Python | 100% (58/58) |
| Regression-free | No new rule classes appear after the patch | 93% (54/58) |
| Targeted-clean | The specific rule that fired stops firing | 97% (56/58) |
| Targeted-full (all three above) | The patch did its job | 90% (52/58) |
In other words: of every patch Redlyne emits, 9 out of 10 successfully remove the targeted vulnerability without breaking syntax or introducing a new rule class.
python tests/bench_baselines.py # detection across 5 datasets (Redlyne + 4 baselines)
python tests/bench_remediation.py # auto-fix head-to-head (Redlyne + PatchitPy + Semgrep)Both produce JSON + Markdown reports under benchmarks/ in a few minutes.
Evaluated May 2026 across 1700+ vulnerable Python samples spanning five public benchmarks. Reproducible: python tests/bench_baselines.py.
Tools compared: Bandit · Semgrep · Pylint · DeVAIC v2 · Redlyne.
Across every dimension that matters — files actually analyzed, accuracy, speed, fix safety — Redlyne is the only tool that wins on all of them at once.
| Bandit | Semgrep | Pylint | DeVAIC v2 | Redlyne | |
|---|---|---|---|---|---|
| Analyzed (% of PoisonPy parsed, not skipped) | 17% | 86% | 17% | 100% | 100% ✓ |
| Recall (PoisonPy, % of vulns caught) | 5.8% | 20.6% | 18.7% | 64.5% | 96.8% ✓ |
| F1 (PoisonPy) | 0.107 | 0.318 | 0.280 | 0.662 | 0.822 ✓ |
| Speed (ms per file) | ~20 | ~700 | ~55 | ~0.5 | ~1.4 (top tier) |
| Auto-fix? | ✗ | partial (~5% of rules) | ✗ | ✗ (2 of 441 rules) | ✓ (70+ rules) |
| Versus | Recall gap | Analyzed gap | Speed |
|---|---|---|---|
| Bandit | +91.0 pp | +83 pp | ~14× faster |
| Pylint | +78.1 pp | +83 pp | ~40× faster |
| Semgrep | +76.2 pp | +14 pp | ~500× faster |
| DeVAIC v2 (same engine, original rule set) | +32.3 pp | tied at 100% | comparable |
Redlyne ▏ 0.4 s
DeVAIC v2 ▏ 0.2 s
Bandit █████ 6.2 s
Pylint ███████████████ 17 s
Semgrep ████████████████████████████████████████████████████ 217 s
Datasets used: PoisonPy · SafeCoder · SecurityEval · Copilot CWE Scenarios · PromSec.
The advantage isn't a PoisonPy artifact. Headline metric per dataset, all five tools side-by-side — F1 for paired datasets (where precision is measurable), recall for vulnerable-only:
| Dataset | n | Bandit | Semgrep | Pylint | DeVAIC v2 | Redlyne |
|---|---|---|---|---|---|---|
| PoisonPy (paired, F1) | 310 | 0.107 | 0.318 | 0.280 | 0.662 | 0.822 |
| SafeCoder (paired, F1) — real OSS commit fixes | 1052 | 0.435 | 0.515 | 0.449 | 0.501 | 0.556 |
| SecurityEval (recall) | 121 | 40.5% | 34.7% | 59.5% | 63.6% | 93.4% |
| Copilot CWE Scenarios (recall) | 150 | 84.7% | 51.3% | 93.3% | 68.0% | 89.3% |
| PromSec (recall) | 600 | 92.8% | 87.0% | 98.8% | 85.2% | 97.0% |
On Copilot and PromSec, Pylint's "flag almost everything" mode nudges it slightly above Redlyne on raw recall, but at the cost of a 49.7% accuracy on the paired benchmarks — effectively a random classifier on the only datasets where precision is measurable. What matters in production is F1 on paired data, and there Redlyne leads on both paired datasets.
Evaluated May 2026 on 155 PoisonPy vulnerable samples + 526 SafeCoder real commit-based fixes. Reproducible: python tests/bench_remediation.py.
Tools compared: Semgrep --autofix · PatchitPy · Redlyne.
Auto-remediation is the dimension where Redlyne has no real competition. Of the open-source Python tools we tested:
| Tool | Patches emitted | Successful fix rate | Latency per file |
|---|---|---|---|
Semgrep --autofix |
7 / 155 (4.5%) | 71% | ~4700 ms |
| Redlyne | 58 / 155 (37%) | 90% ✓ | ~3 ms ✓ |
Of every patch Redlyne emits, 9 out of 10 successfully remove the targeted vulnerability — verified by an independent rule re-scan, with the patched source compiling and no new vulnerability classes introduced. Redlyne is also ~1500× faster than Semgrep autofix per file.
DeVAIC v2 stock ships only 2 remediation rules out of 441 (0.5%), so it isn't a remediation tool — it stays in the detection comparison above. On SafeCoder (526 real-world commit fixes) Redlyne applies a patch on 19% of samples, with 69% of those passing the same check; the drop reflects how often production fixes involve function-level refactoring rather than the drop-in substitutions our regex-based remediations target.
Redlyne builds on two peer-reviewed lines of research:
Cotroneo, D., De Luca, R., Liguori, P. (2025). DeVAIC: A tool for security assessment of AI-generated code. Information and Software Technology, 177, 107572. DOI 10.1016/j.infsof.2024.107572
The detection rule schema. Redlyne extends the v2.0 rule set introduced in this paper to 459 patterns and adds the
pattern_not_filedirective for scope-aware sanitization detection.
Altiero, F., Cotroneo, D., De Luca, R., Liguori, P. (2025). Securing AI Code Generation Through Automated Pattern-Based Patching. 2025 55th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), pp. 282–289. DOI 10.1109/DSN-W65791.2025.00077
The automated remediation approach. Redlyne extends pattern-based patching with multi-line template rules, syntax-safety verification, and the targeted-clean rule re-scan.
The PoisonPy benchmark dataset is published in:
Cotroneo, D., Improta, C., Liguori, P., Natella, R. (2024). Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks. Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension (ICPC '24), pp. 280–292. DOI 10.1145/3643916.3644416
From the VS Code Marketplace (recommended):
- Open VS Code
- Extensions panel (
Ctrl+Shift+X) - Search for
Redlyne - Click Install
From the command line:
code --install-extension redlyne.redlyne-aiFrom a .vsix (manual install):
- Download the latest
redlyne-x.y.z.vsixfrom Releases - In VS Code, run
Extensions: Install from VSIX...from the Command Palette - Select the downloaded file
- Open a Python file (
.py) in VS Code - Select the block of code you want to analyze
- Right-click the selection → Redlyne: Run Analysis
- Review the notifications
- Confirm to apply the suggested patch
The same command is also available from the Command Palette (Ctrl+Shift+P → "Redlyne: Run Analysis").
Redlyne runs natively on Windows, macOS, and Linux — no WSL, no bash, no jq.
Before using the extension, make sure you have:
- Python 3.10+ on
PATH(python3on macOS/Linux,pythonorpyon Windows) - VS Code 1.89 or newer
Redlyne is in early access. The detection engine is fast, cross-platform, and ready for daily code review, but the rule set is still growing:
- Platform: Windows, macOS, Linux — single Python codebase, no platform shims.
- Performance: typical analysis runs in under 100 ms per snippet.
- Rule coverage: 442 detection rules from the Devaic v2.0 schema, with ~30 of them carrying drop-in safe-replacement remediation. Coverage is expanding release by release.
- Languages: Python only. Other languages may follow based on demand.
If something doesn't work the way you expect, please open an issue — we read everything.
- Detection and remediation depend on the bundled rule set; novel or obfuscated patterns may not be flagged.
- Selections that are too short (a single statement out of context) may not provide enough signal for accurate analysis.
Contributions of all sizes are welcome — bug reports, feature ideas, rule submissions, documentation fixes, and code.
- Read CONTRIBUTING.md for the development setup and pull request process
- Read CODE_OF_CONDUCT.md for community expectations
- Browse open issues, especially those tagged
good first issue - Join the discussion in GitHub Discussions
- General inquiries / partnerships / press: info@redlyne.io
- Bugs and feature requests: GitHub Issues
- Questions and ideas: GitHub Discussions
- Code of Conduct reports: conduct@redlyne.io
- Website: redlyne.io
Redlyne uses a dual-license model:
| Component | License |
|---|---|
| Source code (extension, scripts, build) | Apache License 2.0 |
| Detection rules, vulnerability patterns, datasets | CC BY-NC-SA 4.0 (non-commercial) |
This separation keeps the engineering side fully open source — fork it, extend it, ship it commercially with attribution — while preserving the value of the curated rule set.
For commercial licensing of the rules and datasets, partnership, or enterprise inquiries: redlyne.io · info@redlyne.io
The names "Redlyne" and the Redlyne logo are trademarks. See NOTICE for details.