Inference-time scaling for LLMs-as-a-judge.
An graph-eval framework for LLM's
Evaluation framework for LLM knowledge inputs — prompts, RAG corpora, skills, agent workflows. Fix the model, vary the artifact. Built-in statistical rigor: bootstrap CI, Krippendorff α, length-debias, saturation curves.
An end-to-end AI agent project that transcribes audio files, embeds user queries, and searches in Qdrant and web browser via the Brave API. A Streamlit interface powered by OpenAI GPT models delivers actionable health insights from both the archive and the latest research.
ProductionOS v1.0 — Claude Code plugin with 76 agents, 39 commands, and 12 hooks. Deploys specialized agents that review, score, and improve your entire codebase. Smart routing, recursive convergence, self-evaluation.
StructAI offers a robust toolkit for LLM interaction—such as structured outputs, context management, and parallel execution.
Extensible benchmarking suite for evaluating AI coding agents on web search tasks. Compare native search vs MCP servers (You.com, expanding) across multiple agents (Claude Code, Gemini, Droid, Codex, expanding) with automated Docker workflows and statistical analysis.
Alexey Vorobey's experimental expert circle for Claude Code — 16 distilled mental models (Eric Seufert, Andrew Chen, Elena Verna, ...) that auto-refresh on cadence from LinkedIn, RSS, YouTube, podcasts.
A real-time, zero-trust governance layer that sits between AI agents and the outside world.
EvalBot — local-first chatbot security & quality evaluation (FastAPI + Next.js). Evaluate chatbot answers against your own docs & guidelines with ML/NLP + AI-judge scoring. Apache-2.0.
Eval-driven Customer Support FTE using OpenAI Agents SDK. Multi-agent routing, guardrails, and systematic quality evaluation.
AI数字人外呼多轮对话评测系统。将任务指令编译为状态机， 用覆盖率反馈驱动用户模拟，同等预算下P0风险发现率翻倍。
First-place official system for JOKER 2026 (CLEF) Task 1 English: a three-stage humor retrieval pipeline pairing hybrid sparse-dense retrieval and cross-encoder reranking with a rationale-distilled LLM judge ensemble. 0.6347 MAP. Team VANGUARD.
Adversarial clinical AI evals suite : physician-curated golden test cases exposing safety-critical failure modes in healthcare AI pipelines. SOAP notes, LLM judge, ambient scribe.
Local-first AI job-prep tool: turn collected JDs into requirement maps and learning paths, then practice with a note-grounded RAG interview coach.
Prompt Design & LLM Judge
Lightweight NLP/LLM evaluation toolkit — metrics, LLM-as-Judge, significance testing, prompt robustness
An open-source tool to compare NVIDIA's free LLMs side-by-side with streaming responses, judge scoring, and AI-powered model recommendations.
⚖️ Dual-Judge: 让AI测试结果真正有说服力 | 双LLM交叉验证消除单模型偏见 | 独立于具体Agent的通用评估框架 | Making AI Evaluation Trustworthy
Add a description, image, and links to the llm-judge topic page so that developers can more easily learn about it.
To associate your repository with the llm-judge topic, visit your repo's landing page and select "manage topics."