RAG Evaluation Guide: Metrics, Frameworks, and Python Examples
A layered guide to retrieval metrics, generation faithfulness, synthetic test sets, citations, refusals, and production monitoring.
Read RAG guideAI Engineering
A practical map for building LLM systems, agent workflows, RAG applications, evaluation layers, and MLOps foundations that can survive contact with production.
Start here
Start here
AI engineering is the work around the model: retrieval, orchestration, data contracts, schemas, evaluation, monitoring, and product boundaries. The writing here is for teams that need AI features to be reviewed, shipped, and improved rather than only demoed.
Production questions
Production questions
Useful AI systems need answers to concrete questions. What sources are allowed? What output shape is valid? Which failures require escalation? Which examples become regression tests? Which metrics show quality after launch?
Internal links
Internal links
The related articles below connect architecture to implementation: agent evaluation, tool integration, uncertainty, document AI, MLOps, and reproducible machine learning environments.
Topic hub
AI engineering writing
AI engineering is the discipline of turning model capability into a dependable workflow. It includes retrieval design, tool boundaries, schemas, logging, deployment, monitoring, and the product constraints that decide whether an AI feature can be trusted.
A good production AI system makes its evidence, actions, and failure modes inspectable. The model is only one component. The surrounding system has to decide which documents are allowed, which tool calls are safe, how outputs are structured, when humans review the result, and what regressions block a release.
This section follows the starter-blog pattern of metadata-driven topic pages: each card shows a canonical article link, summary, date, reading time, and tags. The page is a hub, not a duplicate post archive. For the evaluation layer of retrieval systems, start with the dedicated RAG evaluation guide for metrics, test sets, and monitoring. For agent reliability, use the Copilot agent golden test set guide as the practical test-case companion.
AI Agent Evaluation Is More Important Than Prompting
Why production AI agents need custom eval sets, trajectory checks, calibrated judges, regression tests, and business-ready metrics.
Read articleTesting and Evaluating Copilot Agents
Schema-first evaluation, test sets, release gates, and human review for Copilot-style agents.
Read articleCopilot Agent Golden Test Set
How to turn Copilot agent scenarios into reusable cases, rubrics, schema checks, and regression gates.
Read test set guideModel Context Protocol Guide
Tool and context integration patterns for connected AI systems.
Read articleAgentic AI Systems with n8n
Workflow boundaries, tool use, and operational constraints for agentic automation.
Read articleLlamaReport and Document AI Systems
Document transformation architecture, review workflows, and enterprise AI design.
Read articleMLOps Essential Skills
Model packaging, deployment, monitoring, and production ML operations.
Read articleFor future notes on applied AI systems, evaluation, data products, and product workflows, subscribe to Yangming Li's Newsletter.
Subscribe for updates