Tag: hallucination detection
Production-Grade AI Eval Systems. What I Learned Putting LLMs on Call
Anjaiah Methuku | | AI evals, AI guardrails, AI observability, AI quality monitoring, AI reliability engineering, AI SRE, AI testing, answer relevance, CI evals, context precision, context recall, faithfulness, hallucination detection, human-in-the-loop, LLM evaluation, LLM judges, LLM reliability, model monitoring, production AI, production-grade AI, RAG evaluation, RAG metrics, regression testing, retrieval-augmented generationProduction-grade AI reliability requires more than uptime and latency. A layered eval system helps teams detect hallucinations, RAG failures and quality regressions before customers do ...
The Death of the Four Golden Signals: Designing Telemetry for Non-Deterministic Infrastructure
Jyostna Seelam | | AI architecture, AI compliance, AI Governance, AI guardrails, AI infrastructure, AI monitoring, AI observability, AI operations, AI operations strategy., AI performance metrics, AI platform monitoring, AI reliability, AI resilience, AI safety, AI SLIs, AI SLOs, AI system reliability, AI telemetry, AI trust, Enterprise AI, generative AI monitoring, guardrail intervention rate, hallucination detection, LLM monitoring, LLM observability, modern SRE, non-deterministic systems, Observability Engineering, OpenTelemetry, platform engineering, production AI, RAG observability, RAG systems, semantic drift, site reliability engineering, SRE, time to first token, token throughput, TTFT, vector database monitoringIn complex software systems, our traditional definition of operational health has always been comfortably binary. For over a decade, site reliability engineering (SRE) teams have relied on the industry-standard ‘Four Golden Signals’ ...
