Tag: production AI
Beyond Log Search: What We Learned Building a RAG-Based Incident Diagnosis System
Abhimanyu Garg | | AI for SRE, AI hallucinations, AI incident diagnosis, AIOps, devops, devops automation, FAISS, incident management, incident remediation, incident response, knowledge management, knowledge retrieval, LLM grounding, LLM incident response, observability, postmortems, production AI, RAG AIOps, re-ranking, retrieval-augmented generation, root cause analysis, Runbooks, semantic search, SREA RAG-based AIOps framework can cut incident diagnosis time by grounding LLM reasoning in real runbooks, tickets and postmortems, improving root-cause accuracy while giving SREs source-backed answers they can trust ...
Production-Grade AI Eval Systems. What I Learned Putting LLMs on Call
Anjaiah Methuku | | AI evals, AI guardrails, AI observability, AI quality monitoring, AI reliability engineering, AI SRE, AI testing, answer relevance, CI evals, context precision, context recall, faithfulness, hallucination detection, human-in-the-loop, LLM evaluation, LLM judges, LLM reliability, model monitoring, production AI, production-grade AI, RAG evaluation, RAG metrics, regression testing, retrieval-augmented generationProduction-grade AI reliability requires more than uptime and latency. A layered eval system helps teams detect hallucinations, RAG failures and quality regressions before customers do ...
The Death of the Four Golden Signals: Designing Telemetry for Non-Deterministic Infrastructure
Jyostna Seelam | | AI architecture, AI compliance, AI Governance, AI guardrails, AI infrastructure, AI monitoring, AI observability, AI operations, AI operations strategy., AI performance metrics, AI platform monitoring, AI reliability, AI resilience, AI safety, AI SLIs, AI SLOs, AI system reliability, AI telemetry, AI trust, Enterprise AI, generative AI monitoring, guardrail intervention rate, hallucination detection, LLM monitoring, LLM observability, modern SRE, non-deterministic systems, Observability Engineering, OpenTelemetry, platform engineering, production AI, RAG observability, RAG systems, semantic drift, site reliability engineering, SRE, time to first token, token throughput, TTFT, vector database monitoringIn complex software systems, our traditional definition of operational health has always been comfortably binary. For over a decade, site reliability engineering (SRE) teams have relied on the industry-standard ‘Four Golden Signals’ ...
