BR Academy

Prompt Engineering and LLM Apps in Production: Evals, Cost and Safety

Exit course

← Prompt Engineering and LLM Apps in Production: Evals, Cost and Safety

1Prompt Design and Patterns0/5

Prompt Basics and Mental Models8m
Few-Shot Learning and In-Context Examples
Chain-of-Thought and Reasoning Patterns
Role-Play and Persona Prompting
Testing Prompts with Structured Evals
Module quiz

2Structured Outputs and Tool Use0/5

Structured JSON Outputs and Parsing
Pydantic Models and Runtime Validation
Tool Use and Function Calling
Multi-Tool Routing and Agentic Loops
Hands-On Lab: Build a Multi-Tool Agent
Module quiz

3Retrieval Systems (RAG)0/5

RAG Fundamentals and Architecture
Chunking Strategies and Embedding Selection
Retrieval, Ranking, and Reranking
Measuring RAG Quality: Precision, Recall, NDCG
Hands-On Lab: Build and Evaluate a RAG System
Module quiz

4Evals and Regression Testing0/5

Eval Frameworks and Metrics
LLM-as-Judge and Semantic Scoring
Building Regression Test Suites
Detecting Prompt Drift and Model Changes
Hands-On Lab: Write Evals for Your Own System
Module quiz

5Cost, Latency, and Optimization0/5

Token Costs and Budget Planning
Latency SLOs and Tail Percentiles
Prompt Caching and KV Cache
Batching and Request Coalescing
Hands-On Lab: Optimize Cost and Latency
Module quiz

6Safety, Guardrails, and Monitoring0/5

Guardrails and Input/Output Filtering
Red-Teaming Your Own LLM System
Detecting Jailbreaks and Adversarial Inputs
Monitoring and Alerting in Production
Hands-On Lab: Add Guardrails and Monitor
Module quiz

7Capstone Project0/1

Ship an LLM Feature End to End
Module quiz

Capstone Project·Lesson 31 of 31·90 min

Ship an LLM Feature End to End

This lesson is for enrolled learners

The concept The capstone is where you put this course together in one feature you could imagine shipping. You will not be judged on breadth for its own sake. You will be judged on whether the pieces cohere: a clear user job, a specified output or tool surface, validation, evaluation with a ship rule, a cost and latency story, and……

Enroll to continue Already enrolled? Sign in