Session 1 activities
ACTIVITIES · 31 MINFictional data only
You can do every activity here. Sign in to save your responses and add your vote to the class tally.
Symptom or cause?
Open codes should name the first thing that went wrong (a cause), not how the answer felt. Sort each ShipTrack (fictional) note.
- Called get_shipment with MF-20471 when the user wrote MF-20417
- Bad answer
- Gave an ETA without calling get_eta
- Customer seemed unhappy
- Opened a handoff, then kept answering as if no handoff happened
- Agent was confusing
Trace clinic: T-017
Branching. User: 'Any update on MF-20417? It was supposed to arrive Tuesday.' The agent called get_shipment and get_eta with MF-20471, got 'Delivered, Fall River' and eta null, and replied 'Good news! Your shipment MF-20417 was delivered in Fall River.'
What is the first thing that went wrong?
Tests or evals?
Poll, then discuss.
Which statement best describes how agent evals differ from ordinary unit tests?
Where would your agent fail?
Online, after session 1.
Think of an agent or AI feature you build, buy, or support (sanitized). Name three ways it could fail, the first step where each would show up in a trace, and how costly each would be.
- Three failure modes, each a cause, not a symptom.
- Where in the trace it shows (router, tool call, retrieval, reply).
- Severity: high, medium, low.
- Which one you would test first and why.