Budding note
The structure is here, the details are still changing.
An agent is a model in a loop: it looks at the state, picks an action (often a tool call), observes the result, and repeats until it’s done.
The basic loop
- Read the goal and current context
- Decide: answer, or call a tool?
- Run the tool, append the result
- Repeat until the stop condition
Failure modes I keep seeing
- Looping: calling the same tool with the same input
- Over-confidence: stopping before the task is actually done
- Context bloat: tool outputs crowd out the original goal
Open question
What’s the simplest eval that catches all three of these?
Links
- Builds on Running LLMs locally
- Retrieval as a tool: Evaluating RAG pipelines