Monitors
Writing monitors: functions that observe a step and report a suspicion score.
WarningOutline
This page is an outline: a summary and the sections planned for it, not yet the documentation. See Getting Started for a worked example.
Overview
How to write a monitor: a @monitor factory returning a function that takes a Context and a step and returns an Observation, or None to abstain. Covers the stages a monitor can watch and what each step carries, scalar and structured suspicion, writing explanations and references, when to write a monitor rather than a protocol, and grouping several functions into one monitor that shares state.
Planned Sections
- The
@monitordecorator and factories (parameters,name=,version=) - Stages: BeforeToolCall and AfterToolCall;
BeforeGenerateandAfterGenerate(planned) - What a step carries:
call,message,view,result,inputandhistory - Context: what the agent was asked to do, and why it is trusted where
stepis not - Observation: score(),
flag(), structured suspicion ({"sabotage": 0.8, ...}),references - Abstaining with
None - Monitor or protocol? Keeping thresholds out of monitors
- Several functions in one monitor: MonitorGroup
- Prompting an LLM monitor safely: separating instructions from evidence; prompt helpers such as
monitor_prompt()(planned) - Failures: what happens when a monitor raises;
fail="open"(planned) - Testing a monitor