16 · Meeting the real world
Capstone: out of the browser
A real provider does not take your dicts and does not hand them back. It answers with a stream of text lines in which a tool call's arguments arrive as fragments of a JSON string.
This lesson builds on lesson 15. New here? Start at lesson 01, or carry on: every lab is self-contained.
A build tool waits inside await shell.arun("sleep 600") and never looks at its signal. Three ticks into the run you call h.cancel(). A hundred ticks after that, is h.is_running still true?
cancel() sets the flag, and the loop reads the flag at the top of every turn, so the run is over
await, and the loop will not run another line until that returns.await is not a second worker; it is this worker, waiting.The flag went up and nobody read it, so stop_run fell back to the backstop and said hard. Everything in that cell was fake, and it still took ten simulated minutes to not stop. Today the slow thing is a socket, and it is the last fake left.
import asyncio, harness, lab
SYSTEM = "You are a careful assistant."
ws = lab.Workspace({})
shell = lab.Shell(ws)
def deaf_build():
"""A build tool that waits for the command and never looks at its signal."""
async def execute(arguments, signal=None):
return (await shell.arun("sleep 600"))[0]
return {"name": "build", "description": "Build the project. Takes ten minutes.",
"parameters": {"type": "object", "properties": {}}, "execute": execute}
async def consume(run):
try:
async for event in run:
pass
finally:
await run.aclose()
model = lab.ScriptedModel([lab.reply(lab.call("build")), lab.say("Built.")])
h = harness.Harness(model, SYSTEM, [deaf_build()])
task = asyncio.ensure_future(consume(h.prompt("Build it.")))
await lab.ticks(3)
h.cancel()
print("after cancel(): is_running:", h.is_running)
await lab.ticks(100)
print("100 ticks later: is_running:", h.is_running,
" record:", lab.shape(list(h.messages)), " model calls:", len(model.calls))
print("stop_run says:", await harness.stop_run(h, task, grace=5),
" and then is_running:", h.is_running)
after cancel(): is_running: True 100 ticks later: is_running: True record: U A[c1] model calls: 1 stop_run says: hard and then is_running: False
A session log has 40 lines. The last of them is a compaction entry with "first_kept_id": "e31"; the other 39 are messages. A fresh process opens the file and calls replay(). How many messages does it get?
user message beginning Previous conversation summary:. Without it the model is handed a conversation that starts in the middle of a job.replay() reads the file
replay() does.Forty lines on disk, ten messages in the harness's hands, and the kept tail still opens on a prompt and pairs every call with its result. That is three of your lessons agreeing with each other over one file. Nothing below touches any of it.
import harness, lab
ws = lab.Workspace({})
log = harness.SessionLog(ws, "session.jsonl")
for i in range(1, 18): # e1 to e34: seventeen short turns
log.append_message(harness.user_message(f"Question {i}."))
log.append_message(lab.say(f"Answer {i}."))
c1, c2 = lab.call("read", {"path": "a.py"}, id="c1"), lab.call("read", {"path": "b.py"}, id="c2")
log.append_message(harness.user_message("Compare a.py and b.py.")) # e35
log.append_message(lab.reply(lab.text("Reading both."), c1, c2)) # e36
log.append_message(lab.tool_result(c1, "A = 1\n")) # e37
log.append_message(lab.tool_result(c2, "B = 2\n")) # e38
log.append_message(lab.say("a.py sets A, b.py sets B.")) # e39
log.append_compaction("Seventeen small questions were asked and answered.", "e31") # e40
print("lines in the file:", len(log.entries()))
print("the last line: ", ws.read_text("session.jsonl").splitlines()[-1][:96] + "...")
print()
messages = log.replay()
print("replay() returns: ", len(messages), "messages")
print(lab.shape(messages))
print(lab.show(messages))
print()
print("would a provider take that list?", lab.validate(messages) == [])
lines in the file: 40
the last line: {"id": "e40", "type": "compaction", "summary": "Seventeen small questions were asked and answere...
replay() returns: 10 messages
U U A U A U A[c1,c2] R(c1) R(c2) A
user -> "Previous conversation summary:\nSeventeen small questions were asked and answered... (81 characters)"
user -> "Question 16."
assistant -> "Answer 16."
user -> "Question 17."
assistant -> "Answer 17."
user -> "Compare a.py and b.py."
assistant -> "Reading both." + toolCall c1 read({"path": "a.py"}) + toolCall c2 read({"path": "b.py"})
toolResult c1 -> "A = 1\n"
toolResult c2 -> "B = 2\n"
assistant -> "a.py sets A, b.py sets B."
would a provider take that list? True
Everything except the model
Sixteen lessons, and every model you have met was a script somebody typed. The workspace was a dict of bytes. The shell was a simulator that refused pipes and could not lose a file. The person who said yes at the gate was a list of answers. Each of those fakes was there so that one thing at a time could go wrong on purpose.
Only one of them is interesting to replace, because only one of them is a protocol. A real provider will not take the dicts your loop builds, and what comes back is not a reply but a connection that dribbles text for thirty seconds. Before any of that, the question worth a minute of your time is how much of your own code it costs you.
Tap everything that has to change, or be written, before the harness you finished last lesson can talk to a real provider.
run_agent: it is the code that asks the model for a replyrun_tooland the three toolscontext_for_modelandrepair_tool_historyHarness, and the events it yields- the message dicts themselves: roles, content blocks,
tool_call_id - one new object with an
acomplete(system, messages, tools, signal)on it - one new file that turns those dicts into a vendor's JSON, and a vendor's answer back into one of them
Nothing in harness.py. The loop calls one method on one object it was handed, and it has never known or cared what that object does. The cell below hands it an object written out in the cell itself, six lines of code, with nothing from lab inside it.
Your lesson 15 harness, byte for byte, driven by a six-line class whose replies are plain dicts written out in the cell. No lab.ScriptedModel. Look at what the harness asked of it.
import harness, lab
class Postcard:
"""Not lab.ScriptedModel, and not a real client either: six lines of code, one of them the
method your loop calls. The replies are plain dicts, written out in full, in your harness's
vocabulary."""
def __init__(self, replies):
self._replies, self._asked = list(replies), []
async def acomplete(self, system, messages, tools=(), signal=None):
self._asked.append(len(messages))
return self._replies.pop(0)
model = Postcard([
{"role": "assistant", "stop_reason": "toolUse",
"content": [{"type": "text", "text": "Let me look."},
{"type": "toolCall", "id": "toolu_01", "name": "read",
"arguments": {"path": "README.md"}}]},
{"role": "assistant", "stop_reason": "stop",
"content": [{"type": "text", "text": "The port is 9090."}]},
])
ws = lab.Workspace({"README.md": "The port is 9090.\n"})
h = harness.Harness(model, "You are a careful coding agent.", [harness.make_read_tool(ws)])
await lab.drive(h.prompt("What port does the server use?"))
print("all there is on the model object:", [n for n in dir(model) if not n.startswith("_")])
print("messages in each request: ", model._asked)
print("the file it read: ", ws.reads)
print()
print(lab.show(list(h.messages), stop_reason=True))
print()
print("the run's answer:", harness.text_of(h.messages[-1]))
print("a provider would take that transcript:", lab.validate(list(h.messages)) == [])
all there is on the model object: ['acomplete']
messages in each request: [1, 3]
the file it read: ['README.md']
user -> "What port does the server use?"
assistant -> "Let me look." + toolCall toolu_01 read({"path": "README.md"}) [stop_reason=toolUse]
toolResult toolu_01 -> "The port is 9090.\n"
assistant -> "The port is 9090." [stop_reason=stop]
the run's answer: The port is 9090.
a provider would take that transcript: True
One method. That is the whole contract between sixteen lessons of code and the outside world, and it is why this lesson edits nothing you have already written. What you write instead is an adapter.py, that knows one vendor and is the only thing in the program that does. Two pure functions, no network, no state.
to_anthropic(system, messages, tools)builds the JSON body of a request.parse_sse(lines)reads the answer back and ends with one whole assistant message.
The second is the hard one, and it is hard for a reason you have not met yet. Here is what actually comes back.
The tape
[general] A streamed reply is not delivered; it is spelled out. The connection stays open and the vendor writes lines to it until it has finished. Below is one whole reply: the model was asked what port the server uses, and it wants to read README.md first. Each of these lines really arrives with a line before it naming the same type again and a blank line after it; the two-line filter that drops those is given to you in the lab, so they are left out here. Nothing else is left out.
1 data: {"type": "message_start", "message": {"id": "msg_tool", "type": "message", "role": "assistant", "content": [], "model": "claude-opus-5", "stop_reason": null, "stop_sequence": null, "usage": {"input_tokens": 412, "output_tokens": 1}}}
2 data: {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}}
3 data: {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "I'll read"}}
4 data: {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": " the README."}}
5 data: {"type": "content_block_stop", "index": 0}
6 data: {"type": "content_block_start", "index": 1, "content_block": {"type": "tool_use", "id": "toolu_01", "name": "read", "input": {}}}
7 data: {"type": "content_block_delta", "index": 1, "delta": {"type": "input_json_delta", "partial_json": ""}}
8 data: {"type": "content_block_delta", "index": 1, "delta": {"type": "input_json_delta", "partial_json": "{\"pa"}}
9 data: {"type": "content_block_delta", "index": 1, "delta": {"type": "input_json_delta", "partial_json": "th\": \"READ"}}
10 data: {"type": "content_block_delta", "index": 1, "delta": {"type": "input_json_delta", "partial_json": "ME.md\"}"}}
11 data: {"type": "content_block_stop", "index": 1}
12 data: {"type": "message_delta", "delta": {"stop_reason": "tool_use", "stop_sequence": null}, "usage": {"output_tokens": 41}}
13 data: {"type": "message_stop"}
Read line 8 again, and then 9 and 10; the rows run off the side of the page, so drag one to read it out. The arguments of the call arrive as pieces of a JSON string, split wherever the model happened to pause. {"pa is not a dict, and it is not half a dict either; it is four characters.
Your program is reading that tape line by line, and a person is watching the screen. Tap two lines: the first at which you could show them something, and the first at which you could run something.
- 1 — the reply has begun
- 2 — a block of text has begun
- 3 —
text_delta"I'll read" - 6 — the call's name and id are known:
read,toolu_01 - 10 — the last fragment arrives, and the pieces spell
{"path": "README.md"} - 11 —
content_block_stopfor block 1 - 13 — the reply is over
Text can go on the screen the moment it exists, because a sentence that is half written is still half a sentence. A call cannot be run until its block ends, because until then nobody has said that the pieces are all in: at line 10 the string happens to parse, and the only thing that tells you no fifth fragment is coming is line 11. The format has a name, parse_sse yields.
The same stream, through the reference parse_sse, with the number of the line that produced each event. Once lab 2 has passed, this runs on yours.
import lab_fixtures
# A cell always loads the file you are editing under the name `harness`; here that file is
# adapter.py, so this is your parse_sse (the reference one until lab 2 passes).
from harness import parse_sse
seen = [0]
def tape(lines):
"""The stream from the page, counting the data: lines as they are handed over."""
for line in lines:
seen[0] += line.startswith("data:")
yield line
def short(event):
if "delta" in event:
return repr(event["delta"])
if "call" in event:
return f"{event['call']['name']} {event['call']['arguments']}"
if "message" in event:
return f"stop_reason={event['message']['stop_reason']!r}"
return ""
print("data line event")
for event in parse_sse(tape(lab_fixtures.anthropic_tool_call)):
print(f"{seen[0]:>9} {event['type']:<14} {short(event)}")
data line event
1 start
3 text_delta "I'll read"
4 text_delta ' the README.'
6 toolcall_start read {}
7 toolcall_delta ''
8 toolcall_delta '{"pa'
9 toolcall_delta 'th": "READ'
10 toolcall_delta 'ME.md"}'
11 toolcall_end read {'path': 'README.md'}
13 done stop_reason='toolUse'
Six kinds of event come out: start, text_delta, toolcall_start, toolcall_delta, toolcall_end, and at the end exactly one done. toolcall_start exists so that a screen can show read(…) spelling itself out; toolcall_end is the only one a caller may run anything on, and it carries the finished block. That is your two tapped lines, as an interface.
Which leaves the question of what happens when the pieces do not spell anything.
Same call, different model. The fragments arrive as {"path": , README, .md}, and joined up they are {"path": README.md}, which no JSON parser will take: the value has no quotes. Your parse_sse is holding that text and the block has just ended. What should it do?
read was handed a path that was not a string. What is new here is who pays: raising inside the stream takes down a run that has already been billed for, and the reply the model did write is never written down.The call travels with arguments = {"_raw_arguments": '{"path": README.md}'} — run_tool hands that dict to read, str_arg finds no string under path and raises, the boundary turns it into one result marked is_error, and the model reads path must be a string and writes the call again properly. A bad guess costs one turn instead of the whole run, and nobody needed a new mechanism for it.
Where this failure comes from: Tau's accumulator does the same, in two lines — parse the joined fragments, and when they do not parse, keep them under _raw_arguments (src/tau_ai/anthropic.py:413-422).
import harness, lab
ws = lab.Workspace({"README.md": "The port is 9090.\n"})
tools = [harness.make_read_tool(ws)]
def read_the_error(request):
"""A model that looks at the result it just got, as a real one does."""
last = request.last_result
if last is not None and last["is_error"]:
return lab.reply(lab.text("Quoting it, then."),
lab.call("read", {"path": "README.md"}, id="toolu_01"))
return lab.say("The port is 9090.")
model = lab.ScriptedModel([
# what the adapter passes on when the model's arguments did not parse as JSON
lab.reply(lab.text("Reading it."),
lab.raw_args("read", '{"path": README.md}', id="toolu_04")),
lab.forever(read_the_error),
])
h = harness.Harness(model, "You are a careful coding agent.", tools)
await lab.drive(h.prompt("What port does the server use?"))
print(lab.show(list(h.messages)))
print()
print("model calls:", len(model.calls), " input tokens billed:", model.bill,
" files read:", ws.reads)
print("the record a provider would be sent next:", lab.validate(list(h.messages)) == [])
user -> "What port does the server use?"
assistant -> "Reading it." + toolCall toolu_04 read({"_raw_arguments": "{\"path\": README.md}"})
toolResult toolu_04 -> "path must be a string" [is_error]
assistant -> "Quoting it, then." + toolCall toolu_01 read({"path": "README.md"})
toolResult toolu_01 -> "The port is 9090.\n"
assistant -> "The port is 9090."
model calls: 3 input tokens billed: 644 files read: ['README.md']
the record a provider would be sent next: True
Three ways for a reply to end badly
A stream is a connection, and connections end in ways an answer never did. The model can run out of room halfway through a call. The vendor can send an error after it has already sent words. The connection can simply stop, mid-sentence, with nothing to say about it. Your loop has one shape for all of that, from lesson 5: a failed turn is a message on the record, never an exception.
This reply hits the cap on its own output halfway through a call's arguments: the vendor says "stop_reason": "max_tokens", and the fragments so far spell {"path": "READ. The block never gets its closing brace. What does the adapter hand your loop, and what does the loop then do?
length, with the half-written call in it. That is what the vendor said about this reply, and the adapter's job is to pass it on faithfully
stop_reason, from the renderer to the person reading the session tomorrow, would be told the model ran out of room when it was in the middle of asking for a tool.toolUse holding a call whose arguments are the half-written text, and the loop runs it, fails it and lets the model try again
error reply with the unfinished call left out. The turn did not complete, so there is nothing valid to act on
The block that the stream never closed is closed by the adapter, so the arguments are kept as raw text, and because the reply holds a call the label comes out toolUse whatever the vendor wrote. length would have been honest about the writing and wrong about the reply. Your loop then does exactly what the last cell showed: one wasted turn, one error the model can read.
Where this comes from: Tau recomputes the stop reason the same way, and a reply holding calls is toolUse whatever arrived on the wire (src/tau_ai/stream.py:80-85).
import lab_fixtures
from harness import parse_sse # a cell loads adapter.py under the name `harness`
events = list(parse_sse(iter(lab_fixtures.anthropic_truncated)))
message = events[-1]["message"]
why = next(l for l in lab_fixtures.anthropic_truncated if '"message_delta"' in l)
print("what the vendor said about the ending:")
print(" ", why)
print()
print("events: ", [event["type"] for event in events])
print("stop_reason:", repr(message["stop_reason"]))
for block in message["content"]:
print("block: ", block)
what the vendor said about the ending:
data: {"type": "message_delta", "delta": {"stop_reason": "max_tokens", "stop_sequence": null}, "usage": {"output_tokens": 16}}
events: ['start', 'text_delta', 'toolcall_start', 'toolcall_delta', 'toolcall_end', 'done']
stop_reason: 'toolUse'
block: {'type': 'text', 'text': 'Reading it now.'}
block: {'type': 'toolCall', 'id': 'toolu_05', 'name': 'read', 'arguments': {'_raw_arguments': '{"path": "READ'}}
The other two endings. In one, the vendor sends an error event down a connection whose HTTP status was 200, after some words have already been shown. In the other, the lines simply stop: no error, no ending, nothing. How many events of the kind that ends a stream should each of those yield?
One, always, and it is the last thing yielded — the acomplete takes the assistant message out of that event, and the loop appends that message and reads its stop_reason. None means a caller left holding nothing and a run that raises instead of recording what happened; two means the message that gets kept is not the message the model sent. When the vendor never sends one, the adapter invents it.
Both endings, through the reference parse_sse: the events, how many of them are terminal, and the message each one carries. Watch what is inside the message in each case.
import lab_fixtures
from harness import parse_sse # a cell loads adapter.py under the name `harness`
for name in ("anthropic_overloaded", "anthropic_cut"):
stream = getattr(lab_fixtures, name)
last = [line for line in stream if line][-1]
print(f"{name}: {len(stream)} lines, ending {last[:72]}")
events = list(parse_sse(iter(stream)))
print(" events: ", [event["type"] for event in events])
print(" terminals:", sum(event["type"] in ("done", "error") for event in events))
message = events[-1]["message"]
print(" message: ", {key: message[key] for key in ("content", "stop_reason",
"error_message")})
print()
anthropic_overloaded: 12 lines, ending data: {"type": "error", "error": {"type": "overloaded_error", "message":
events: ['start', 'text_delta', 'error']
terminals: 1
message: {'content': [{'type': 'text', 'text': 'Let me'}], 'stop_reason': 'error', 'error_message': 'Overloaded'}
anthropic_cut: 12 lines, ending data: {"type": "content_block_delta", "index": 0, "delta": {"type": "tex
events: ['start', 'text_delta', 'text_delta', 'error']
terminals: 1
message: {'content': [{'type': 'text', 'text': 'The answer is'}], 'stop_reason': 'error', 'error_message': 'stream ended without a terminal event'}
Both keep the words that were already on the screen, which is the point: the user read them, and a record that disagrees with the screen is a record that lies. The invented one says so in its error_message, so that whoever reads the session tomorrow can tell a model that stopped from a connection that died.
Where this comes from: Tau ends every stream the same way. If nothing terminal arrived, it makes one, with the text Provider stream ended without a terminal event (src/tau_ai/stream.py:225-232), and the loop has a second net under that (src/tau_agent/loop.py:141-144).
The other direction
That is the answer coming back. Going out, the job is smaller and stranger: the same conversation, said in someone else's words. Here is a transcript you have seen a hundred times — a prompt, a reply with two calls in it, and the two results.
user "Compare a.py and b.py."
assistant "Reading both." + toolCall toolu_02 read(a.py) + toolCall toolu_03 read(b.py)
toolResult toolu_02 "A = 1"
toolResult toolu_03 "File not found: b.py" [is_error]
One thing is missing from that listing. While the two reads were running, the user typed "Forget b.py, it was deleted. Just describe a.py.", and lesson 9 held it until the run reached a place where it could land. This vendor, meanwhile, knows two roles, user and assistant, and no third one.
Those four messages and the steer, as the five messages of the request body. Put them in the order they go out in.
"user":"Compare a.py and b.py.""assistant": a text block and twotool_useblocks"user": onetool_resultblock,tool_use_idtoolu_02"user": onetool_resultblock,tool_use_idtoolu_03"user":"Forget b.py, it was deleted. Just describe a.py."
With only two roles to spend, a result is something the user says, and the steer lands after both of them: four of the five wire messages are user, three of them in a row. One wire message per message of the transcript, in the same order, nothing joined together — it looks wrong the first time you see it and is exactly right. The results are paired to their calls by tool_use_id, not by position, which is lesson 2's rule surviving a change of vocabulary. [general] Hosted APIs tend to take consecutive turns of one role and join them as they read; the adapter does not do it in advance, because merging is where a result and the steer after it become one message and stop meaning what they meant.
That transcript through the reference to_anthropic: the roles that go out, every wire message in full, and the rest of the body. Once lab 1 has passed, this runs on yours.
import json
import lab, lab_fixtures
from harness import to_anthropic # a cell loads adapter.py under the name `harness`
case = lab_fixtures.request_two_calls_and_steer
body = to_anthropic(case["system"], case["messages"], case["tools"])
print("the transcript:", lab.shape(case["messages"]))
print("wire roles: ", [message["role"] for message in body["messages"]])
print()
for index, message in enumerate(body["messages"]):
print(index, json.dumps(message))
print()
print("system:", json.dumps(body["system"]))
print("tools: ", json.dumps(body["tools"]))
print("and the rest of the body:", {k: v for k, v in body.items()
if k not in ("messages", "system", "tools")})
the transcript: U A[toolu_02,toolu_03] R(toolu_02) R(toolu_03) U
wire roles: ['user', 'assistant', 'user', 'user', 'user']
0 {"role": "user", "content": "Compare a.py and b.py."}
1 {"role": "assistant", "content": [{"type": "text", "text": "Reading both."}, {"type": "tool_use", "id": "toolu_02", "name": "read", "input": {"path": "a.py"}}, {"type": "tool_use", "id": "toolu_03", "name": "read", "input": {"path": "b.py"}}]}
2 {"role": "user", "content": [{"type": "tool_result", "tool_use_id": "toolu_02", "content": "A = 1\n", "is_error": false}]}
3 {"role": "user", "content": [{"type": "tool_result", "tool_use_id": "toolu_03", "content": "File not found: b.py", "is_error": true}]}
4 {"role": "user", "content": "Forget b.py, it was deleted. Just describe a.py."}
system: "You are a careful coding agent."
tools: [{"name": "read", "description": "Read a text file from the workspace.", "input_schema": {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}}]
and the rest of the body: {'model': 'claude-opus-5', 'max_tokens': 16000, 'stream': True}
Three things in that output are worth a second look. The system prompt is a field of the body, not a message with a role. A tool is advertised as name, description and input_schema — the same schema you wrote in lesson 2, under a different key. And stop_reason, usage, error_message and tool_name are nowhere: they are notes this program keeps for itself, and [general] a hosted API tends to refuse a request carrying a field it has never heard of rather than ignore it.
A vendor's own JSON, as against your neutral dicts, is its
| Same transcript, other vendor | Anthropic Messages | OpenAI chat completions |
|---|---|---|
| where a tool result goes | a tool_result block inside a user message | its own message, "role": "tool" |
| what pairs it to its call | tool_use_id | tool_call_id |
| a call's arguments | a dict, input | a JSON string, arguments |
| the system prompt | a field of the body | a message with "role": "system" |
Tau writes both: the tool role and the arguments dumped back to a string (src/tau_ai/openai_compatible.py:1221-1227, src/tau_ai/openai_compatible.py:1242-1250) beside the Anthropic pair you are about to write (src/tau_ai/anthropic.py:652-672). Neither is your message model. That is the reason your message model exists: two vendors cannot both be the way you store things, so neither of them is.
Build: the one file that knows a vendor
Everything above was about a boundary, so the code goes in a file of its own. adapter.py imports json and nothing else — not harness.py, and certainly not the other way round. Your harness is beside it in the editor, read-only, so that you can see it does not change.
Figure 16.1 to_anthropic: three messages of the record on the left, the JSON that goes out on the right, row for row. The tool result changes role on the way and keeps its id.
Write to_anthropic(system, messages, tools), about 20 lines, returning the dict that json.dumps will send as the body of the request. messages is what the loop sends, context_for_model(record); tools is tool_specs(tools). One wire message per transcript message, in order, nothing merged.
- a
usermessage keeps its role and its plain text; - an
assistantmessage keeps its role and translates its blocks: a text block stays a text block, and atoolCallbecomes{"type": "tool_use", "id", "name", "input": <the arguments dict>}; - a
toolResultbecomes ausermessage holding one{"type": "tool_result", "tool_use_id", "content", "is_error"}block; - the body also carries
model,max_tokens,stream,systemandtools, each tool as{"name", "description", "input_schema"}.
Two rules with teeth. A role, or a block type, that you have no wire form for is a bug in whoever called you: raise, rather than send a request the transcript does not mean. And build new dicts — the list you were handed is the harness's record, and the session log, the next request and lab.validate all read it after you.
Eleven hidden tests. Two of them compare the whole body against an expected request written out in full; the rest are about one decision each, including the one that looks wrong.
- The body has six keys and only one of them is interesting. Start there: a list you build by walking
messagesonce. How many branches does that walk need, and what is the default one? - One
forovermessages, appending exactly one wire message each time, with a branch per role and araiseat the end of the chain. An assistant's blocks want a helper of their own, with the same shape: one branch per block type, and araiseunder them. Then return the body with the two constants at the top of the file, the system prompt as a field, and the tools rebuilt withparametersrenamed. - In outline.
wire = []; for each message:userappends{"role": "user", "content": message["content"]};assistantappends{"role": "assistant", "content": [...]}with each block through your helper;toolResultappends a"user"message whose content is a one-item list holding thetool_resultblock, built fromtool_call_id,contentandis_error; anything else raisesValueError. The helper: atextblock becomes{"type": "text", "text": ...}, atoolCallbecomes thetool_useblock, anything else raises. Return{"model": MODEL, "max_tokens": MAX_TOKENS, "stream": True, "system": system, "tools": [...], "messages": wire}.
Twenty-odd lines, and your transcript is now something a real API would accept. Nothing in that file knows what a loop is, what a tool does, or that a network exists; it is a translation between two ways of writing the same conversation down, and it is the only place in the program where the word tool_use appears.
- One user message, one tool: the body is exactly the expected request, with the model, max_tokens, stream, the system prompt as a field of its own, and the tool's schema under input_schema.
- A transcript with text, a two-call turn, a result, an error result and a steer translates to exactly the expected request body.
- Nothing is merged: two results and a steer are three consecutive user messages, in transcript order.
- A toolResult goes out as a user message holding one tool_result block that names its call with tool_use_id. This vendor has no "tool" role.
- is_error travels: a result with is_error True goes out with "is_error": true.
- A toolCall block becomes a tool_use block whose input is the arguments dict itself, not a JSON string (that is the other vendor).
- stop_reason, usage, error_message, tool_name and the neutral block names stay home: a wire message has only role and content.
- Each tool goes out as name, description and input_schema, and nothing else; no tools is an empty list.
- The system prompt goes out as the body's "system" field; the wire messages begin with the user's.
- A message role, or a block type, that the adapter has no wire form for stops the request; it is never quietly left out of it.
- Translating is reading: after to_anthropic the transcript is exactly what it was, so the harness's record still speaks the neutral model.
Build: read the tape
Now the other direction, in the same file. The worked half is given to you and marked in the diff: the data: filter, the first event, text blocks and their deltas, and the line that reads why the model stopped. What is left is everything to do with a call, and the promise about endings.
Figure 16.2 parse_sse: the lines on the left, the event each one yields on the right. Badge 1 is the first moment text may be shown. Badge 2 is the terminal event, and the note beside it is about your loop, which takes its calls out of the finished message rather than out of toolcall_end — which is why a reply that turns out to have failed never has its calls run.
Finish parse_sse(lines), about 28 lines in three marked gaps. It takes an iterator of lines without their newlines, exactly as a reader hands them over, and yields the six kinds of event the cells above printed — the last kind being the terminal one, done or error.
- State, above the loop.
parts: the argument fragments of each call still being spelled out, kept by block index, because two calls can open one after the other in one reply.close(index): the block is finished, so join its fragments and parse them once, here; the result goes into that block's"arguments"and the event{"type": "toolcall_end", "call": <the block>}comes back. Text that is not JSON is neither raised nor thrown away: it travels as{"_raw_arguments": <the joined text>}. No fragments at all means{}— a tool that takes no arguments is not a parse failure.message(stop, error=None): the whole assistant message the terminal event carries, withcontent,usageandstop_reason, pluserror_messagewhen there was one. It closes any block the stream never closed, and if any block is atoolCallthe reason istoolUse— unless the reply failed, which keeps the reason it failed for. - Five more branches on the chain the starter begins: a
content_block_startthat is atool_useopens a call with empty arguments; aninput_json_deltais one fragment of them; acontent_block_stopfor a block that is a call yields whatclosegives you;message_stopis the good ending anderrorthe bad one, and after either you stop reading. Anything else is not an event of ours. - After the loop. The lines ran out and nothing terminal was yielded: the connection died mid-reply. Yield the
errorthe vendor never sent, keeping whatever content there is.
Twelve hidden tests, and the last three are not about parse_sse at all: they build a provider out of your two functions and one of those streams, and drive your unchanged lesson 15 harness through it — a call, its result, a final answer; then a model whose arguments do not parse, correcting itself; then a stream that fails, twice over, with the session still answering the next prompt.
- Three questions, one per gap. In
close: the fragments are text and the block wants a dict — what are the two ways that conversion can go, and which of them is an error for the model rather than for you? Inmessage: which is more trustworthy about why a reply stopped, the vendor's word or what is in the reply? After the loop: how does the code that called you tell "the model finished" from "the wire went quiet", if the only thing it ever sees is events? partsis a dict from block index to a list of strings, and both helpers are closures over it,contentandusage.close(index)pops that index, joins, and triesjson.loadsinside atry; onValueErrorthe dict is{"_raw_arguments": <the joined text>}; either way the block already sitting incontentgets it, and the event carries that same block.messagecloses what is still open by callingcloseon every index left inparts, then decides the label. The five branches all readdata, and the two endingsreturnafter they yield. The invented terminal is the last statement in the function, outside thefor: if it is reached, no ending was sent.- In outline.
parts = {}.close(index):raw = "".join(parts.pop(index));content[index]["arguments"] = json.loads(raw) if raw else {}, and onValueError{"_raw_arguments": raw}; return thetoolcall_endevent carryingcontent[index].message(stop, error=None):for index in list(parts): close(index);calls= any block of typetoolCall; the message isrole,content,usageandstop_reason, which isstopif there was an error, else"toolUse"ifcalls, elsestop; adderror_messagewhenerror. Branches, in the chain after the given ones:content_block_start(whatever is left is atool_use) appends thetoolCallblock, opensparts[data["index"]] = []and yieldstoolcall_start;content_block_deltawith aninput_json_deltaappendsdata["delta"]["partial_json"]to that index and yieldstoolcall_delta;content_block_stopwithdata["index"] in partsyieldsclose(data["index"]);message_stopyieldsdonewithmessage(stop_reason)and returns;erroryieldserrorwithmessage("error", data["error"]["message"])and returns. Last line of the function: yield oneerrorevent withmessage("error", <say what happened>).
Twenty-eight lines, and the last three tests are the point of the whole course: your lesson 15 harness, not one byte of it different, fed by a stream of text lines instead of by a dict somebody typed. The fake kept the contract, so the wire fits where the fake was.
What changed since the previous lab
The line-by-line diff needs JavaScript. The whole file this exercise starts from is printed at the end of it.
- A text-only stream, with a ping and non-data lines in it, yields one start, one text_delta per piece, and one done carrying the whole sentence.
- A call whose arguments arrive as four fragments of a JSON string is announced when it opens, once per fragment, and once more when it is whole; only then are its arguments {"path": "README.md"}.
- Two calls in one reply end up as two toolCall blocks, in order, each with the arguments its own fragments spelled out.
- Arguments the model got wrong ({"path": README.md} is not JSON) neither raise nor vanish: they arrive as {"_raw_arguments": <the text>}.
- A tool that takes nothing sends no argument fragments at all, and its call arrives with arguments {} rather than with empty text kept as raw.
- The output cap, or a dead connection, can stop a reply halfway through a call's arguments: the block is still closed, its text is kept, and the stream still ends in exactly one terminal.
- A reply can fail after a call has begun, and half a call is not a call: the reply keeps the label 'error', and the loop above never runs what the model did not finish asking for.
- A provider can fail after the answer has begun, on a connection whose HTTP status was 200: that is one error event, carrying the vendor's words and the text the user has already seen.
- A dropped connection sends no error and no message_stop, it simply ends: the adapter invents the terminal event the vendor never sent, and keeps the partial answer in it.
- The payoff: a model whose network is a recording, driven through your to_anthropic and parse_sse, runs the lesson 15 harness end to end -- a tool call, its result, a final answer -- with nothing in harness.py changed.
- End to end: the model writes arguments that are not JSON, str_arg refuses them, the error goes back as that call's one result, and the model's corrected second call is answered.
- End to end: an error event, and a connection that simply dies, each end the run with one failed assistant message in the record -- no exception escapes -- and the session goes on to answer the next prompt.
A provider whose network is two of those streams, built out of your two functions and nothing else. The crank is turned by hand here, as it was in lesson 3, because a cell can only load one of your two files at a time — the version that drives your real loop is the last hidden test above, and page 2 runs it on your machine. No tests: read the two replies, the usage that came down the wire with them, and the bytes of the second request.
import json
import lab, lab_fixtures, lab_wire
from harness import parse_sse, to_anthropic # a cell loads adapter.py under the name `harness`
READ = {"name": "read", "description": "Read a text file from the workspace.",
"parameters": {"type": "object", "properties": {"path": {"type": "string"}},
"required": ["path"]}}
SYSTEM = "You are a careful coding agent."
ws = lab.Workspace({"README.md": "The port is 9090.\n"})
# A provider whose network is two of the page's streams, through YOUR two functions and nothing else.
model = lab_wire.ReplayModel([lab_fixtures.anthropic_tool_call, lab_fixtures.anthropic_text],
to_anthropic, parse_sse)
messages = [lab.user("What port does the server use?")]
reply = await model.acomplete(SYSTEM, messages, [READ])
messages.append(reply)
print(lab.show([reply], stop_reason=True, usage=True))
call = reply["content"][-1] # turning the crank by hand, as in lesson 3
messages.append(lab.tool_result(call, ws.read_text(call["arguments"]["path"])))
messages.append(await model.acomplete(SYSTEM, messages, [READ]))
print(lab.show(messages[-2:], stop_reason=True, usage=True))
print()
print("what went out on the wire the second time:")
for message in model.requests[1]["messages"]:
print(" ", json.dumps(message))
print()
print("transcript:", lab.shape(messages), " valid:", lab.validate(messages) == [])
assistant -> "I'll read the README." + toolCall toolu_01 read({"path": "README.md"}) [stop_reason=toolUse] [usage: {"input": 412, "output": 41, "cache_read": 0}]
toolResult toolu_01 -> "The port is 9090.\n"
assistant -> "The port is 9090." [stop_reason=stop] [usage: {"input": 25, "output": 9, "cache_read": 0}]
what went out on the wire the second time:
{"role": "user", "content": "What port does the server use?"}
{"role": "assistant", "content": [{"type": "text", "text": "I'll read the README."}, {"type": "tool_use", "id": "toolu_01", "name": "read", "input": {"path": "README.md"}}]}
{"role": "user", "content": [{"type": "tool_result", "tool_use_id": "toolu_01", "content": "The port is 9090.\n", "is_error": false}]}
transcript: U A[toolu_01] R(toolu_01) A valid: True
Look at the second request. It carries the conversation so far, the tool's output as something the user said, and a tool called read that a dict in your lesson 2 file describes. Nothing in it was assembled by a framework; you can point at the line that built every key.
- Hold a conversation: it knows what was said, and who said it.
- Run a tool the model asks for and show it the result.
- Keep going until the model stops asking.
- Tell the model when a tool fails, and carry on.
- Stop a runaway, and survive a provider failure.
- Keep every tool result within a budget.
- Report what it is doing, as events, to any frontend.
- Own its transcript: one writer at a time.
- Take your input mid-run, at a safe point.
- Send a valid transcript even after an interruption.
- Survive a power cut: an append-only log, resumed by replay.
- Outlast the context window: summarise the old, keep the recent word for word, delete nothing.
- Brief the model from what is really there: the enabled tools, the project's files, an index of skills.
- Refuse a tool call at the door, in your code, and hand the model the reason.
- Be stopped mid-build, politely or not, and be usable a second later.
- Talk to a real provider: one file of translation, and a stream that always ends in one message.
Go deeper: showing the words as they arrive
Your loop takes the finished message from acomplete, so a person watching sees nothing for thirty seconds and then a paragraph. The deltas are already there; they are just being thrown away inside the provider object. Six lines in run_agent forward them: ask the model for astream instead, yield {"type": "message_start"} when the reply begins and {"type": "message_update", "delta": …} for each piece of text, and take the assistant message from the terminal event as before. Nothing else in the file changes, because an event nobody handles is ignored by every frontend you wrote.
Tau has both events for this, with the whole partial message on each one rather than the fragment (src/tau_agent/events.py:34-45).
Figure 16.3 Lesson 7's figure, with the optional step's two events added before each assistant message. Nothing else about the run changed.
agent_startandagent_endenclose the run, andagent_endcarries only this run's new messages.- Two turns, each between a
turn_startand aturn_end. - One
message_endfor every message appended, tied to its slip: the message is already on the record when its event is yielded. tool_execution_startandtool_execution_endenclose the result's ownmessage_end: recorded and announced before the execution ends.- The optional step:
message_start, then onemessage_updateper delta, before each assistantmessage_end.
Say it in your own words
A colleague has been building their agent against a real model from day one, and says the fake was a waste of your time: you had to write the real thing in the end anyway. In a sentence or two, what did keeping the fake buy you, and what did it not buy you?
Count the lines you changed in harness.py today. Then ask what would have had to be true of the fake for that number to be zero.
The fake and the real provider answer the same method with the same shapes, so the code between them never had to know which one it had; that is the only reason today was two new functions and no edits. What it did not buy you is any evidence about the model. Every green test in this course pins your side of the contract, and a scripted reply proves nothing about what a real model will ask for, how often it will get it wrong, or whether the answer is any good.
Common answers, and what each one misses
- "It let me test without spending money." True, and the smallest of the three. Free is nice; repeatable is what let lesson 10 script a damaged transcript and lesson 15 stop a run at every event position in turn.
- "It kept the real provider's complexity out of the lessons." It is the other way round: the fake made the complexity visible one piece at a time, and today you met the pieces the fake could not show — fragments, endings, two vocabularies.
- "It proved the design." It proved the harness holds its contract. The design is also a bet about a model's behaviour, and nothing you have run so far could win or lose that bet. Page 2 is where you find out.
Only the edges changed. The harness you built against a fake runs against the real thing, because the fake kept the same contract.
Tau makes the same promise your last gap does: however a provider's stream ends, or fails to end, a terminal event always comes out of it carrying a whole assistant message, and where the vendor sent nothing terminal Tau writes that event itself.
yield {"type": "error",
"message": message("error",
"stream ended without a terminal event")}
if not started:
yield AssistantStartEvent(partial=_snapshot(partial))
if not terminal:
error = partial.model_copy(deep=True)
error.stop_reason = "error"
error.error_message = "Provider stream ended without a terminal event"
error.usage = Usage()
yield AssistantErrorEvent(reason="error", error=error)
The same parts, piece by piece. Tau's adapter for this vendor is one file, src/tau_ai/anthropic.py, and the two halves you wrote are both in it.
- The contract the loop depends on is a
Protocolwith one method (src/tau_agent/provider.py:19-37), and Tau's fake satisfies it structurally, with no base class and no registration (src/tau_ai/fake.py:13-41). That is the first cell on this page, typed by somebody else. - Outbound: one wire message per message, a user message as plain text (
src/tau_ai/anthropic.py:610-618), a tool result as atool_resultblock in ausermessage (src/tau_ai/anthropic.py:652-672), a tool asname,description,input_schema(src/tau_ai/anthropic.py:687-699). - Inbound: the
data:filter, in a function of its own (src/tau_ai/anthropic.py:702-707); fragments collected per block index and parsed once at the end, with the same raw fallback (src/tau_ai/anthropic.py:272-280,src/tau_ai/anthropic.py:413-422); content beating the vendor's label (src/tau_ai/stream.py:80-85). - And the same idea for the other vendor, in a second file of its own:
"role": "tool", and arguments dumped back into a string (src/tau_ai/openai_compatible.py:1242-1250). Two adapters, one loop, and the loop imports neither.
What Tau adds. Retries, for a start. Yours has none; Tau retries a status that suggests the request might work next time (src/tau_ai/anthropic.py:377-380) and waits between attempts in small steps so that a thirty-second backoff does not swallow a Stop (src/tau_ai/retry.py:46-62). There is one condition on all of it that is easy to miss, and it is the first question below. Both questions are about Tau's code, so nothing waits on them.
A reply is streaming, and the words are going on the screen as they arrive. Two deltas in, the connection drops. The client retries the request, which is the same request, so the model writes the same answer again. What does the person watching see?
Which is why Tau's retry after a connection failure asks one more question first: has anything been emitted yet? (src/tau_ai/anthropic.py:345) Before the first delta a retry is invisible; after it, a retry is a second answer glued to half of the first. A harness that never shows deltas can retry more freely, and pays for that freedom elsewhere.
ANSWER = ["The bug is ", "in const.py", ", line 3."]
def attempt(drops_after):
"""One streamed reply. It sends its pieces until the connection drops."""
sent = []
for index, piece in enumerate(ANSWER):
if index == drops_after:
return sent, "the connection dropped"
sent.append(piece)
return sent, None
screen = []
pieces, failure = attempt(drops_after=2)
screen += pieces
print(f"attempt 1 ({failure}), on the screen:")
print(" ", repr("".join(screen)))
pieces, failure = attempt(drops_after=None) # the client retries the request from the start
screen += pieces
print("attempt 2 (it works), on the screen:")
print(" ", repr("".join(screen)))
attempt 1 (the connection dropped), on the screen:
'The bug is in const.py'
attempt 2 (it works), on the screen:
'The bug is in const.pyThe bug is in const.py, line 3.'
Tau also carries thinking blocks, with their provider-owned signatures, and refuses to replay one vendor's signature to another (src/tau_ai/anthropic.py:629-641); its parser is two stages, a vendor reader and a canonicaliser, with about a dozen event types and a content_index on each (src/tau_ai/stream.py:88-232); and it can replay one provider's history to another, which needs call ids that survive the trip.
A conversation started at a vendor whose call ids look like call|7, and is being continued at one that takes only letters, digits, - and _. The obvious fix is to replace the characters that are not allowed. What is wrong with it?
call|7 put something of its own in there, and after the replacement nobody can read it back
call|7 and call_7 both become call_7. Tau hashes anything that does not already fit the safe alphabet instead, and leaves ids that do fit exactly as they are, so same-vendor replay is untouched (src/tau_ai/tool_call_ids.py:14-24). A one-way function is the cheap way to be sure two different inputs stay two different things.
import re
from hashlib import sha256
def sanitise(call_id):
"""The obvious fix: replace every character the vendor will not take."""
return "".join(ch if re.fullmatch(r"[A-Za-z0-9_-]", ch) else "_" for ch in call_id)
def portable(call_id):
"""Tau's fix: leave a safe id alone, and hash anything else."""
if re.fullmatch(r"[A-Za-z0-9_-]{1,64}", call_id):
return call_id
return "tc_" + sha256(call_id.encode("utf-8")).hexdigest()[:40]
native = ["call|7", "call_7"] # two different calls, one session, one turn
print("as the other vendor wrote them:", native)
print("replacing the bad character: ", [sanitise(one) for one in native],
" still two ids:", len({sanitise(one) for one in native}) == 2)
print("hashing it: ", [portable(one) for one in native],
" still two ids:", len({portable(one) for one in native}) == 2)
as the other vendor wrote them: ['call|7', 'call_7'] replacing the bad character: ['call_7', 'call_7'] still two ids: False hashing it: ['tc_173e7c5a009a660b724e0d7ad421cce697580c6d', 'call_7'] still two ids: True
Where yours is weaker. No retries at all: a 429 ends your run with a message you can read, and nothing tries again. One vendor, one API version, and the two constants at the top of your file, a model name and an output cap, are this lesson's choice rather than the vendor's. No prompt caching, so you pay full price for the whole transcript every turn. No thinking blocks — a block type your parser cannot read takes an empty place in content so that later deltas still find their own block by index, which keeps the rest of the reply honest and is not the same as handling it. That gap is the first thing page 2 makes you look at.
Where Tau is weaker. One thing, small and real, and the question above is what found it. emitted_content is the flag that decides whether a dropped connection may be retried, and a tool call sets it twice over: once when the call opens (src/tau_ai/anthropic.py:239-248) and once for every fragment of its arguments (src/tau_ai/anthropic.py:272-280). Neither of those yields an event, so nothing has left the parser and nothing can have reached a screen — and yet, in a reply that opens straight into a call, a connection that dies halfway through {"path": "READ is not retried, where the same failure one line earlier, before the call's block opened, would have been. A retry there would be invisible. The flag answers "has anything been parsed" and is read as "has anything been shown".
Where the ideas stop being small. The ideas in this course really are small — the loop is nine lines, the adapter is two functions — while the program that ships them is not. Tau's session layer is 5,095 lines (src/tau_coding/session.py:5095) and its terminal is 8,647 (src/tau_coding/tui/app.py:8647). None of that is in tau_agent, which is the part you have been rebuilding, and which is smaller than either.
src/tau_ai/stream.py:225-232 · pinned to commit 9fe6a71 · view on GitHub
Tomorrow you want to run the same harness against OpenAI's chat completions API instead. Name the functions you have to write, and the lines of harness.py you have to change. Be exact about the second number.
The table above lists four differences between the two vendors. Which of your files would have to learn any of them?
Two functions and zero lines. A second file beside adapter.py with its own to_openai and its own stream reader, handed to the same object that owns the socket, and the loop cannot tell. The reason zero is the right answer is that no word from either vendor's vocabulary — tool_use, tool_use_id, input_schema, partial_json, data: — appears anywhere in harness.py; if one of them had leaked in during lesson 2, the answer today would be "everywhere".
Common answers, and what each one misses
- "Two functions, and a small change where the arguments get parsed." That vendor sends arguments as a JSON string rather than a dict, so it is the new stream reader that parses it — in exactly the place yours already parses one, at the end of the block. The message your loop receives has a dict, as it always did.
- "Two functions, plus a flag to say which vendor is in use." A flag inside the harness is the vendor arriving in the harness. The choice belongs where the object is built, which is one line of the program that starts everything, and page 2 is where you read it.
- "Rewrite the message model to match theirs." Then the other vendor breaks, and so does every session file you have already written. The neutral model is what makes the answer zero.
- You hit
- a provider that takes none of your dicts and returns none of them, and a tool call that arrives as four pieces of a JSON string
- You built
adapter.py:to_anthropic(), one wire message per message, andparse_sse(), which accumulates, parses once, and always ends in exactly one terminal event- The principle
- Only the edges changed. The harness you built against a fake runs against the real thing, because the fake kept the same contract.
- Your harness now
- run_tool
- run_agent
- context_for_model
- repair_tool_history
- Harness
- SessionLog
- persist_to
- summarize
- compact
- build_system_prompt
- guard
- CancelToken
- stop_run
- to_anthropic
- parse_sse
- Your answers
- Still open
- All of that ran in a browser tab, against streams somebody wrote out by hand. The key, the socket, the disk, the shell and the person who says yes are on the other side of the screen. Page 2.