10 · Watching and steering a run

The poisoned transcript

The user walked away while the tests were starting. They come back and type "never mind". The provider answers 400, and so does the prompt after that, and the one after that.

~50 min · 1 lab · builds on 09 But I had something to say

This lesson builds on lesson 09. New here? Start at lesson 01, or carry on: every lab is self-contained.

A job of three steps: the model asks for c1, c2 and c3 in one reply. While step 2 is running, the user types "Actually, use spaces." and your harness takes it. Where does their message land in the record?

Straight after step 2's result: U A[c1,c2,c3] R(c1) R(c2) U R(c3) A

That has a user message going in the moment it arrives. It would sit between c3 and the result for c3, and the whole of lesson 9 was about the two places where that cannot happen.

After all three results, at the top of the next turn: U A[c1,c2,c3] R(c1) R(c2) R(c3) U A

The loop asks for steering once, when the batch is answered, and the words go in where a user message is allowed to be.

After the run's last answer, as the start of a new turn: U A[c1,c2,c3] R(c1) R(c2) R(c3) A U A

That is the follow-up queue doing the steering queue's job. Both hold user messages; the difference is the moment the loop asks, and steering is asked for first.

Two model calls, and the second one carried the steer at the end of a transcript a provider accepts. Every call in that reply got its result, in the order it was asked for, before anything else was allowed in. Hold on to the word every. Today it stops being true.

import harness, lab


def step(arguments):
    """The job's tool. While step 2 is running, the user types into the harness."""
    if arguments["n"] == 2:
        h.steer("Actually, use spaces.")
    return f"step {arguments['n']} done"


tools = [{"name": "step", "description": "Do one step of the job.",
          "parameters": {"type": "object",
                         "properties": {"n": {"type": "integer", "description": "Which step."}},
                         "required": ["n"]},
          "execute": step}]
model = lab.ScriptedModel([
    lab.reply(lab.text("Three steps, then."), lab.call("step", {"n": 1}),
              lab.call("step", {"n": 2}), lab.call("step", {"n": 3})),
    lab.say("Done, with spaces."),
])
h = harness.Harness(model, harness.SYSTEM, tools)
await lab.drive(h.prompt("Do the three steps."))

print("the record:      ", lab.shape(h.messages))
print("model calls:     ", len(model.calls))
print("second request:  ", lab.shape(model.calls[1].messages))
print("valid record:    ", lab.validate(list(h.messages)) == [])

the record:       U A[c1,c2,c3] R(c1) R(c2) R(c3) U A
model calls:      2
second request:   U A[c1,c2,c3] R(c1) R(c2) R(c3) U
valid record:     True

A teammate's harness is yours with one line changed: every request is the record, exactly as it stands. Their session is going fine when the provider returns 503 overloaded on one call. What happens to the prompt after that?

Nothing much. The 503 was recorded as a message like any other, and the next prompt gets a fresh model call

It was recorded as a message like any other, and that is the trouble: lesson 5's failure message has nothing in its content, and an assistant message with nothing in it is not a turn a provider will take.

It is refused, and so is every prompt after it, until something stops sending that message

One outage, and the session is over. Count the refusals in the output.

It is refused once. The prompt after that works, because by then the failure is no longer the newest message

That has the trouble ageing out. The record only grows, and the empty message stays in it for good; what is sent is a decision somebody has to make on every request.

Your context_for_model is the line they are missing: it leaves the empty failed reply out of what you send, and leaves it in the record where it can still be read. Two prompts died there for good. Keep the shape of that in mind: a message the record must keep and a request must not carry.

import harness, lab

# Their harness is yours with one line changed: every request is the record, as it stands.
harness.context_for_model = lambda messages: list(messages)

model = lab.ScriptedModel([
    lab.say("config.py sets PORT = 9090."),
    lab.fail("503 overloaded"),
    lab.say("It is back."),
    lab.say("Still here."),
])
h = harness.Harness(model, harness.SYSTEM, [])
for text in ("What is in config.py?", "And main.py?", "Never mind. Hello?", "Anyone?"):
    await lab.drive(h.prompt(text))
    last = h.messages[-1]
    print(f"{text!r:24} -> {last.get('error_message') or harness.text_of(last)}")
print()
print("the record:", lab.shape(h.messages))

'What is in config.py?'  -> config.py sets PORT = 9090.
'And main.py?'           -> 503 overloaded
'Never mind. Hello?'     -> 400 invalid_request: messages[3]: assistant message has empty content
'Anyone?'                -> 400 invalid_request: messages[3]: assistant message has empty content

the record: U A U A(error) U A(error) U A(error)

The user walks away

"Reformat main.py and run the tests." The model says "Running the tests." and asks for bash. Your harness yields tool_execution_start, the screen shows a spinner, and the user closes the tab.

Lesson 8 built you the door for that: the consumer closes the run, the finally in _run runs, is_running goes back to false, and the harness is ready for the next prompt. Nothing crashed. Nobody lost any work. An hour later the same user comes back and types something else.

The run was closed at tool_execution_start, so pytest never ran and nothing was recorded for the call. Now the user types "Never mind. What is in config.py?" What does the model answer, and what about the two prompts after that?

Nothing, three times: each request is refused, and the refusal names the call from an hour ago

The newest prompt is fine. The damage is behind it, in a pair that was never completed, and it goes out with every request.

The answer. The model can see it asked for pytest and never got anything back, so it carries on without it

That treats a broken transcript as untidy rather than invalid. The model never sees this list: the shape is checked before the model is given anything to read, so nothing downstream gets the chance to be forgiving.

The first prompt is refused. The second works: context_for_model builds a fresh list for every request, and a list built now does not carry an hour-old mistake

It does build a fresh list, and freshness is not the point. It builds that list from the record, and the record still holds a call nobody answered.

400 invalid_request: messages[1]: toolCall c1 has no toolResult, word for word, three times. The tool never ran, so there is nothing you forgot to write down; the record is an honest account of a job that was abandoned halfway. It is also the list every later request is built from, so one hour-old accident is what comes back to three different questions. One unfinished pair has bricked the session.

Where this failure comes from: the note Tau wrote about it begins "Older cancellation and persistence paths could save either side of a tool exchange without the other" (dev-notes/tool-history-recovery.md:19-21). Preventing new damage was not enough, because it "does not make existing user sessions usable".

import harness, lab

ws = lab.Workspace({"main.py": "def main():\n\treturn 1\n", "config.py": "PORT = 9090\n"})
started = []
shell = lab.Shell(ws)


def run_command(arguments):
    started.append(arguments["command"])
    return shell.run(arguments["command"])[0]


tools = [harness.make_read_tool(ws),
         {"name": "bash", "description": "Run a shell command.",
          "parameters": {"type": "object",
                         "properties": {"command": {"type": "string", "description": "The line."}},
                         "required": ["command"]},
          "execute": run_command}]
model = lab.ScriptedModel([
    lab.reply(lab.text("Running the tests."), lab.call("bash", {"command": "pytest"})),
    lab.say("config.py sets PORT = 9090."),
    lab.say("Yes, still here."),
    lab.say("Shall we start again?"),
])
h = harness.Harness(model, harness.SYSTEM, tools)

# The consumer takes five events and then walks away: run.close(), explicitly.
events = await lab.drive(h.prompt("Reformat main.py and run the tests."), stop_after=5)
print("the last event the screen got:", events[-1]["type"])
print("commands that ran:            ", started)
print("the record:                   ", lab.shape(h.messages))
print()
for text in ("Never mind. What is in config.py?", "Are you still there?", "Hello?"):
    await lab.drive(h.prompt(text))
    last = h.messages[-1]
    print(f"{text!r:36} -> {last.get('error_message') or harness.text_of(last)}")
print()
print("the record:", lab.shape(h.messages))

the last event the screen got: tool_execution_start
commands that ran:             []
the record:                    U A[c1]

'Never mind. What is in config.py?'  -> 400 invalid_request: messages[1]: toolCall c1 has no toolResult
'Are you still there?'               -> 400 invalid_request: messages[1]: toolCall c1 has no toolResult
'Hello?'                             -> 400 invalid_request: messages[1]: toolCall c1 has no toolResult

the record: U A[c1] U A(error) U A(error) U A(error)

That last pair has a name worth having: a dangling call. [general] Hosted providers reject a request in which a tool call has no result; the wording differs from one to the next. Here the refusal arrives as an ordinary message, because the lab's model does that shape check itself, so you can watch it happen without a network. Figure 10.1 is the whole failure on one page.

Figure 10.1 One list doing two jobs. The hole in the record travels into the request, and the request comes back refused.

Note what you must not do, before anyone suggests it. You could delete the assistant message that asked for pytest. The list would be valid again, and it would say that the agent never tried to run the tests, which is false. Deleting is the one move that is always available and always a lie.

So: no deletions. The record keeps every message it has. What is the smallest true thing you could add to that list to make it a request a provider would take?

A provider's complaint is very specific: one call, no result. What is the least you could say that is still a tool result, and still true?

One message: a toolResult for c1, right after the reply that asked for it. It carries the call's id and the tool's name, because those you know, and for its text the only thing anybody can honestly say: no result was recorded for this call. That is one message added and nothing taken away, and the second half of this lesson is about how few things you are allowed to claim in it.

Common answers, and what each one misses

"Run the tool now and add the real result." An hour late, on a machine whose files have moved on, for a user who has said "never mind". You would be answering a question nobody is asking any more, and paying for it.
"Add a user message explaining what happened." Prose is not protocol, as lesson 2 had it. The complaint is about a missing toolResult, and a sentence in the wrong role does not answer it.
"Delete the call from the assistant message and leave the sentence." Tidier, valid, and the record now shows a model that said "Running the tests." and asked for nothing. You have rewritten history to make it fit.

Five wrecks

A dangling call is one way a transcript breaks. It is not the only one, and a repair that only knows this one will meet the others on a Friday. Your lab has a read-only file, damaged.py, holding five records that a crash, a careless harness or a provider really can leave behind. Figure 10.2 draws four of those breakages beside a healthy pair, on a job of its own.

Figure 10.2 One call in, one result out, right after it. The panels after the first are the ways that sentence gets broken.

Five cards follow. Each shows you a record; you decide what the request built from it should look like. Each card runs the repair you are about to write; until you have written it, the code runs against a finished one. There are only four moves in the whole lesson: make one up, drop one, move one, or leave it alone.

Where most of these come from: Tau's recovery note lists the shapes a repair "must therefore tolerate" — a call without a result, a result without a call, a result separated from its call by another message, duplicate results for one call, and parallel results saved in the wrong order (dev-notes/tool-history-recovery.md:28-34). Four of those are the cards below; the fifth, results saved out of order, waits in your lab's tests. Card 5 is this course's own.

Card 1, damaged.missing_result. One reply asked for c1 (read) and c2 (bash). c1 has its result. c2 has none, and a user message follows. What goes into the request?

A result for c2, made up by you, directly after c1's

One call, one result, in call order. The only question left is what that result says, and that is two cards away.

Drop the whole reply and c1's result with it. Half a turn is worse than no turn

That throws away a file the model read and reasoned from, to fix a complaint about a different call. It is also the deletion you refused a minute ago, wearing a bigger coat.

Nothing yet. Send it and let the provider tell you which call it is unhappy about, then fix that one

That makes a paid, failed request into a diagnostic step. Everything the provider is about to tell you is already in the list in front of you.

Four messages went in and five come out, in the order the model asked. The record still has its hole; this is a list built for sending. The made-up message has the shape of every other tool result, because it has to be one.

import damaged, harness, lab

record = damaged.missing_result
view = harness.repair_tool_history(record)
print(f"record   {lab.shape(record):32} refused: {lab.validate(record)[0]}")
print(f"view     {lab.shape(view):32} accepted")
print()
print("what stands in c2's place:")
print(lab.show(view[3:4], clip=200))

record   U A[c1,c2] R(c1) U               refused: messages[1]: toolCall c2 has no toolResult
view     U A[c1,c2] R(c1) R(c2) U         accepted

what stands in c2's place:
toolResult c2 -> "Tool call interrupted: no result was recorded. It may not have run, or may have run partly; check before repeating it."  [is_error]

Card 2, damaged.orphan_result. The reverse: a toolResult for c7 sits in the record, and no message anywhere asked for c7. The assistant message that did was lost. What goes into the request?

Put the call back: the result names the tool and the id, so build an assistant message asking for read with the obvious arguments

"The obvious arguments" is you guessing which file was read, and writing the guess down in the model's own voice. Look at the last line of the output: the arguments are the one thing the record cannot tell you.

Leave the result out. Without its call it is not a message you can send

Nothing invented, nothing claimed. What is lost is one file's contents, and the model can ask for it again.

Keep it and change its role to user, so at least the text survives

Then a file's contents arrive in the request as something the user typed. Hold that thought until lesson 14; for now, a message's role is a claim about who spoke, and this one would be false.

The result is dropped and no call is invented for it. This is the rung to be stubborn about: an invented call is not a tidy-up, it is a sentence in the model's mouth that the model never said, and everything it does next is reasoning from it. A dropped result costs one tool call to recover. An invented one is never recovered at all.

Where this rule comes from: Tau's repair omits a result with no call, "because a missing call's arguments cannot be reconstructed safely" (src/tau_agent/tool_history.py:44-45).

import damaged, harness, lab

record = damaged.orphan_result
view = harness.repair_tool_history(record)
print(f"record   {lab.shape(record):26} refused: {lab.validate(record)[0]}")
print(f"view     {lab.shape(view):26} accepted")
print()
print("the result nobody asked for:")
print(lab.show(record[2:3]))
asked = [call for m in record if m["role"] == "assistant"
         for call in harness.tool_calls(m) if call["id"] == "c7"]
print("everything the record can tell you about the call it answers:")
print("  tool_call_id:", record[2]["tool_call_id"])
print("  tool_name:   ", record[2]["tool_name"])
print("  arguments:   ", asked[0]["arguments"] if asked else "no message in the record asked for c7")

record   U A R(c7) U                refused: messages[2]: toolResult c7 has no toolCall before it
view     U A U                      accepted

the result nobody asked for:
toolResult c7 -> "def main():\n\treturn 1\n"
everything the record can tell you about the call it answers:
  tool_call_id: c7
  tool_name:    read
  arguments:    no message in the record asked for c7

Card 3, damaged.late_result. The call for c1, then a user message, then c1's real result. Two writers on one list, which is the bug lesson 8 closed. The result is genuine and it is one message too late. What goes into the request?

Nothing to do. Every call has a result somewhere in the list, and the pairing is by id

By id, yes, and a provider also wants them adjacent: read the second complaint in the output. Ids answer "which result"; position answers "whose turn is this".

Move it up, so it sits directly after the reply that asked for it

Nothing is thrown away and nothing is made up. The user's message keeps its place in the order, one slot later.

Make one up for c1 where it belongs, and drop the late one. One rule for every hole is simpler

Simpler, and it tells the model that a file it really did read was never read. A real result always beats a made-up one, however much more code that costs.

Moved, not replaced: the output shows zero made-up results and the moved message is the very object that was recorded, not a copy of it. A repair that could not tell "late" from "missing" would quietly lose the contents of main.py here. The four moves are not interchangeable.

import damaged, harness, lab

record = damaged.late_result
view = harness.repair_tool_history(record)
print(f"record   {lab.shape(record):30} refused, twice over:")
for complaint in lab.validate(record):
    print(f"         {complaint}")
print(f"view     {lab.shape(view):30} accepted")
print()
print("what the model is shown for c1:")
print(lab.show(view[2:3]))
print("is that the very message that was recorded?", view[2] is record[3])
print("made-up results in the view:",
      sum(m["role"] == "toolResult" and m["content"] == harness.INTERRUPTED for m in view))

record   U A[c1] U R(c1) A              refused, twice over:
         messages[1]: toolCall c1 has no toolResult
         messages[3]: toolResult c1 must directly follow its assistant message, in call order
view     U A[c1] R(c1) U A              accepted

what the model is shown for c1:
toolResult c1 -> "def main():\n\treturn 1\n"
is that the very message that was recorded? True
made-up results in the view: 0

Card 4, damaged.duplicate_result. One call for c2, and two results for it: "3 passed", then "3 passed (second run)". Some retry recorded its answer twice. One of them must go. Which one stays?

The first one recorded

It is the one the run went on to act on. The assistant message after it was written by a model that had been shown that text and no other.

The last one recorded. It is the newest, so it is the most nearly true

Newest is a good rule for facts about the world and a bad one for a transcript, which is a record of what was said when. The reply that follows these two was an answer to the first.

Keep both and let the model decide which to believe

A provider settles that before the model gets a chance: one call, one result. And a model shown two answers to one question has been handed your problem.

The first wins and the second is dropped. The test is not which text is better; it is which text the rest of the transcript was built on. Change that and the assistant message after it stops making sense.

import damaged, harness, lab

record = damaged.duplicate_result
view = harness.repair_tool_history(record)
print(f"record   {lab.shape(record):30} refused: {lab.validate(record)[0]}")
print(f"view     {lab.shape(view):30} accepted")
print()
print("the two results on the record, in the order they were recorded:")
print(lab.show(record[2:4]))
print("the one the model is shown:")
print(lab.show(view[2:3]))

record   U A[c2] R(c2) R(c2) A          refused: messages[3]: toolResult c2 is a second result for a call that already has one
view     U A[c2] R(c2) A                accepted

the two results on the record, in the order they were recorded:
toolResult c2 -> "3 passed"
toolResult c2 -> "3 passed (second run)"
the one the model is shown:
toolResult c2 -> "3 passed"

Card 5. A new one. The reply is cut off mid-stream: it carries the words "Writing the file." and half a write call, with a path and no content, and it is labelled error. Lesson 5's loop acts on none of a failed reply, so nothing ran. What does the next request look like?

The reply is dropped from the request, calls and all, the way lesson 5 drops a failed reply

Lesson 5 drops a failed reply that is empty. This one got words out before the connection died, and those words are the only explanation the model will have for the gap. Look at what the view keeps.

The reply stays as it is, and its half-built call gets a made-up result, like any other call with none

A call the model can see is a call the provider expects an answer to, whatever the reply's label says.

The loop should run the call now and put the real result in. The path is there; the arguments can be filled in

Half the arguments of a write is not a write; it is an unknown amount of a file being replaced by an unknown amount of nothing. Note where the belief sits: not in the repair, but in a loop that would act on a message that never finished arriving.

Nothing was written to the disk, the record keeps the failed reply exactly as it came, and the next request pairs that call with an interrupted result like any other. Two rules meet here and neither bends: never act on a reply that failed, and never send a call without a result.

Where the first rule comes from: in Tau's loop a reply that came back error ends the run on the spot, just above the loop that would have run its calls (src/tau_agent/loop.py:148-151).

import harness, lab

ws = lab.Workspace({"main.py": "def main():\n\treturn 1\n", "config.py": "PORT = 9090\n"})
tools = [harness.make_read_tool(ws), harness.make_write_tool(ws)]
model = lab.ScriptedModel([
    lab.fail("stream ended before the message was complete",
             lab.text("Writing the file."), lab.call("write", {"path": "main.py"})),
    lab.say("Nothing was written. Shall I try again?"),
])
h = harness.Harness(model, harness.SYSTEM, tools)
await lab.drive(h.prompt("Reformat main.py."))
print("the reply that came back:")
print(lab.show(list(h.messages)[1:2]))
print("writes to the disk: ", ws.writes)
print("the record:         ", lab.shape(h.messages))
print()
await lab.drive(h.prompt("What happened?"))
print("the next request:   ", lab.shape(model.calls[1].messages))
print(lab.show(model.calls[1].messages[2:3], clip=200))
print("the record:         ", lab.shape(h.messages))

the reply that came back:
assistant -> "Writing the file." + toolCall c1 write({"path": "main.py"})  [error: stream ended before the message was complete]
writes to the disk:  []
the record:          U A[c1](error)

the next request:    U A[c1](error) R(c1) U
toolResult c1 -> "Tool call interrupted: no result was recorded. It may not have run, or may have run partly; check before repeating it."  [is_error]
the record:          U A[c1](error) U A

Choosing the words

Two of the five cards needed a message that nobody produced. It has to be a toolResult, it has to carry the call's id and the tool's name, and then there is a blank where a tool's output would be. Three candidates for that blank. One of them is Tau's.

The walk-away record again: c2 asked for pytest and nothing came back. Which of the three should the model be shown, and should it be marked is_error?

"Tool call interrupted by user", marked as an error. Short, and it says who did it

It reads like a line in an operations log, written for whoever is on call. The model is the reader here, and it will reason from the sentence: a user interrupted this, so they meant to stop it. Nobody knows that. The same record is left by a crash, a kill -9 and a dead connection.

"Tool call interrupted: no result was recorded. It may not have run, or may have run partly; check before repeating it.", marked as an error

Every clause is something the program can actually see, and the last one tells the model what to do about it.

"The tool ran.", not an error. The command almost certainly finished; saying so keeps the model calm

"Almost certainly" is the whole problem. If it did not run, the model now believes the tests were run; if it did, it believes they passed. Either way you have put a fact into the transcript that no one checked.

All three are accepted. [general] A provider checks the shape of the list, not the truth of it, so nothing downstream will catch a comfortable lie for you. The record supports exactly two claims: this call was made, and no result was recorded for it. is_error is true because, as far as anybody knows, the tool did not do its job.

Where the wording matters: Tau's constant reads Tool call interrupted by user (src/tau_agent/tool_history.py:16), and the same function is reached on a plain provider failure, where no user did anything. This course does not copy that text.

import damaged, harness, lab

record = damaged.missing_result             # U A[c1,c2] R(c1) U ; c2 got no result
call = harness.tool_calls(record[1])[1]

candidates = [("(a) Tool call interrupted by user", "Tool call interrupted by user", True),
              ("(b) " + harness.INTERRUPTED[:31] + "...", harness.INTERRUPTED, True),
              ("(c) The tool ran. (not an error)", "The tool ran.", False)]
for label, content, is_error in candidates:
    made = {"role": "toolResult", "tool_call_id": call["id"], "tool_name": call["name"],
            "content": content, "is_error": is_error}
    view = record[:3] + [made] + record[3:]
    print(f"{label:38} provider: {'accepted' if lab.validate(view) == [] else 'refused'}")

print()
print("what the record actually says about", call["id"] + ":")
print("  it was asked for:       ", call["name"], call["arguments"])
print("  results recorded for it:",
      sum(m["role"] == "toolResult" and m["tool_call_id"] == call["id"] for m in record))
print("  anything else:          ", "nothing was written down")

(a) Tool call interrupted by user      provider: accepted
(b) Tool call interrupted: no resul... provider: accepted
(c) The tool ran. (not an error)       provider: accepted

what the record actually says about c2:
  it was asked for:        bash {'command': 'pytest'}
  results recorded for it: 0
  anything else:           nothing was written down

One thing you do know, and you have lesson 7 to thank for it. Your loop records a result and only then announces it, so there is no moment at which a consumer can stop listening and leave a finished tool unrecorded. What that does not rule out is the tool that started, did half its work, and was still going when the process died — which is why the sentence says "may have run partly" and not "did not run".

When does it run?

You have a rule for every wreck. The next question is where to put it. The obvious place is where the damage is discovered: the session is opened, the list is checked, the holes are filled once, and every later request is built from a list that is already sound.

A session is loaded from yesterday, which ended in a walk-away, and it is repaired once as it loads. The first prompt today is answered. Then the user walks away again, mid-tool, and comes back. What does the next prompt get?

It works. The repair has been run on this session, and a session is repaired or it is not

That has damage as a property of a session rather than an event that can happen at any minute. The second walk-away is the same accident as the first, at half past three instead of yesterday.

It is refused, for the new hole, exactly as yesterday's prompt was

"Repair on load" answers the question "was it broken when I found it". The useful question is "is what I am about to send valid".

It is refused, and that is what to build on: catch the refusal, repair, and send again

A repair you already know how to run, run only after a request has been refused and paid for. And the retry has to work out from the provider's error text that this particular refusal is the repairable kind. Lesson 12 meets a harness that matches error text for a different refusal, and what that costs.

Two harnesses, the same two walk-aways. The one that repaired at load answers the first prompt and is refused for the second, and its record now carries a made-up result from yesterday that it will never be able to tell apart from a real one. The one that repairs before every request answers both. The question is not when the damage happened; it is what is in this request.

import harness, lab

YESTERDAY = [lab.user("Reformat main.py and run the tests."),
             lab.reply(lab.text("Running the tests."),
                       lab.call("bash", {"command": "pytest"}, id="y1"))]

LESSON_5_VIEW = lambda messages: [m for m in messages if not (
    m["role"] == "assistant" and not m["content"] and m.get("stop_reason") == "error")]
EVERY_REQUEST = harness.context_for_model


def answer(request):
    """Asks for pytest when told to run the tests; otherwise answers in words."""
    said = [m["content"] for m in request.messages if m["role"] == "user"][-1]
    if said == "Run the tests.":
        return lab.reply(lab.text("Running them."), lab.call("bash", {"command": "pytest"}))
    return lab.say("Carrying on.")


async def session(label, view, messages):
    harness.context_for_model = view
    shell = lab.Shell(lab.Workspace({"config.py": "PORT = 9090\n"}))
    h = harness.Harness(lab.ScriptedModel([lab.forever(answer)]), harness.SYSTEM,
                        [harness.make_bash_tool(shell)], messages=messages)
    print(label)
    await lab.drive(h.prompt("What is in config.py?"))
    print("   prompt 1, on yesterday's hole:  ", h.messages[-1].get("error_message") or "answered")
    await lab.drive(h.prompt("Run the tests."), stop_after=5)     # the user walks away again
    await lab.drive(h.prompt("Never mind. Hello?"))
    print("   prompt 2, after a second hole:  ", h.messages[-1].get("error_message") or "answered")
    print("   the record:                     ", lab.shape(h.messages))


await session("repaired once, when the session was loaded:",
              LESSON_5_VIEW, harness.repair_tool_history(YESTERDAY))
print()
await session("repaired before every request:", EVERY_REQUEST, list(YESTERDAY))

repaired once, when the session was loaded:
   prompt 1, on yesterday's hole:   answered
   prompt 2, after a second hole:   400 invalid_request: messages[6]: toolCall c1 has no toolResult
   the record:                      U A[y1] R(y1) U A U A[c1] U A(error)

repaired before every request:
   prompt 1, on yesterday's hole:   answered
   prompt 2, after a second hole:   answered
   the record:                      U A[y1] U A U A[c1] U A

Before every request means thousands of times a session, nearly always on a list with nothing wrong with it. Tap every property the repair needs before that is a sane thing to do.

Running it on its own output changes nothing: repair(repair(x)) == repair(x)
A transcript that is already valid comes back equal
It reports how many messages it changed
It is faster than the model call it precedes

Those two. Without the first, a made-up result gets a made-up result of its own on the next pass and the list grows every time you send it. Without the second, the thousands of healthy requests are quietly rewritten by a function nobody is watching, and you find out which way when something downstream stops matching. Counters are useful for a log and change nothing about safety, and speed is not the objection: walking a list is nothing beside the model call that follows it.

The first property has a name: the repair is idempotent. It is the property that lets you stop worrying about when the damage happened.

Two lists

One question left, and it is the one this lesson is really about. The repair produces a list. Does that list replace the one your harness is holding?

A record that has both of this lesson's problems in it: an empty 503 failure from lesson 5, and a call with no result. Where does the repaired list go?

Into h.messages: repair the stored list in place, so it is correct from now on and nothing has to remember to do it again

Then the record says a tool result exists for a call whose tool may never have started, in a message shaped exactly like one a tool produced. And lesson 5's failure message would have to go too, so nobody can ever ask why the session stalled at half past three.

Into context_for_model, and nowhere else: drop the empty failure, repair the pairs, hand that list to the model, leave the record alone

Two steps, one function, one list built per request. The record keeps everything that happened, holes included.

Into h.messages, and keep an untouched copy of the record beside it for diagnostics

Now two lists both claim to be the session, and every later message has to be appended to both. You have the second copy already: it is the record, and the sent list is the cheap one to rebuild.

Five messages in the record, five in the view, and they are not the same five: the view leaves out a failure only the record can still explain, and holds a made-up result the record must never claim. Four of the five are the very same objects, because nothing here edits a message. This is the sentence the whole lesson was for, and it has a number: the record is not the view. Keep everything that happened; work out what you send.

import harness, lab

call = lab.call("bash", {"command": "pytest"}, id="c1")
record = [lab.user("Run the tests."),
          {"role": "assistant", "content": [], "stop_reason": "error",
           "error_message": "503 overloaded"},
          lab.user("Try again."),
          lab.reply(call),
          lab.user("Never mind. What is in config.py?")]

before = lab.shape(record)
view = harness.context_for_model(record)
print("the record:", before)
print("the view:  ", lab.shape(view))
print("the record, after building the view:", lab.shape(record))
print()
print("in the record and not in the view:")
print(lab.show(record[1:2]))
print("in the view and not in the record:")
print(lab.show(view[3:4], clip=200))
print()
print("messages the view shares with the record:",
      sum(any(m is r for r in record) for m in view), "of", len(view))

the record: U A(error) U A[c1] U
the view:   U U A[c1] R(c1) U
the record, after building the view: U A(error) U A[c1] U

in the record and not in the view:
assistant -> (nothing)  [error: 503 overloaded]
in the view and not in the record:
toolResult c1 -> "Tool call interrupted: no result was recorded. It may not have run, or may have run partly; check before repeating it."  [is_error]

messages the view shares with the record: 4 of 5

Figure 10.3 is Figure 10.1 again, with that split drawn in.

Figure 10.3 The record keeps its hole and is never rewritten. The view, built for this request and thrown away after it, has the hole filled with a message that says so.

Build: repair the view, keep the record

Everything is decided. Every call is followed at once by exactly one result, in call order. A recorded result is kept and moved into place; of two for one call the first wins; a result whose call is nowhere — an orphan result — is dropped; and a call with no result gets a synthetic result that says so. The list that comes back is new, holds the same message objects, and is built afresh for every request. That whole paragraph is one function, and the name it goes by is repair.

This lab gives you no skeleton: a paragraph of spec, the names the tests import, and two gaps. The five transcripts of the cards above are in damaged.py, sitting read-only beside your file, and five of the hidden tests are exactly them.

Two gaps at the bottom of region 3, about nineteen lines in all.

context_for_model(messages), the upper gap. Lesson 5's body is in it already. One change: what it returns goes through your repair.
INTERRUPTED and repair_tool_history(messages), the lower gap, which holds the spec and nothing else. The constant's exact text is in that spec, and a test checks it word for word. The function returns a new list in which every tool call is followed at once by exactly one result, in the order the calls were written. Four decisions, all of them yours from the cards above: keep a recorded result and move it, let the first of two duplicates win, drop a result whose call is nowhere, and make one up for a call that has none, carrying the call's id, the tool's name, INTERRUPTED as its content and is_error true.

Pure, and the tests are strict about it: messages is not touched, not even the messages inside it, the list that comes back is not the list that went in, and a valid transcript comes back equal and holding the very same message objects. Idempotent: repairing a repaired list changes nothing. You may take call ids to be unique.

Fourteen hidden tests. Five are the transcripts of damaged.py. The other nine: results come out in call order when the record recorded them out of order; a record that ends on its own damage, with no user message after it; a valid transcript back unchanged and holding the very same message objects; a repaired list repaired again; the input untouched; the view repaired while the record still shows the failure and the hole; the walk-away itself, where the two prompts after it must be answered and the dangling call ends up in the middle of the history; the calls of a failed reply never executed; and, last, a three-call job with the consumer closing the run after every one of its events in turn, each time followed by two more prompts that have to be answered.

Walk the record from the top, once. At an assistant message you know which calls need results and in what order. At a result you do not yet know whether it is early, late, duplicate or an orphan. Which of the two is the thing to build the loop around, and what would you need to have looked up before you start?
Two passes. The first one looks at tool results only and answers a single question: for a given call id, which recorded result is the one the model should see? The first one carrying that id, wherever in the list it sits. Store those by id. The second pass builds the new list from the top: copy every message that is not a result, and after each assistant message put, for each of its calls in order, the result you looked up, or a new one if there is none. Nothing is appended to messages; nothing is copied that you did not ask for. Notice what you have not written by the end of it: an orphan and a spare duplicate are both simply results that nothing ever looked up, so they never reach the new list.
In outline. Pass one: an empty mapping from call id to message; for each message with role toolResult, record it under its tool_call_id only if that id is not in the mapping yet, which is what makes the first one win. Pass two: an empty list; for each message, skip it if it is a toolResult (they go back in below); otherwise put it on the list, and if its role is assistant, walk tool_calls(message) in order and put, for each call, the message stored under that call's id, or, if there is none, a fresh dict shaped like every other tool result, with INTERRUPTED for its content. Return the list. Then in context_for_model, the existing expression becomes the argument of one more call.

Nineteen lines, and a session that cannot be bricked. Look at what you did not write: a branch for the orphan, a branch for the duplicate, a check for whether anything is wrong before starting, and any code at all that runs when the damage happens. The last test closed the run after every one of twenty-one events, twenty-one times, and every one of those sessions carried on.

What changed since lesson 09

The line-by-line diff needs JavaScript. The whole file this exercise starts from is printed at the end of it.

damaged.missing_result: c2 gets a result right after c1's, marked is_error, whose text is INTERRUPTED, word for word.
damaged.orphan_result: the result whose call is nowhere is left out, and no call is invented for it.
damaged.late_result: the result recorded after a user message is moved to directly after its call; nothing is thrown away and nothing made up.
damaged.duplicate_result: the first result for c2 is kept and the second dropped.
damaged.failed_reply_with_a_call: the reply failed mid-stream with a half-built call in it; in the view that call is answered with INTERRUPTED.
An assistant message asked for c1, c2, c3; the record holds c3's result and then c1's. Repaired: c1's, INTERRUPTED for c2, c3's.
The record as the walk-away leaves it, before anyone types again: it ends on the assistant message, or on c1's result with c2 unanswered. Repaired, it ends on INTERRUPTED; a valid transcript that ends on a result comes back equal.
Repairing a valid transcript returns an equal list made of the very same message objects; so does repairing twice, and a moved result is the recorded object, not a copy.
For each of the five damaged transcripts, and for all five glued together, repair(repair(x)) == repair(x).
After repair_tool_history(x), x is exactly what it was: same length, same messages, nothing added to or removed from any message.
context_for_model drops empty failed replies (lesson 5) and then repairs: what it returns is valid, and the record it was given still shows the failure and the dangling call.
The consumer closes the run at tool_execution_start; the next TWO prompts are answered, the model is shown INTERRUPTED for c1, and h.messages still ends the first run on the dangling call.
A reply fails mid-stream with a half-built write call in it: the tool does not run, the next prompt is answered, and the model is shown INTERRUPTED for that call.
The same three-call job, closed after k events, for every k: two further prompts are answered, and the model is never shown INTERRUPTED for a call whose tool ran. Left to finish, the record itself is valid.

import json from collections import deque # == 1 messages == # A conversation is a list of messages. Each message is a dict tagged with the role that spoke. SYSTEM = "You are a helpful assistant." # standing instructions, sent with every call def user_message(text): """A message from the user. Its content is a plain string.""" return {"role": "user", "content": text} def text_of(message): """The words of an assistant message. Its content is a list of blocks, not a string (lesson 2 explains why); the words are in the blocks of type "text".""" return "".join(block["text"] for block in message["content"] if block["type"] == "text") def tool_calls(message): """The tool calls an assistant message asks for, in the order it asked. Worked out from `content` on every call and stored nowhere else.""" return [block for block in message["content"] if block["type"] == "toolCall"] def error_message(text): """An assistant message that says why a run stopped: nothing in `content`, stop_reason "error", and the reason in `error_message`. A stop is written into the record, not raised.""" return {"role": "assistant", "content": [], "stop_reason": "error", "error_message": text} # == 2 tools == # A tool is a dict. "name", "description" and "parameters" are advertised to the model. # "execute" is a function, execute(arguments) -> str, that only this program ever runs. def str_arg(arguments, name): """The string argument `name`, checked. The schema asked the model for a string and nobody made it send one, so every tool reads its arguments through here. Raises ValueError("<name> must be a string") when it is missing or not a string.""" value = arguments.get(name) if not isinstance(value, str): raise ValueError(f"{name} must be a string") return value def int_arg(arguments, name, default): """The optional whole-number argument `name`, checked; `default` if the model left it out. Raises ValueError("<name> must be an integer") for anything else.""" value = arguments.get(name) if value is None: return default if not isinstance(value, int): raise ValueError(f"{name} must be an integer") return value def tool_specs(tools): """What the model is told about the tools: everything except the function.""" return [{"name": tool["name"], "description": tool["description"], "parameters": tool["parameters"]} for tool in tools] def run_tool(tools, call): """Run one tool call and return the toolResult message that answers it. One call in, exactly one result out, on every path. Tools just raise; this is the one place that turns a failure into a result the model can read. is_error means the tool could not do its job. It never raises Exception.""" tools_by_name = {tool["name"]: tool for tool in tools} tool = tools_by_name.get(call["name"]) if tool is None: content, is_error = f"Tool {call['name']} not found", True else: try: content, is_error = tool["execute"](call["arguments"]), False except Exception as exc: content, is_error = str(exc), True return {"role": "toolResult", "tool_call_id": call["id"], "tool_name": call["name"], "content": content, "is_error": is_error} MAX_LINES = 2000 # The budget of one tool result. Whatever a tool returns is re-read, and MAX_BYTES = 50_000 # paid for, on every later turn. Bytes, not characters: "é" is two. def truncate_head(text, max_lines=MAX_LINES, max_bytes=MAX_BYTES): """The start of `text` that fits the budget, for files: (kept, lines_shown). Whole lines only, newlines included, sized in UTF-8 bytes. If even the first line is too big, the answer is ("", 0): half a line is worse than none.""" kept, size = [], 0 for line in text.splitlines(keepends=True): size += len(line.encode()) if len(kept) == max_lines or size > max_bytes: break kept.append(line) return "".join(kept), len(kept) def truncate_tail(text, max_lines=MAX_LINES, max_bytes=MAX_BYTES): """The end of `text` that fits the budget, for command output, where the verdict comes last: (kept, lines_shown). Whole lines only, newlines included, sized in UTF-8 bytes.""" kept, size = [], 0 for line in reversed(text.splitlines(keepends=True)): size += len(line.encode()) if len(kept) == max_lines or size > max_bytes: break kept.append(line) return "".join(reversed(kept)), len(kept) def make_read_tool(ws, max_lines=MAX_LINES, max_bytes=MAX_BYTES): """The `read` tool, over the workspace `ws`. One call returns at most the budget, and says so in a last line that tells the model how to ask for the rest.""" def execute(arguments): path = str_arg(arguments, "path") offset, limit = int_arg(arguments, "offset", 1), int_arg(arguments, "limit", None) if offset < 1 or (limit is not None and limit < 1): raise ValueError("offset and limit must be 1 or more") if not ws.exists(path): raise FileNotFoundError( f"File not found: {path}. Files here: {', '.join(ws.listdir('.'))}") lines = ws.read_text(path).splitlines(keepends=True) wanted = lines[offset - 1:] if limit is None else lines[offset - 1:offset - 1 + limit] kept, shown = truncate_head("".join(wanted), max_lines, max_bytes) if offset > max(len(lines), 1): raise ValueError(f"offset {offset} is past the end of {path}: {len(lines)} lines") if wanted and shown == 0: skip = f" Use offset={offset + 1} to skip it." if offset < len(lines) else "" return (f"[Line {offset} of {path} is {len(wanted[0].encode())} bytes, over the " f"{max_bytes}-byte budget of one read, so none of it is shown.{skip}]") last = offset + shown - 1 if last < len(lines): kept += (f"[Showing lines {offset}-{last} of {len(lines)}. " f"Use offset={last + 1} to continue.]") return kept return { "name": "read", "description": "Read a text file and return its contents. One call returns at most " f"{max_lines} lines or {max_bytes} bytes; use offset to read on.", "parameters": { "type": "object", "properties": {"path": {"type": "string", "description": "Path of the file to read."}, "offset": {"type": "integer", "description": "First line, from 1."}, "limit": {"type": "integer", "description": "How many lines."}}, "required": ["path"], }, "execute": execute, } def make_write_tool(ws): """The `write` tool, over the workspace `ws`: create a file, or replace all of it.""" def execute(arguments): path, content = str_arg(arguments, "path"), str_arg(arguments, "content") ws.write_text(path, content) return f"Successfully wrote to {path}." return { "name": "write", "description": "Write a text file. Creates the file, or replaces its whole contents.", "parameters": { "type": "object", "properties": {"path": {"type": "string", "description": "Path of the file to write."}, "content": {"type": "string", "description": "The new contents."}}, "required": ["path", "content"], }, "execute": execute, } def make_bash_tool(shell, max_lines=MAX_LINES, max_bytes=MAX_BYTES): """The `bash` tool: run one command line in `shell` and return what it printed. A command that fails has still been run: its exit code is news for the model, not a tool error, so it comes back as an ordinary result that ends with a status line.""" def execute(arguments): output, code = shell.run(str_arg(arguments, "command")) total = len(output.splitlines()) output, shown = truncate_tail(output, max_lines, max_bytes) if shown < total: output += f"\n[Showing the last {shown} of {total} lines. Narrow the command for more.]" if code != 0: output += f"\nCommand exited with code {code}" return output return { "name": "bash", "description": "Run a shell command and return its output (stdout and stderr together). " f"Only the last {max_lines} lines or {max_bytes} bytes come back.", "parameters": { "type": "object", "properties": {"command": {"type": "string", "description": "The command line."}}, "required": ["command"], }, "execute": execute, } # == 3 loop == # The code that talks to the model. The model keeps nothing between calls. def run_agent(model, system, messages, tools, prompt, *, max_turns=None, get_steering=None, get_follow_ups=None): """Put `prompt` to the model and keep going until it stops asking for tools, a reply comes back failed (stop_reason "error"), or `max_turns` model calls have been made (None: no limit). A generator of events: plain dicts with a "type", which say what happened and draw nothing. They come in balanced pairs on every exit: agent_start ... agent_end around the run, and turn_start ... turn_end around each turn. The one ordering rule is mutate, then announce: a message is appended to `messages` (the caller's list, the record) first, and only then is its message_end yielded. The final message is the last of agent_end's "messages". The user may speak during a run; the loop asks for their words only where a user message keeps the transcript valid. `get_steering` and `get_follow_ups` take no arguments and return a list of user messages, often empty: steering is asked for once a turn's whole tool batch is answered, follow-ups only when the run would otherwise end. The loop never sees a queue.""" new = [] # this run's messages, for agent_end def record(message): """Append `message` to the record, then make the event that announces it.""" messages.append(message) new.append(message) return {"type": "message_end", "message": message} def ask(get): """The user messages a callable has for us: [] if it has none, or was never passed.""" return list(get()) if get else [] yield {"type": "agent_start"} turn, calls, failed = 0, [], False # User messages waiting to open the next turn. Steering sent while idle goes in with the prompt. pending = [user_message(prompt)] + ask(get_steering) while pending: # outer loop: the prompt, then each follow-up while calls or pending: # inner loop: one pass is one turn turn += 1 yield {"type": "turn_start"} for message in pending: # only here, at the top of a turn, may a user speak yield record(message) pending = [] if max_turns is not None and turn > max_turns: # checked before the call, never after reply = error_message(f"Agent stopped after max_turns={max_turns}") else: reply = model.complete(system, context_for_model(messages), tool_specs(tools)) yield record(reply) # A failed reply: none of it is acted on and nobody is asked for more. failed = reply["stop_reason"] == "error" calls = [] if failed else tool_calls(reply) for call in calls: yield {"type": "tool_execution_start", "call": call} result = run_tool(tools, call) yield record(result) # no yield between the tool returning and this yield {"type": "tool_execution_end", "call": call, "result": result} yield {"type": "turn_end", "message": reply} if not failed: pending = ask(get_steering) # every call of the turn has its result if not failed: pending = ask(get_follow_ups) # no calls, no steering: the run would end here yield {"type": "agent_end", "messages": new} # the one way out, so every exit is balanced def context_for_model(messages): """What the model is sent. The record is not the view: the record keeps what happened, and this builds, for every request, a transcript a provider accepts. Assistant messages that failed and are empty are left out (a failed reply that got something out stays in), and then the calls and results are paired up. Returns a new list; the record is never changed.""" # --- your code --- # This is lesson 5's body. One change: what it returns goes through repair_tool_history. return [m for m in messages if not (m["role"] == "assistant" and not m["content"] and m.get("stop_reason") == "error")] # --- end --- # --- your code --- # Spec only. The tests import two names: INTERRUPTED and repair_tool_history. # # INTERRUPTED = ("Tool call interrupted: no result was recorded. It may not have run, or may " # "have run partly; check before repeating it.") # # repair_tool_history(messages) -> list. A copy of `messages` that a strict provider accepts: # every tool call is followed at once by exactly one result, in call order. A recorded result is # kept and moved to its place; of two results for one call the first wins; a result whose call # is nowhere is dropped; a call with no result gets a toolResult whose content is INTERRUPTED # and whose is_error is True. Pure: it never changes `messages` or any message in it, and the # list it returns holds the same message objects. Idempotent: repair(repair(x)) == repair(x), # and a valid transcript comes back equal. Call ids may be taken to be unique. # # The five transcripts in damaged.py, the lab's read-only file, are five of the hidden tests. # Look at them first: import damaged, then lab.shape(damaged.missing_result). # --- end --- # == 4 harness == # One transcript, one writer. The loop keeps the mechanics and stays stateless; this holds the list. class Harness: """Owns the one transcript, and the settings that stay the same from run to run, and lets one run at a time write to it. Harness(model, system, tools, messages=(), max_turns=None) copies `messages` into a list of its own. .messages a tuple, a snapshot of the transcript: nobody outside can append to it or pop from it. (The dicts inside are not copied.) .is_running True from the call of prompt() until that run is exhausted, closed or dies. .prompt(text) returns the run: run_agent's events, over the harness's own list. While a run is going on it raises RuntimeError("already running; use steer() or follow_up()"), and it raises on the caller's line, so prompt is a plain def and the generator is _run. .steer(text), .follow_up(text) queue a user message: steer for the next gap between turns ("use spaces"), follow_up for when the run would otherwise end ("now run the tests"). Callable at any time, by anyone. A queued message is not history until the loop asks for it, one message per ask. Queues outlive runs.""" def __init__(self, model, system, tools, messages=(), max_turns=None): self._model, self._system, self._tools = model, system, list(tools) self._messages, self._max_turns = list(messages), max_turns self._running = False self._steering, self._follow_ups = deque(), deque() @property def messages(self): return tuple(self._messages) @property def is_running(self): return self._running def prompt(self, text): if self._running: # checked now, not at the first next() raise RuntimeError("already running; use steer() or follow_up()") self._running = True return self._run(text) def steer(self, text): self._steering.append(user_message(text)) def follow_up(self, text): self._follow_ups.append(user_message(text)) def _drain(self, queue): """What the loop is handed when it asks: the oldest queued message, one at a time.""" return [queue.popleft()] if queue else [] def _run(self, text): run = run_agent(self._model, self._system, self._messages, self._tools, text, max_turns=self._max_turns, get_steering=lambda: self._drain(self._steering), get_follow_ups=lambda: self._drain(self._follow_ups)) try: for event in run: yield event finally: # exhausted, closed by the consumer, or dead run.close() self._running = False # == 5 frontends == # A frontend is a fold over the events of one run: render(event) for each event, then finish(), # which says whether the run ended well. The loop never learns what its events look like. class JsonRenderer: """Every event as one line of JSON, for another program to read.""" def __init__(self): self.failed = False def render(self, event): print(json.dumps(event)) if event["type"] == "message_end" and event["message"].get("stop_reason") == "error": self.failed = True def finish(self): return not self.failed class FinalTextRenderer: """Only the answer, for `> answer.txt`: nothing is printed while the run goes on. finish() prints the text of the last assistant message and returns True; if that message failed (stop_reason "error") it prints "Error: <its error_message>" instead and returns False.""" def __init__(self): self.last = None # the newest assistant message seen so far def render(self, event): if event["type"] == "message_end" and event["message"]["role"] == "assistant": self.last = event["message"] def finish(self): if self.last is None: return False if self.last["stop_reason"] == "error": print(f"Error: {self.last['error_message']}") return False print(text_of(self.last)) return TrueRunning code needs JavaScript. Above is the file this exercise starts from (harness.py). You can run its tests on your own machine: python lab/lab_check.py --harness harness.py --lesson 10 tests.py

A three-step job, walked away from at three different moments: before any step ran, between step 1 and step 2, and in the middle of the second turn. Each time, two more prompts follow. No tests: read the six lines under each walk-away. Once the lab has passed, this runs against your code.

import harness, lab

WORDS = ("Never mind, what is in config.py?", "And main.py?")


def job(request):
    """Two steps, then a third, then done. Any other prompt is answered in words."""
    said = [m["content"] for m in request.messages if m["role"] == "user"][-1]
    results = sum(m["role"] == "toolResult" for m in request.messages)
    if said != "Do the job.":
        return lab.say("Carrying on.")
    if results == 0:
        return lab.reply(lab.text("Two steps first."),
                         lab.call("step", {"n": 1}), lab.call("step", {"n": 2}))
    if results == 2:
        return lab.reply(lab.text("One more."), lab.call("step", {"n": 3}))
    return lab.say("Done.")


async def walk_away_at(k):
    ran = []
    step = {"name": "step", "description": "Do one step of the job.",
            "parameters": {"type": "object",
                           "properties": {"n": {"type": "integer", "description": "Which step."}},
                           "required": ["n"]},
            "execute": lambda arguments: ran.append(arguments["n"]) or f"step {arguments['n']}: ok"}
    model = lab.ScriptedModel([lab.forever(job)])
    h = harness.Harness(model, harness.SYSTEM, [step])
    events = await lab.drive(h.prompt("Do the job."), stop_after=k)
    left = lab.shape(h.messages)
    for words in WORDS:
        await lab.drive(h.prompt(words))
    numbers = {block["id"]: block["arguments"]["n"] for m in h.messages
               if m["role"] == "assistant" for block in harness.tool_calls(m)}
    made_up = sorted({numbers[m["tool_call_id"]] for request in model.calls
                      for m in request.messages
                      if m["role"] == "toolResult" and m["content"] == harness.INTERRUPTED})
    refused = any(m.get("stop_reason") == "error" for m in h.messages)
    print(f"walked away at event {k} of 21, on {events[-1]['type']}")
    print(f"   steps that ran:             {ran}")
    print(f"   the record it left:         {left}")
    print(f"   two prompts later:          {'refused' if refused else 'answered, answered'}")
    print(f"   the record now:             {lab.shape(h.messages)}")
    print(f"   steps shown as interrupted: {made_up}")
    print(f"   ... that really ran:        {[n for n in made_up if n in ran]}")


for k in (5, 8, 14):
    await walk_away_at(k)

walked away at event 5 of 21, on tool_execution_start
   steps that ran:             []
   the record it left:         U A[c1,c2]
   two prompts later:          answered, answered
   the record now:             U A[c1,c2] U A U A
   steps shown as interrupted: [1, 2]
   ... that really ran:        []
walked away at event 8 of 21, on tool_execution_start
   steps that ran:             [1]
   the record it left:         U A[c1,c2] R(c1)
   two prompts later:          answered, answered
   the record now:             U A[c1,c2] R(c1) U A U A
   steps shown as interrupted: [2]
   ... that really ran:        []
walked away at event 14 of 21, on tool_execution_start
   steps that ran:             [1, 2]
   the record it left:         U A[c1,c2] R(c1) R(c2) A[c3]
   two prompts later:          answered, answered
   the record now:             U A[c1,c2] R(c1) R(c2) A[c3] U A U A
   steps shown as interrupted: [3]
   ... that really ran:        []

Three abandoned runs, six prompts after them, no refusals. Read the last two lines of each block together: the model was told that steps 1 and 2 were interrupted in the first run, step 2 alone in the second and step 3 in the third, and in none of the three was it told that about a step whose tool had actually finished. That is lesson 7's ordering rule paying for itself — a finished tool's result is on the record before anybody is told the tool finished — and it is why your made-up message can afford to say "no result was recorded" rather than "this did not happen".

And look at the records. Not one of them has been tidied: each still carries the call nobody answered, exactly where the run stopped.

Hold a conversation: it knows what was said, and who said it.
Run a tool the model asks for and show it the result.
Keep going until the model stops asking.
Tell the model when a tool fails, and carry on.
Stop a runaway, and survive a provider failure.
Keep every tool result within a budget.
Report what it is doing, as events, to any frontend.
Own its transcript: one writer at a time.
Take your input mid-run, at a safe point.
Send a valid transcript even after an interruption.

Say it in your own words

Your harness has just answered six prompts on top of three broken transcripts, and every one of those transcripts is still broken. In a sentence or two: why did you not fix them?

Somebody asks you tomorrow whether the tests ever ran. Which of the two lists can answer that, and what does the other one say?

Because the record's job is to be true, and a call with no result is the truth about that run. Filling the hole in the record would make the transcript valid by writing down something nobody observed, and it would be indistinguishable ever after from a message a tool produced. The repair belongs to the copy you send, which exists for one request and is thrown away.

Common answers, and what each one misses

"Because it would be more code." It is less code: you would repair once instead of on every request. The reason is not cost.
"Because the record is immutable." Nothing stops you appending to it; lesson 11 does very little else. What you must not do is append a claim that nothing supports.
"Because the model does not need it." The model does need it, on every request, which is exactly why the view has it. The record has a different reader: you, tomorrow.

Never send a broken file; never rewrite the original to hide that it broke.

Tau has the same function, with the same name, enforcing the same invariant, and its four decisions are the four you argued your way to on the cards — one of them settled more carefully than yours.

def repair_tool_history(messages):
    recorded = {}
    for m in messages:
        if m["role"] == "toolResult":
            recorded.setdefault(m["tool_call_id"], m)
    ...

def repair_tool_history(messages: tuple[AgentMessage, ...]) -> ToolHistoryRepair:
    """Return history where every tool call has exactly one adjacent result.

    Existing result messages are moved beside their calls. Missing results get a
    deterministic interruption error. Results with no call are omitted because
    a missing call's arguments cannot be reconstructed safely. When duplicate
    results exist, a real result is preferred over Tau's synthetic interruption.
    """

Tau is async; read async for as for and await f(x) as f(x) until lesson 15.

The same shapes.

The rebuild is your second pass, line for line: skip the results, copy everything else, and after each assistant message put back one result per call, in call order (src/tau_agent/tool_history.py:134-144).
It is applied to the view, not the record, and the function that does it says so in its name and its docstring: "an empty failed or aborted turn is not model context and must not poison the next request" (src/tau_agent/loop.py:185-201). That function is lesson 5's drop and today's repair, in that order: one filter and one call.
In the loop it runs on every request, never once at load: the one place it is called is where the provider call is built (src/tau_agent/loop.py:130). Tau's session layer repairs on load as well, which is the paragraph below this list.
The regression test is the three-line version of this page's opening, and its last two lines are the whole of I3: the orphan is not in what was replayed, and it is still in messages (tests/test_agent_loop.py:403-411).
The other four decisions have a test each (tests/test_tool_history.py:23,45,60,72,85), and the first of them is yours: a valid history comes back unchanged.
The reasoning is written down (the dev note), including the property you tapped for: "Applying the policy again is a no-op, so repeated resumes do not add more repair branches or diagnostics".

What Tau adds: it writes the repair down as well. Yours repairs the view and nothing else. Tau does that too, as a backstop, and on top of it appends the synthetic results to its own record at the start of every run (src/tau_agent/harness.py:168-173) and again on the way out of a cancelled one (src/tau_agent/harness.py:193-202). The comment above the first of those says why: so that the made-up results travel as events and reach the things that are listening, which in Tau means the screen and the file the session is saved in. Tau's session layer goes further still and never edits a line: it appends a repaired branch plus a tau.session-history-repair entry carrying the counts, leaving the original entries where they are (src/tau_coding/session.py:3425-3431).

And why it has to. Tau emits tool_execution_end before the events that announce the result message, and appends the message after both (src/tau_agent/loop.py:325-340). That leaves a window your loop does not have: a consumer told that a tool has ended, which stops listening right there, leaves a finished tool with no result on the record. Lesson 7's rule — mutate, then announce — is what closes it, and closing it is what lets the toy get away with repairing the view alone.

Where yours is weaker. You assume call ids are unique. Tau does not, because they are not: it pairs a result with a call by occurrence, reserving results that are already adjacent to their own turn first (src/tau_agent/tool_history.py:48-55,65-78). It also refuses to let a synthetic result stand where a real one is available, in a third pass written for exactly that (src/tau_agent/tool_history.py:113-132), where your rule is the blunter "the first result recorded wins". The question below is what that costs you.

A provider hands out c1 in one turn and c1 again two turns later. Each call got its own real result, recorded right where it belongs, and nothing is missing. What does your repair make of it?

Nothing: there is nothing to repair, so every message comes back where it was

There is nothing to repair, and the repair does not know that. It looks up results by id, and both calls have the same id.

It shows the second call the first call's result

"The first result for this id" is the whole of your rule, and here it is the wrong result for the second call.

It makes up an interrupted result for the second call, since its result was used up by the first

That would at least be visibly wrong. It is not what the code does: the mapping still has an entry for c1, so the second call is answered with it.

The model is told that b.py contains A = 1, with no sign anywhere that anything went wrong. The lab's strict model catches it here only because it also assumes unique ids within a request, and says so; a real provider that issued those ids would not. Tau pairs by occurrence for this reason. It is the one place in this lesson where the simplification can produce a confident lie rather than a missing file.

import harness, lab

first = lab.call("read", {"path": "a.py"}, id="c1")
second = lab.call("read", {"path": "b.py"}, id="c1")       # a provider reused the id
record = [lab.user("Read a.py, then b.py."),
          lab.reply(first), lab.tool_result(first, "A = 1\n"),
          lab.reply(second), lab.tool_result(second, "B = 2\n"),
          lab.user("Which is bigger?")]
view = harness.repair_tool_history(record)
print("the record:", lab.shape(record))
print("the view:  ", lab.shape(view))
print()
print("what the model is shown:")
print(lab.show(view))
print()
print("the strict model's verdict on the view:", lab.validate(view)[0])

the record: U A[c1] R(c1) A[c1] R(c1) U
the view:   U A[c1] R(c1) A[c1] R(c1) U

what the model is shown:
user -> "Read a.py, then b.py."
assistant -> toolCall c1 read({"path": "a.py"})
toolResult c1 -> "A = 1\n"
assistant -> toolCall c1 read({"path": "b.py"})
toolResult c1 -> "A = 1\n"
user -> "Which is bigger?"

the strict model's verdict on the view: messages[3]: toolCall id c1 is used twice in this request

What else yours cannot do. Nothing in your harness ever finds out that a repair happened. Tau counts what it changed and hands the counts out for diagnostics (src/tau_agent/tool_history.py:30-37); yours returns a list and says nothing, so a session that has been quietly patched on every request for a week looks exactly like a healthy one. A line in a log would fix that and is not on the syllabus.

Declared in Tau, read by nothing. AgentHarness exposes a public append_interrupted_tool_results() that returns how many it added (src/tau_agent/harness.py:234-237). Nothing in src/ calls it; the two places that repair the record call the private one beside it. Today it exists for a test.

Where both are guessing. [general] The refusal you have been reading all lesson is real, and its wording is not: Tau's note quotes one that is, No tool call found for function call output with call_id ... (dev-notes/tool-history-recovery.md:24), where the lab's stage prop says toolCall c1 has no toolResult. Our simplification: the lab's model checks the pairing itself and returns the refusal as an ordinary message, so you can see it without a network. What no provider tells you is why the pair is broken, which is the whole reason the synthetic result has to be written as carefully as it is.

src/tau_agent/tool_history.py:40-47 · pinned to commit 9fe6a71 · view on GitHub

One more case

Different provider, different complaint, same Friday afternoon.

A provider refuses any request in which two user messages sit next to each other. Your record has plenty: a steer and a follow-up that arrived together, a prompt right after a queued message. Do you fix the record, or the view? Write the rule you would apply, in one sentence, so that it also settles the next case like it.

One of the two lists has a reader who needs the request to be legal. The other has a reader who needs to know what actually happened.

The view. The rule: anything a particular provider needs is a property of the request, so it is computed in context_for_model; anything that happened is a property of the record, so it is appended and never changed. Two user messages in a row is a fact about what was typed, and merging them into one is a translation for a reader who cannot take them separately — which is what an adapter is, and lesson 16 builds one. [general] Most hosted APIs take two user messages in a row quite happily; some chat templates insist that roles alternate, and only those need the merge. Note which way the rule points on the day you meet a provider that does not mind: nothing changes, because the record never carried the accommodation in the first place.

Common answers, and what each one misses

"Merge them in the record, it is the same text." It is the same text and not the same event: two things typed a minute apart, one of them mid-run. Lesson 11 writes the record to a disk, and a merge is a line you can never get back.
"Refuse to queue two user messages in the first place." Now a provider's quirk has reached up into your harness's API and taken away something the user is entitled to do. Fix it where it is a problem: in the request.
"Add an empty assistant message between them." A message that nobody wrote, in the model's voice, to satisfy a shape check. You spent this whole lesson on what makes that a bad idea.

You hit

a user who walked away mid-tool, a record ending on a call nobody answered, and a session in which every later prompt was refused

You built

INTERRUPTED and repair_tool_history(): pure, idempotent, called from context_for_model on every request

The principle

Never send a broken file; never rewrite the original to hide that it broke. The record is not the view.

Your harness now

run_tool
run_agent
context_for_model
INTERRUPTED
repair_tool_history
Harness
steer
follow_up
FinalTextRenderer
JsonRenderer

Your answers

Still open

Everything your harness knows is a Python list in one process. The agent writes two files, and somebody trips over the power cable. Lesson 11.

Previous 09 But I had something to say Next Checkpoint B Transcript doctor II Five harnesses, five ways of being nearly right. Which idea is missing from each?