side quest S1 · optional

One line, exactly once

The model read 2,000 lines of a 3,000-line file, wrote the fix back with your write tool, and was told "Successfully wrote to app.py."

~50 min · 1 lab · builds on 06 The firehose

What this page is

A side quest: optional, about fifty minutes, and nothing after it depends on it. Checkpoint A and lessons 7 to 16 are all written against the harness.py you had at the end of lesson 6, so you can do this page now, later, or never, and the course carries on either way. It exists because one tool in your harness is quietly the most expensive thing you own.

The shape is a lesson's: a run that goes wrong, a few questions with a cell behind each, one lab with hidden tests, and the comparison with Tau at the end.

A one-line fix

app.py is 3,000 lines of somebody's working program. Line 12 says TIMEOUT = 30 and it should say 5. That is the whole job, and your harness has everything it needs: read, budgeted since lesson 6, and write, which has been in your tool list since lesson 4 and does exactly what its description says — it creates a file, or replaces the whole of it.

So the model reads the file, and then it writes the file back with one line changed. There is no other move: your write tool replaces the whole file, so the only way it can change one line is to be handed every other line as well.

The run is two tool calls: read, then write. Nothing fails, nothing raises, and the model does its job perfectly. How many lines does app.py have when the run is over?

Two thousand. Your read tool kept to its budget and showed the model the first 2,000 lines, with the notice that told it so; the model then wrote back the file it had been shown. A thousand lines of that program are gone, the tool said Successfully wrote to app.py., and nothing in the run noticed. Run the cell and look at the last two figures as well.

Your lesson 6 harness, unchanged, on the one-line fix. The model is scripted to write back the text it was shown, with TIMEOUT = 30 replaced and the truncation notice dropped, because a notice in brackets is plainly not part of the program. It invents nothing and it changes one line.

import harness, lab

# app.py: 3,000 short lines of somebody's working program. Line 12 is the bug.
LINES = [f"    step_{n:04d}()\n" for n in range(1, 3001)]
LINES[11] = "    TIMEOUT = 30\n"
APP = "".join(LINES)


def fix_the_whole_file(request):
    """The model writes back the file it was shown, with the one line changed. It drops the
    truncation notice, because a notice in brackets is plainly not part of the program."""
    shown = request.last_result["content"]
    shown = shown[:shown.index("[Showing")]
    return lab.reply(lab.call("write", {"path": "app.py",
                                        "content": shown.replace("TIMEOUT = 30", "TIMEOUT = 5")}))


ws = lab.Workspace({"app.py": APP})
model = lab.ScriptedModel([lab.reply(lab.call("read", {"path": "app.py"})),
                           fix_the_whole_file,
                           lab.say("Done. TIMEOUT is 5 seconds now.")])
messages = []
harness.run_agent(model, harness.SYSTEM, messages,
                  [harness.make_read_tool(ws), harness.make_write_tool(ws)],
                  "app.py times out too late. Set TIMEOUT to 5.")

read_result, write_call, write_result = messages[2], messages[3], messages[4]
shown = read_result["content"]
sent = harness.tool_calls(write_call)[0]["arguments"]["content"]
after = ws.read_text("app.py")

print(f"app.py, before the run          {len(APP.splitlines()):,} lines")
print(f"the read tool showed the model  {len(shown.splitlines()) - 1:,} lines, then this notice:")
print(f"                                {shown.splitlines()[-1]}")
print(f"the model wrote back            {len(sent.splitlines()):,} lines, "
      f"{write_call['usage']['output']:,} output tokens, to change one of them")
print(f"the tool answered               {write_result['content']} "
      f"(is_error={write_result['is_error']})")
print(f"app.py, after the run           {len(after.splitlines()):,} lines")
print()
print(f"Lines of that program now gone: {len(APP.splitlines()) - len(after.splitlines()):,}. "
      "Line 12 does say TIMEOUT = 5.")
print(f"The run cost {model.bill:,} input tokens and "
      f"{sum(m['usage']['output'] for m in messages if 'usage' in m):,} output tokens.")

app.py, before the run          3,000 lines
the read tool showed the model  2,000 lines, then this notice:
                                [Showing lines 1-2000 of 3000. Use offset=2001 to continue.]
the model wrote back            2,000 lines, 8,525 output tokens, to change one of them
the tool answered               Successfully wrote to app.py. (is_error=False)
app.py, after the run           2,000 lines

Lines of that program now gone: 1,000. Line 12 does say TIMEOUT = 5.
The run cost 26,372 input tokens and 8,562 output tokens.

Two costs, and the smaller one is the bill. Eight and a half thousand output tokens to change one line is money and a wait, and it is at least visible. The other cost is that a tool called write did what it promised and truncated a file, because the only file the model could give it was the part of the file it had seen.

You cannot fix this inside write. A tool that replaces whole files has one argument that means "the new contents", and any model using it has to produce them. The fix is a different tool.

Where this comes from: Tau ships a write tool and an edit tool side by side (src/tau_coding/tools.py:170-185), and tells the model in so many words to use the second one for precise changes (src/tau_coding/tools.py:548-551). The 2,000 lines your read showed the model is Tau's budget too, the same one lesson 6 copied (src/tau_coding/tools.py:41-42).

Send the change, not the file

The tool almost writes itself. The model names the text it wants gone and the text that replaces it, and Python has a method for that:

def execute(arguments):
    path = str_arg(arguments, "path")
    old = str_arg(arguments, "oldText")
    new = str_arg(arguments, "newText")
    if not ws.exists(path):                      # the check read has had since lesson 4
        raise FileNotFoundError(
            f"File not found: {path}. Files here: {', '.join(ws.listdir('.'))}")
    ws.write_text(path, ws.read_text(path).replace(old, new))
    return f"Successfully edited {path}."

Nine lines, the arguments checked the way lesson 4 checks them, the missing path answered the way read answers it. Below are five calls a model could make to it, against a thirteen-line app.py in which the line print("fetching", url) appears twice — once in fetch, once in retry.

Tap every call this tool answers with Successfully edited app.py.

the text the model named is in the file twice
the text is not in the file: the model typed two spaces where the file has one
the text is in the file exactly once
newText is the same as oldText
the path is not in the workspace

Four of the five. The only call this tool refuses is the one the borrowed path check catches: there is no app.pyc in the workspace, so the check raises and lesson 4's boundary turns it into an error result. Every other call is a success as far as the tool can tell, including the one that changed two lines when the model meant one and the two that changed nothing at all.

The model cannot see the file. The result string is the only thing it gets, and of those four identical sentences only one is something the model could safely act on.

The five calls, each against a fresh copy of app.py: what the model is told, and how many of the thirteen lines actually changed.

import harness, lab

APP = ("import json\n"
       "\n"
       "TIMEOUT = 30\n"
       "\n"
       "\n"
       "def fetch(url):\n"
       '    print("fetching", url)\n'
       '    return {"url": url, "timeout": TIMEOUT}\n'
       "\n"
       "\n"
       "def retry(url):\n"
       '    print("fetching", url)\n'
       "    return fetch(url)\n")

PRINT_LINE = '    print("fetching", url)'


def make_naive_edit_tool(ws):
    """Send the change, not the file: one replacement, straight through str.replace."""
    def execute(arguments):
        path = harness.str_arg(arguments, "path")
        old = harness.str_arg(arguments, "oldText")
        new = harness.str_arg(arguments, "newText")
        if not ws.exists(path):                      # the check read has had since lesson 4
            raise FileNotFoundError(
                f"File not found: {path}. Files here: {', '.join(ws.listdir('.'))}")
        ws.write_text(path, ws.read_text(path).replace(old, new))
        return f"Successfully edited {path}."

    return {"name": "edit",
            "description": "Change a text file by replacing exact text.",
            "parameters": {"type": "object",
                           "properties": {"path": {"type": "string"},
                                          "oldText": {"type": "string"},
                                          "newText": {"type": "string"}},
                           "required": ["path", "oldText", "newText"]},
            "execute": execute}


CALLS = [
    ("the text is in the file twice", PRINT_LINE, "    log(url)"),
    ("the text is not in the file (two spaces, not one)", "TIMEOUT  = 30", "TIMEOUT = 5"),
    ("the text is in the file exactly once", "TIMEOUT = 30", "TIMEOUT = 5"),
    ("newText is the same as oldText", "TIMEOUT = 30", "TIMEOUT = 30"),
]

for what, old, new in CALLS:
    ws = lab.Workspace({"app.py": APP})
    result = harness.run_tool([make_naive_edit_tool(ws)],
                              lab.call("edit", {"path": "app.py", "oldText": old,
                                                "newText": new}, id="c1"))
    changed = sum(1 for was, now in zip(APP.splitlines(), ws.read_text("app.py").splitlines())
                  if was != now)
    print(f"{what}:")
    print(f"    the model is told   {result['content']}  (is_error={result['is_error']})")
    print(f"    lines changed       {changed} of 13")

ws = lab.Workspace({"app.py": APP})
result = harness.run_tool([make_naive_edit_tool(ws)],
                          lab.call("edit", {"path": "app.pyc", "oldText": "TIMEOUT = 30",
                                            "newText": "TIMEOUT = 5"}, id="c1"))
print("the path is not in the workspace:")
print(f"    the model is told   {result['content']}  (is_error={result['is_error']})")
print(f"    lines changed       {0} of 13")

the text is in the file twice:
    the model is told   Successfully edited app.py.  (is_error=False)
    lines changed       2 of 13
the text is not in the file (two spaces, not one):
    the model is told   Successfully edited app.py.  (is_error=False)
    lines changed       0 of 13
the text is in the file exactly once:
    the model is told   Successfully edited app.py.  (is_error=False)
    lines changed       1 of 13
newText is the same as oldText:
    the model is told   Successfully edited app.py.  (is_error=False)
    lines changed       0 of 13
the path is not in the workspace:
    the model is told   File not found: app.pyc. Files here: app.py  (is_error=True)
    lines changed       0 of 13

So the tool has to answer three questions before it touches anything. Is the text there at all? Is it there exactly once? And did anything actually change? Each one has a failure behind it that a success message would hide: an edit that matched nothing, an edit that hit the wrong one of two identical lines, an edit that replaced text with itself and told the model its work was done.

That is one edit. Now let the model send several in one call, which is what it will do the moment a refactor needs two lines changed in the same file.

A tool that takes a list of edits and does each one as it comes. The model sends two: TIMEOUT = 30 to 5, and one that quotes the last line as return fetch(url), with two spaces where the file has one. The second edit matches nothing, so the tool raises and the call comes back an error. What has happened to the file, and what does the model do next?

The first edit is on disk. The model is told the call failed, so it fixes the edit it was told about and sends both again — and now the first one cannot be applied either

Both calls came back as errors, and the file changed anyway. Look at the two lines of app.py the cell prints at the end: the change that was actually asked for never happened, and one the model has twice been told did not happen is on disk.

The same thing as checking every edit up front. Whether you check first or check as you go, that edit fails and that error comes back; where the check sits is tidiness

Where a check sits decides what is true when it fires. Check first and the file is untouched when the refusal goes out, so the model can send the call again; check as you go and the refusal is about a file that has already moved under it.

It should answer applied 1 of 2. A half-applied call is unusual, and saying so is honest enough for the model to carry on from

Honest, and unusable. The model now has to work out which half landed, from a file it cannot see, and every retry it writes depends on that guess. The tool is the only party that knows, and it can arrange never to be in that state.

The model did the one sensible thing with the refusal it was handed, and made the file worse. A call that comes back as an error tells the model that it did not happen; the only way to keep that promise is to apply nothing until every edit is known to be applicable. All of them or none, and the tool has to know which before it writes a byte.

import harness, lab

APP = ("import json\n"
       "\n"
       "TIMEOUT = 30\n"
       "\n"
       "\n"
       "def fetch(url):\n"
       '    print("fetching", url)\n'
       '    return {"url": url, "timeout": TIMEOUT}\n'
       "\n"
       "\n"
       "def retry(url):\n"
       '    print("fetching", url)\n'
       "    return fetch(url)\n")


def make_eager_edit_tool(ws):
    """Many edits in one call, each one done as it comes."""
    def execute(arguments):
        path = harness.str_arg(arguments, "path")
        edits = arguments["edits"]
        for index, edit in enumerate(edits):
            content = ws.read_text(path)
            if edit["oldText"] not in content:
                raise ValueError(f"Could not find edits[{index}] in {path}.")
            ws.write_text(path, content.replace(edit["oldText"], edit["newText"], 1))
        return f"Successfully edited {path} ({len(edits)} edit(s) applied)."

    return {"name": "edit", "description": "Change a text file by replacing exact text.",
            "parameters": {"type": "object",
                           "properties": {"path": {"type": "string"},
                                          "edits": {"type": "array"}},
                           "required": ["path", "edits"]},
            "execute": execute}


TIMEOUT_EDIT = {"oldText": "TIMEOUT = 30", "newText": "TIMEOUT = 5"}
TYPO = {"oldText": "    return  fetch(url)", "newText": "    return fetch(url.lower())"}
FIXED = {"oldText": "    return fetch(url)", "newText": "    return fetch(url.lower())"}


def step(request):
    """The model sends both changes in one call. Told the call failed, it fixes the edit the
    message named and sends the call again: a call that failed did not happen."""
    result = request.last_result
    if result is None:
        return lab.reply(lab.call("edit", {"path": "app.py", "edits": [TIMEOUT_EDIT, TYPO]}))
    if not result["is_error"]:
        return lab.say("Both changes are in.")
    if "edits[1]" in result["content"]:
        return lab.reply(lab.call("edit", {"path": "app.py", "edits": [TIMEOUT_EDIT, FIXED]}))
    return lab.say("I am being refused for an edit I have already been told was fine. Stopping.")


ws = lab.Workspace({"app.py": APP})
model, messages = lab.ScriptedModel([lab.forever(step)]), []
harness.run_agent(model, harness.SYSTEM, messages, [make_eager_edit_tool(ws)],
                  "Set TIMEOUT to 5, and lower-case the url in retry().")

results = [m for m in messages if m["role"] == "toolResult"]
lines = ws.read_text("app.py").splitlines()

print("call 1: TIMEOUT -> 5, and one edit with two spaces where the file has one")
print(f"    the model is told   {results[0]['content']}  (is_error={results[0]['is_error']})")
print("call 2: the model fixes the edit the message named, and sends both again")
print(f"    the model is told   {results[1]['content']}  (is_error={results[1]['is_error']})")
print()
print("app.py, after a run in which the model was told twice that its call failed:")
print(f"    line  3    {lines[2]}       <- edit 0 of call 1, applied and never mentioned again")
print(f"    line 13    {lines[12].strip()}      <- the change that was actually asked for")
print()
print(f"record: {lab.shape(messages)} . the workspace was written {len(ws.writes)} time(s)")

call 1: TIMEOUT -> 5, and one edit with two spaces where the file has one
    the model is told   Could not find edits[1] in app.py.  (is_error=True)
call 2: the model fixes the edit the message named, and sends both again
    the model is told   Could not find edits[0] in app.py.  (is_error=True)

app.py, after a run in which the model was told twice that its call failed:
    line  3    TIMEOUT = 5       <- edit 0 of call 1, applied and never mentioned again
    line 13    return fetch(url)      <- the change that was actually asked for

record: U A[c1] R(c1) A[c2] R(c2) A . the workspace was written 1 time(s)

Which file is an edit about?

So: check first, then apply. That leaves a question that sounds like a detail and is not. When you look for the second edit's text, which file do you look in — the one the model read, or the one that exists after the first edit has been applied?

Two edits. The first adds the comment # TIMEOUT = 30 until the July incident under the imports; the second changes the real TIMEOUT = 30 further down to 5. The tool searches the file as it stands, each edit in turn. What comes out?

Exactly what the model asked for. It sent the edits in an order, so it was thinking of the file as it changes, comment and all

It saw the file once, in a tool result, and wrote both edits against that text. Nothing shows it the file between two edits of one call, and nothing could: the call has not come back yet.

The second edit lands on the comment the first one just wrote, and the constant on line 4 still says 30

Neither edit did anything wrong. The second one was matched against a file that did not exist when the model wrote it.

Nothing either way. Real edits are in different parts of a file, so searching as you go and searching the original come to the same thing

They come to the same thing on the calls where it does not matter. The interesting call is the one where an edit's replacement text contains another edit's search text, which is exactly what renaming a thing looks like.

Every edit is found in the content the model was given, before any of them is applied. Then the second half of the same rule: apply them from the back, because a replacement that is longer or shorter than the text it replaces moves everything after it. The second half of the cell is that mistake — right positions, applied front to back — and it swallowed the def retry(url): line, the line break after it and the first letter of print.

# No model and no tools here: just the text, and three ways of putting the edits into it.

APP = ("import json\n"
       "\n"
       "TIMEOUT = 30\n"
       "\n"
       "\n"
       "def fetch(url):\n"
       '    print("fetching", url)\n'
       '    return {"url": url, "timeout": TIMEOUT}\n'
       "\n"
       "\n"
       "def retry(url):\n"
       '    print("fetching", url)\n'
       "    return fetch(url)\n")


def as_it_goes(content, edits):
    """Each edit searched for in the file as it stands, one after another."""
    for edit in edits:
        content = content.replace(edit["oldText"], edit["newText"], 1)
    return content


def from_the_front(content, edits):
    """Every edit found in the original, then applied first to last."""
    spans = sorted((content.index(e["oldText"]),
                    content.index(e["oldText"]) + len(e["oldText"]), e["newText"]) for e in edits)
    result = content
    for start, end, new in spans:
        result = result[:start] + new + result[end:]
    return result


def from_the_back(content, edits):
    """Every edit found in the original, then applied last to first."""
    spans = sorted((content.index(e["oldText"]),
                    content.index(e["oldText"]) + len(e["oldText"]), e["newText"]) for e in edits)
    result = content
    for start, end, new in reversed(spans):
        result = result[:start] + new + result[end:]
    return result


def show(title, text, first, last):
    print(title)
    for number, line in list(enumerate(text.splitlines(), start=1))[first - 1:last]:
        print(f"    {number:>2}  {line}")


COMMENT = [{"oldText": "import json",
            "newText": "import json\n# TIMEOUT = 30 until the July incident"},
           {"oldText": "TIMEOUT = 30", "newText": "TIMEOUT = 5"}]

print("Two edits. The first writes a comment that happens to contain the words of the second.")
show("  each edit searched for in the file as it stands:", as_it_goes(APP, COMMENT), 1, 4)
show("  each edit found in the file the model read:", from_the_back(APP, COMMENT), 1, 4)
print()

APART = [{"oldText": "TIMEOUT = 30",
          "newText": "TIMEOUT = 5  # seconds; lowered after the July incident"},
         {"oldText": "    return fetch(url)", "newText": "    return fetch(url.lower())"}]

grew = len(APART[0]["newText"]) - len(APART[0]["oldText"])
print(f"Two edits far apart, both found in the original. The first one is {grew} characters")
print("longer than the text it replaces.")
show("  applied first to last:", from_the_front(APP, APART), 11, 13)
print(f"    (13 lines went in; {len(from_the_front(APP, APART).splitlines())} came out)")
show("  applied last to first:", from_the_back(APP, APART), 11, 13)
print(f"    (13 lines went in; {len(from_the_back(APP, APART).splitlines())} came out)")

Two edits. The first writes a comment that happens to contain the words of the second.
  each edit searched for in the file as it stands:
     1  import json
     2  # TIMEOUT = 5 until the July incident
     3  
     4  TIMEOUT = 30
  each edit found in the file the model read:
     1  import json
     2  # TIMEOUT = 30 until the July incident
     3  
     4  TIMEOUT = 5

Two edits far apart, both found in the original. The first one is 43 characters
longer than the text it replaces.
  applied first to last:
    11      return fetch(url.lower())rint("fetching", url)
    12      return fetch(url)
    (13 lines went in; 12 came out)
  applied last to first:
    11  def retry(url):
    12      print("fetching", url)
    13      return fetch(url.lower())
    (13 lines went in; 13 came out)

What a refusal is for

Every reason to refuse a call ends up as a sentence somebody has to read. That somebody is the model, which has your description of the tool, the file as it stood when it read it, and whatever your result says. Nothing else. So the first question is which calls to refuse at all.

Your tool checks the whole call before it writes anything. Tap every call it must refuse.

two edits that cover some of the same characters
two edits where one ends exactly where the next begins
two edits with the same oldText, which is in the file once
one edit whose oldText is the empty string
one edit whose newText is the empty string
a set of edits that would leave the file exactly as it was

Four to refuse, two to apply.

Overlapping edits have no answer: whichever you apply second is written over text the first one replaced, and the result is a file neither edit asked for.
Edits that touch are ordinary neighbours. Refuse those and a model changing two adjacent lines has to send them as one edit, every time.
The same oldText twice is the overlap rule doing its job, not a rule of its own: each of them matched the same single place in the file.
An empty oldText sits between every pair of characters in the file. It names no one place, which is the only thing an edit is for.
An empty newText is how you delete a line. Nothing wrong with it.
A call that changes nothing is the one people leave out. A success for a file that did not change tells the model its work is done, and it moves on to the next thing.

One of the four is about a single edit; the other three can only be decided by looking at the whole call, which is the reason nothing may be written until every edit has been looked at.

Which leaves the wording. A refusal is true or it is not, and being true is the easy half; the message also has to leave the model somewhere it can go next. The cell below runs the same model against the same file twice, through the same tool, and changes nothing but the sentence that comes back when an edit matches two places.

The model asks to replace the print inside retry(), naming just that line — which appears in fetch() too. One run answers Edit failed.; the other answers Found 2 occurrences of edits[0] in the file. Each oldText must be unique: provide more context to make it unique. What differs at the end of the two runs?

Nothing that matters to the run. Both refusals are true; the wording is for whoever reads the logs afterwards, and the model widens the edit either way

There is no log here and nobody is reading one. The result is a message in a conversation, and it is the only thing the model learns from this call.

Neither run should get that far. An edit the harness cannot apply is a fault in the model's plan, so the run should stop and a person should look at it

A person cannot see the two identical lines either, and the model fixed it in one turn when told what was wrong. Lesson 4's rule holds here exactly as it did there: the reader who can fix this is the model.

The second run finishes the job: the model sends the same change with the line above it and retry() is fixed. The first run stops with the file untouched

Same model, same tool, same file. The only thing that changed is what the tool said about why it would not do the work.

One turn and 435 input tokens between a job done and a job abandoned. The refusal is doing the work of a tool description, at the moment it is needed: it names the count, so the model knows the text was not wrong but ambiguous, and it names the move, so the model knows what a good second attempt looks like. Error text you write for a model is an instruction, which is lesson 4 again with a different tool on the end of it.

Where this wording comes from: it is Tau's, shortened. Tau's refusal for the same case says Found N occurrences of edits[i] in <path>. Each oldText must be unique. Please provide more context to make it unique. (src/tau_coding/tools.py:1140-1149).

import textwrap

import harness, lab

APP = ("import json\n"
       "\n"
       "TIMEOUT = 30\n"
       "\n"
       "\n"
       "def fetch(url):\n"
       '    print("fetching", url)\n'
       '    return {"url": url, "timeout": TIMEOUT}\n'
       "\n"
       "\n"
       "def retry(url):\n"
       '    print("fetching", url)\n'
       "    return fetch(url)\n")

NARROW = [{"oldText": '    print("fetching", url)', "newText": "    log(url)"}]
WIDE = [{"oldText": 'def retry(url):\n    print("fetching", url)',
         "newText": "def retry(url):\n    log(url)"}]


def make_terse_edit_tool(ws):
    """The same tool, with every refusal shortened to one true sentence."""
    tool = harness.make_edit_tool(ws)
    full = tool["execute"]

    def execute(arguments):
        try:
            return full(arguments)
        except ValueError:
            raise ValueError("Edit failed.")

    return {**tool, "execute": execute}


def step(request):
    """The model asks to change the print inside retry(). If it is refused, it looks at what
    the refusal says: a count and a request for more context is something it can act on."""
    result = request.last_result
    if result is None:
        return lab.reply(lab.call("edit", {"path": "app.py", "edits": NARROW}))
    if not result["is_error"]:
        return lab.say("Done: retry() logs instead of printing.")
    if "occurrence" in result["content"].lower() and "more context" in result["content"].lower():
        return lab.reply(lab.call("edit", {"path": "app.py", "edits": WIDE}))
    return lab.say("The edit was refused and I cannot tell what to send instead, so I stopped.")


def field(label, text):
    """One labelled line, wrapped under its own label."""
    body = textwrap.wrap(text, 60) or [""]
    print(f"    {label:<20}{body[0]}")
    for line in body[1:]:
        print(f"    {'':<20}{line}")


for title, make_tool in [("a refusal that is true and says nothing else", make_terse_edit_tool),
                         ("a refusal written for the reader who can act on it",
                          harness.make_edit_tool)]:
    ws = lab.Workspace({"app.py": APP})
    model, messages = lab.ScriptedModel([lab.forever(step)]), []
    harness.run_agent(model, harness.SYSTEM, messages, [make_tool(ws)],
                      "Make retry() log instead of print.")
    lines = ws.read_text("app.py").splitlines()
    print(f"{title}:")
    field("the model is told", messages[2]["content"])
    field("it answers", harness.text_of(messages[-1]))
    field("fetch(), line  7", lines[6].strip())
    field("retry(), line 12", lines[11].strip())
    field("record", f"{lab.shape(messages)}, {model.bill:,} input tokens")

a refusal that is true and says nothing else:
    the model is told   Edit failed.
    it answers          The edit was refused and I cannot tell what to send instead,
                        so I stopped.
    fetch(), line  7    print("fetching", url)
    retry(), line 12    print("fetching", url)
    record              U A[c1] R(c1) A, 500 input tokens
a refusal written for the reader who can act on it:
    the model is told   Found 2 occurrences of edits[0] in the file. Each oldText
                        must be unique: provide more context to make it unique.
    it answers          Done: retry() logs instead of printing.
    fetch(), line  7    print("fetching", url)
    retry(), line 12    log(url)
    record              U A[c1] R(c1) A[c2] R(c2) A, 935 input tokens

Build: an edit that names one place

Two functions, both in region 2, under make_write_tool. The split between them is lesson 4's boundary: make_edit_tool deals with the model's arguments, the workspace and the path, and apply_edits deals with text. apply_edits never learns which file it is working on, so no refusal it writes can name a file; the ones that are about one edit name it as edits[i].

Your harness.py from the end of lesson 6, with the two new functions added and their bodies cut out. About forty lines of code between the two gaps, which is more than any single lab in this course asks for; a side quest can afford to be longer. Nothing else in the file changes, and nothing else in it calls them.

apply_edits(content, edits) returns the content with every edit applied, or raises ValueError naming the first edit that cannot be. An edit is {"oldText": ..., "newText": ...}, both strings, already checked by the tool. Five refusals, all of them written for the model: an empty oldText; an oldText that is not in the content (say that the match is exact, whitespace and all); an oldText that is there more than once (say how many times, and ask for more context); two edits that cover the same characters; a call that would change nothing. Every edit is found in the content you were handed, and the replacements are made from the back.
make_edit_tool(ws)'s execute, where path is already read out for you. arguments["edits"] is whatever the model sent, straight from JSON, so check its shape here the way str_arg checks a string: a non-empty list, every entry an object with a string oldText and a string newText, and say which entry was wrong. The path that is not in the workspace gets read's answer, word for word. Then read the file, put it through apply_edits, write the result back and tell the model what happened — on exactly one path through the function, because every other path raised.

Fourteen hidden tests. Three are about edits that land: one edit that changes only what it names, two edits that both land although the first one makes the file far longer, and two edits that meet end to start. Six are the refusals, and most of them read the message as well — the count and "more context" for the text that is there twice, the word "exact" for the near miss, the word "empty", the word "overlap", edits[1] for the edit that would only have matched after another had been applied, and the call that changes nothing. The last five are the tool: every kind of refused call leaves ws.snapshot() as it was and writes nothing at all; a successful call writes once and names the file; eight sets of arguments the model got wrong, a missing path among them, come back as error results rather than crashes; the tool describes itself with both arguments and the word "unique"; and one run of a model that is refused for ambiguity, reads the message, and retries with the line above.

There is no "what changed" diff on this lab. A side quest is not part of the code thread, so this starter is lesson 6's solution with two functions added, and there was no previous lab to diff it against.

Take the two gaps in order, and start with what has to be true before you may write a single byte. Which of the five refusals can you decide by looking at one edit and the content, and which one needs every edit at once? Then: a match is a stretch of the content, from where the text starts to where it ends. If you wrote all of those down before touching the text, what would you have to do to each of them when a replacement in front of it is longer than the text it replaced — and what order would let you skip that work altogether?
One pass over edits with enumerate, because a refusal has to name which edit failed. Per edit: refuse an empty oldText; count the matches with content.count(old) and refuse 0 and more than 1 with their own messages; then content.index(old) and keep the triple (start, start + len(old), newText). Sort the triples: two neighbours overlap when the second one's start is before the first one's end, which leaves edits that touch legal. Then walk the sorted triples in reverse, cutting each span out of the result and dropping the new text in; since you are working backwards, no span you have not used yet can have moved. Compare the finished text with what you were given and refuse if they are equal — one check on the result, rather than a check on each edit.
In the tool: arguments.get("edits"), refuse anything that is not a non-empty list, then walk it and refuse any entry that is not a dict with two string values, naming edits[i]. Copy read's ws.exists check and its FileNotFoundError message exactly. What is left is four lines: read, apply, write, return a sentence with the path in it.

In outline.

apply_edits(content, edits):
    spans = []
    for index, edit in enumerate(edits):
        old = edit["oldText"]
        if old is empty:            raise ValueError(f"edits[{index}] ... empty ...")
        found = how many times old is in content
        if found == 0:              raise ValueError(... must match exactly ...)
        if found > 1:               raise ValueError(f"Found {found} occurrences of "
                                                     f"edits[{index}] ... more context ...")
        start = where old is in content
        spans.append((start, start + len(old), edit["newText"]))
    sort spans
    for each neighbouring pair: if the next start is before this end, raise (overlap)
    result = content
    for start, end, new in the spans, last first:
        result = result[:start] + new + result[end:]
    if result == content:           raise ValueError(... nothing would change ...)
    return result

make_edit_tool's execute, with path already read:
    edits = arguments.get("edits")
    refuse anything that is not a non-empty list
    refuse any entry that is not an object with a string oldText and a string newText
    if the path is not in ws:  raise FileNotFoundError, read's message word for word
    content = ws.read_text(path)
    updated = apply_edits(content, edits)      # raises for the whole call, or returns
    ws.write_text(path, updated)
    return a sentence naming the path and how many edits were applied

Fourteen tests, and the one worth reading twice is the last. A model that could not see your file asked for a change, was refused by a sentence your code wrote, worked out from that sentence what to send instead, and got it right — and the run ended with one line changed and one write to the workspace. Nothing in your harness knows anything about retrying. That behaviour is what a refusal buys when it says how many and what to do.

One edit replaces exactly the text it names and leaves the rest of the file alone.
Two edits at different places both land, whatever order they arrive in and however much the earlier one changes the file's length.
Two edits that meet end to start are both applied: they do not share a character.
An oldText that matches twice is refused, and the message says how many matches there were and asks for more context.
An oldText that is not in the file, byte for byte, is refused: whitespace counts.
An edit that would only match after an earlier edit has been applied is refused: the model wrote its edits against the file it read, not against a file it never saw.
An empty oldText is refused: it names no place in the file, or every place.
Two edits whose matches overlap are refused, rather than one of them winning.
A set of edits that would leave the file exactly as it was is refused.
A successful edit call writes the file once and tells the model which path it changed.
Every kind of refused edit leaves the file on disk exactly as it was, unwritten.
Arguments the model got wrong, and a path that is not there, come back as error results naming what was wrong, and nothing is written.
A model whose first edit matches twice reads the error, sends the same change with the line above it, and the run ends with only that one place changed.
The model is told about `path` and `edits`, and that each oldText must be unique.

# == 1 messages == # A conversation is a list of messages. Each message is a dict tagged with the role that spoke. SYSTEM = "You are a helpful assistant." # standing instructions, sent with every call def user_message(text): """A message from the user. Its content is a plain string.""" return {"role": "user", "content": text} def text_of(message): """The words of an assistant message. Its content is a list of blocks, not a string (lesson 2 explains why); the words are in the blocks of type "text".""" return "".join(block["text"] for block in message["content"] if block["type"] == "text") def tool_calls(message): """The tool calls an assistant message asks for, in the order it asked. Worked out from `content` on every call and stored nowhere else.""" return [block for block in message["content"] if block["type"] == "toolCall"] def error_message(text): """An assistant message that says why a run stopped: nothing in `content`, stop_reason "error", and the reason in `error_message`. A stop is written into the record, not raised.""" return {"role": "assistant", "content": [], "stop_reason": "error", "error_message": text} # == 2 tools == # A tool is a dict. "name", "description" and "parameters" are advertised to the model. # "execute" is a function, execute(arguments) -> str, that only this program ever runs. def str_arg(arguments, name): """The string argument `name`, checked. The schema asked the model for a string and nobody made it send one, so every tool reads its arguments through here. Raises ValueError("<name> must be a string") when it is missing or not a string.""" value = arguments.get(name) if not isinstance(value, str): raise ValueError(f"{name} must be a string") return value def int_arg(arguments, name, default): """The optional whole-number argument `name`, checked; `default` if the model left it out. Raises ValueError("<name> must be an integer") for anything else.""" value = arguments.get(name) if value is None: return default if not isinstance(value, int): raise ValueError(f"{name} must be an integer") return value def tool_specs(tools): """What the model is told about the tools: everything except the function.""" return [{"name": tool["name"], "description": tool["description"], "parameters": tool["parameters"]} for tool in tools] def run_tool(tools, call): """Run one tool call and return the toolResult message that answers it. One call in, exactly one result out, on every path. Tools just raise; this is the one place that turns a failure into a result the model can read. is_error means the tool could not do its job. It never raises Exception.""" tools_by_name = {tool["name"]: tool for tool in tools} tool = tools_by_name.get(call["name"]) if tool is None: content, is_error = f"Tool {call['name']} not found", True else: try: content, is_error = tool["execute"](call["arguments"]), False except Exception as exc: content, is_error = str(exc), True return {"role": "toolResult", "tool_call_id": call["id"], "tool_name": call["name"], "content": content, "is_error": is_error} MAX_LINES = 2000 # The budget of one tool result. Whatever a tool returns is re-read, and MAX_BYTES = 50_000 # paid for, on every later turn. Bytes, not characters: "é" is two. def truncate_head(text, max_lines=MAX_LINES, max_bytes=MAX_BYTES): """The start of `text` that fits the budget, for files: (kept, lines_shown). Whole lines only, newlines included, sized in UTF-8 bytes. If even the first line is too big, the answer is ("", 0): half a line is worse than none.""" kept, size = [], 0 for line in text.splitlines(keepends=True): size += len(line.encode()) if len(kept) == max_lines or size > max_bytes: break kept.append(line) return "".join(kept), len(kept) def truncate_tail(text, max_lines=MAX_LINES, max_bytes=MAX_BYTES): """The end of `text` that fits the budget, for command output, where the verdict comes last: (kept, lines_shown). Whole lines only, newlines included, sized in UTF-8 bytes.""" kept, size = [], 0 for line in reversed(text.splitlines(keepends=True)): size += len(line.encode()) if len(kept) == max_lines or size > max_bytes: break kept.append(line) return "".join(reversed(kept)), len(kept) def make_read_tool(ws, max_lines=MAX_LINES, max_bytes=MAX_BYTES): """The `read` tool, over the workspace `ws`. One call returns at most the budget, and says so in a last line that tells the model how to ask for the rest.""" def execute(arguments): path = str_arg(arguments, "path") offset, limit = int_arg(arguments, "offset", 1), int_arg(arguments, "limit", None) if offset < 1 or (limit is not None and limit < 1): raise ValueError("offset and limit must be 1 or more") if not ws.exists(path): raise FileNotFoundError( f"File not found: {path}. Files here: {', '.join(ws.listdir('.'))}") lines = ws.read_text(path).splitlines(keepends=True) wanted = lines[offset - 1:] if limit is None else lines[offset - 1:offset - 1 + limit] kept, shown = truncate_head("".join(wanted), max_lines, max_bytes) if offset > max(len(lines), 1): raise ValueError(f"offset {offset} is past the end of {path}: {len(lines)} lines") if wanted and shown == 0: skip = f" Use offset={offset + 1} to skip it." if offset < len(lines) else "" return (f"[Line {offset} of {path} is {len(wanted[0].encode())} bytes, over the " f"{max_bytes}-byte budget of one read, so none of it is shown.{skip}]") last = offset + shown - 1 if last < len(lines): kept += (f"[Showing lines {offset}-{last} of {len(lines)}. " f"Use offset={last + 1} to continue.]") return kept return { "name": "read", "description": "Read a text file and return its contents. One call returns at most " f"{max_lines} lines or {max_bytes} bytes; use offset to read on.", "parameters": { "type": "object", "properties": {"path": {"type": "string", "description": "Path of the file to read."}, "offset": {"type": "integer", "description": "First line, from 1."}, "limit": {"type": "integer", "description": "How many lines."}}, "required": ["path"], }, "execute": execute, } def make_write_tool(ws): """The `write` tool, over the workspace `ws`: create a file, or replace all of it.""" def execute(arguments): path, content = str_arg(arguments, "path"), str_arg(arguments, "content") ws.write_text(path, content) return f"Successfully wrote to {path}." return { "name": "write", "description": "Write a text file. Creates the file, or replaces its whole contents.", "parameters": { "type": "object", "properties": {"path": {"type": "string", "description": "Path of the file to write."}, "content": {"type": "string", "description": "The new contents."}}, "required": ["path", "content"], }, "execute": execute, } def apply_edits(content, edits): """`content` with every edit applied, or a ValueError naming the first one that cannot be. An edit is {"oldText": ..., "newText": ...} and stands for exactly one place in the file, so every oldText must match exactly, and match exactly once. Every edit is found in the ORIGINAL content and they are applied from the back, so applying one cannot move another. All of them or none: an edit that cannot be applied leaves nothing half-done.""" # --- your code --- # Each edit in `edits` is a dict with the string keys "oldText" and "newText"; the tool has # already checked that much. Raise ValueError with a message the *model* can act on, naming # the edit as edits[i], when: # - oldText is empty: it points at every position at once # - oldText is not in `content`: say that it must match exactly, whitespace and all # - oldText is there more than once: say how many times, and ask for more context # - two edits cover the same characters # - nothing would change # Two rules decide the rest. Find every edit in the content you were given, before any of # them is applied, or edit 2 gets to see edit 1's work. Then apply them back to front, or # the first replacement moves every position you worked out after it. # One of the hidden tests, on a file holding the line ` print("fetching", url)` twice: # apply_edits(app, [{"oldText": ' print("fetching", url)', "newText": " log(url)"}]) # -> ValueError("Found 2 occurrences of edits[0] in the file. ... more context ...") # --- end --- def make_edit_tool(ws): """The `edit` tool, over the workspace `ws`: change part of a file by naming the exact text to replace, so a one-line fix costs one line of output instead of the whole file. A call that cannot be applied in full is refused and the file is not touched.""" def execute(arguments): path = str_arg(arguments, "path") # --- your code --- # `arguments["edits"]` is whatever the model sent, straight from JSON: a list it promised # to send, or nothing, or a string, or a list of strings. Check it here, the way # `str_arg` checks a string, and say what was expected. Then read the file, put it # through apply_edits, write the result back, and tell the model what happened. # A path that is not in the workspace gets the same answer `read` gives it. # The file is written on exactly one path through this function: the one where every # edit applied. apply_edits raises for the rest, and run_tool turns that into an error # result the model can read. # --- end --- return { "name": "edit", "description": "Change a text file by replacing exact text. Each edit's oldText must " "appear exactly once in the file, so include enough surrounding lines to " "make it unique. Either every edit is applied or none is.", "parameters": { "type": "object", "properties": { "path": {"type": "string", "description": "Path of the file to edit."}, "edits": { "type": "array", "description": "The replacements to make, in any order.", "items": { "type": "object", "properties": { "oldText": {"type": "string", "description": "Exact text to replace, copied from the file."}, "newText": {"type": "string", "description": "What to put in its place."}}, "required": ["oldText", "newText"], }, }, }, "required": ["path", "edits"], }, "execute": execute, } def make_bash_tool(shell, max_lines=MAX_LINES, max_bytes=MAX_BYTES): """The `bash` tool: run one command line in `shell` and return what it printed. A command that fails has still been run: its exit code is news for the model, not a tool error, so it comes back as an ordinary result that ends with a status line.""" def execute(arguments): output, code = shell.run(str_arg(arguments, "command")) total = len(output.splitlines()) output, shown = truncate_tail(output, max_lines, max_bytes) if shown < total: output += f"\n[Showing the last {shown} of {total} lines. Narrow the command for more.]" if code != 0: output += f"\nCommand exited with code {code}" return output return { "name": "bash", "description": "Run a shell command and return its output (stdout and stderr together). " f"Only the last {max_lines} lines or {max_bytes} bytes come back.", "parameters": { "type": "object", "properties": {"command": {"type": "string", "description": "The command line."}}, "required": ["command"], }, "execute": execute, } # == 3 loop == # The code that talks to the model. The model keeps nothing between calls. def run_agent(model, system, messages, tools, prompt, *, max_turns=None): """Put `prompt` to the model and keep going until it stops asking for tools, a reply comes back failed (stop_reason "error"), or `max_turns` model calls have been made (None: no limit). Returns the final assistant message. `messages` is the caller's list: everything that happens is appended to it, so the caller holds the record even if the run dies. Every exit leaves a record that the next prompt can be sent on top of.""" turn = 0 messages.append(user_message(prompt)) while True: turn += 1 if max_turns is not None and turn > max_turns: # checked before the call, never after reply = error_message(f"Agent stopped after max_turns={max_turns}") else: reply = model.complete(system, context_for_model(messages), tool_specs(tools)) messages.append(reply) if reply["stop_reason"] == "error": # a failed reply: none of it is acted on return reply calls = tool_calls(reply) if not calls: return reply for call in calls: messages.append(run_tool(tools, call)) def context_for_model(messages): """What the model is sent: the record, minus assistant messages that failed and are empty. They are true, so the record keeps them; a provider refuses an empty message, so the view leaves them out. A failed reply that got some text out stays in. Returns a new list.""" return [m for m in messages if not (m["role"] == "assistant" and not m["content"] and m.get("stop_reason") == "error")]Running code needs JavaScript. Above is the file this exercise starts from (harness.py). You can run its tests on your own machine: python lab/lab_check.py --harness harness.py --lesson 6 tests.py

The 3,000-line app.py from the top of this page, with two identical print lines in it, and three changes to make. No tests: read what the model is told, and the two rows at the bottom. Once the lab has passed, this runs against your code.

import textwrap

import harness, lab

# The same 3,000-line app.py as at the top of the page, with two identical print lines in it.
LINES = [f"    step_{n:04d}()\n" for n in range(1, 3001)]
LINES[11] = "    TIMEOUT = 30\n"
LINES[499] = '    print("fetching", url)\n'
LINES[1499] = '    print("fetching", url)\n'
LINES[1500] = "    return fetch(url)\n"
APP = "".join(LINES)

JOB = ("Set TIMEOUT to 5, make the print inside retry() a log() call, and lower-case the url "
       "in the return below it.")
NARROW = [{"oldText": "TIMEOUT = 30", "newText": "TIMEOUT = 5"},
          {"oldText": '    print("fetching", url)', "newText": "    log(url)"},
          {"oldText": "    return fetch(url)", "newText": "    return fetch(url.lower())"}]
WIDE = [NARROW[0],
        {"oldText": '    step_1499()\n    print("fetching", url)',
         "newText": '    step_1499()\n    log(url)'},
        NARROW[2]]


def field(label, text):
    body = textwrap.wrap(text, 58) or [""]
    print(f"    {label:<20}{body[0]}")
    for line in body[1:]:
        print(f"    {'':<20}{line}")


def make_edits_step(ws, writes_when_refused):
    """Three changes in one call; if the refusal names an edit that matched more than once,
    that one edit is sent again with the line above it."""
    def step(request):
        result = request.last_result
        if result is None:
            return lab.reply(lab.call("read", {"path": "app.py"}))
        if result["tool_name"] == "read":
            return lab.reply(lab.call("edit", {"path": "app.py", "edits": NARROW}))
        if result["is_error"] and "occurrence" in result["content"].lower():
            writes_when_refused.append(len(ws.writes))     # the file, at the refused call
            return lab.reply(lab.call("edit", {"path": "app.py", "edits": WIDE}))
        return lab.say("Three changes in, one file touched once.")

    return step


def make_whole_file_step(ws, _writes):
    """The same three changes, written back as a whole file."""
    def step(request):
        result = request.last_result
        if result is None:
            return lab.reply(lab.call("read", {"path": "app.py"}))
        if result["tool_name"] == "read":
            shown = result["content"][:result["content"].index("[Showing")]
            for edit in WIDE:
                shown = shown.replace(edit["oldText"], edit["newText"])
            return lab.reply(lab.call("write", {"path": "app.py", "content": shown}))
        return lab.say("Three changes in.")

    return step


def run(make_step, tool):
    ws, seen = lab.Workspace({"app.py": APP}), []
    model, messages = lab.ScriptedModel([lab.forever(make_step(ws, seen))]), []
    harness.run_agent(model, harness.SYSTEM, messages, [harness.make_read_tool(ws), tool(ws)], JOB)
    output = sum(m["usage"]["output"] for m in messages if "usage" in m)
    return ws, model, messages, output, seen


ws, model, messages, output, seen = run(make_edits_step, harness.make_edit_tool)
results = [m for m in messages if m["role"] == "toolResult"]
lines = ws.read_text("app.py").splitlines()

print("the model read app.py:")
field("it was shown", results[0]["content"].splitlines()[-1])
print("call 1: three edits, in one call")
field("the model is told", results[1]["content"])
field("the workspace", f"written {seen[0]} time(s): one of the three edits could not be "
                       "applied, so none of them was")
print("call 2: the same three, with the middle one given the line above it")
field("the model is told", results[2]["content"])
field("app.py", f"{len(lines):,} lines, written {len(ws.writes)} time(s) in the whole run")
for number in (12, 500, 1500, 1501):
    print(f"        line {number:>4}      {lines[number - 1].strip()}")
print("    (line 500 was never mentioned, so nothing happened to it)")
print()

write_ws, write_model, _, write_output, _ = run(make_whole_file_step, harness.make_write_tool)
print("the same three changes, through the two tools you now have:")
field("with write", f"{len(write_model.calls)} model calls, {write_output:,} output tokens, "
                    f"and app.py came back {len(write_ws.read_text('app.py').splitlines()):,} "
                    "lines long")
field("with edit", f"{len(model.calls)} model calls, {output:,} output tokens, and app.py is "
                   f"still {len(lines):,} lines long")

the model read app.py:
    it was shown        [Showing lines 1-2000 of 3000. Use offset=2001 to
                        continue.]
call 1: three edits, in one call
    the model is told   Found 2 occurrences of edits[1] in the file. Each oldText
                        must be unique: provide more context to make it unique.
    the workspace       written 0 time(s): one of the three edits could not be
                        applied, so none of them was
call 2: the same three, with the middle one given the line above it
    the model is told   Successfully edited app.py (3 edit(s) applied).
    app.py              3,000 lines, written 1 time(s) in the whole run
        line   12      TIMEOUT = 5
        line  500      print("fetching", url)
        line 1500      log(url)
        line 1501      return fetch(url.lower())
    (line 500 was never mentioned, so nothing happened to it)

the same three changes, through the two tools you now have:
    with write          3 model calls, 8,564 output tokens, and app.py came back
                        2,000 lines long
    with edit           4 model calls, 200 output tokens, and app.py is still
                        3,000 lines long

Four model calls instead of three, because one call was refused — and 200 output tokens instead of 8,564, with the file still 3,000 lines long. The refused call is the part to look at: three edits went out, one of them was ambiguous, and the workspace was written zero times. The model then re-sent all three with one of them widened, and the file was written once.

What your harness can do now, if you keep this page's code:

Hold a conversation: it knows what was said, and who said it.
Run a tool the model asks for and show it the result.
Keep going until the model stops asking.
Tell the model when a tool fails, and carry on.
Stop a runaway, and survive a provider failure.
Keep every tool result within a budget.
Change part of a file by naming the exact text to replace, and refuse the whole call rather than leave a file half-edited.

Nothing later on the path — Checkpoint A, then lessons 7 to 16 — assumes those last two functions; every one of those pages is written against the file you had at the end of lesson 6. Keeping them costs you nothing, and if you start a later lesson from its own starter they simply will not be there.

Say it in your own words

Uniqueness is the rule your model will trip over most: it costs a turn every time a line is not as distinctive as it looked. In a sentence or two, what does that rule buy, and who is it for?

The model sent you a piece of text. Ask what the tool would have to do if that text matched three places, and who would find out about the choice it made.

An edit is a way of pointing at one place in a file by quoting it. If the quote matches twice, the tool has to guess which one was meant, and nobody — not the model, not the person whose file it is — will find out which way it guessed until something is broken later. Refusing costs one turn and a sentence; guessing costs a wrong line in a file that nobody is looking at.

Common answers, and what each one misses

"It keeps the implementation simple: you can use str.replace and be done." The rule is not there for the implementation. The cell with the five calls in it had the simplest implementation there is, and three of its five answers left the model believing something that was not so.
"It protects against the model hallucinating." It does not: a model can quote the file perfectly and still be wrong about what it wants changed. The rule is about ambiguity, which is a property of the file, and it fires just as often on text the model copied correctly.
"It is for the user, so they can trust the agent." It is for the model first, in that turn, because the model is the only party that can do anything about the answer. The user gets the benefit at the end of a job that came out right.

An edit names one place in a file the model cannot see. It must match exactly, match once, and be found in the file the model read.

Tau validates every edit against the content it was handed, refuses the whole call if any of them fails, and replaces from the back — the same three decisions, in the same order.

spans = []
for index, edit in enumerate(edits):
    ... the three checks, each naming edits[index] ...
    start = content.index(old)
    spans.append((start, start + len(old), edit["newText"]))
spans.sort()
... refuse a pair whose spans overlap ...
for start, end, new in reversed(spans):
    result = result[:start] + new + result[end:]

    matches: list[tuple[int, int, str]] = []
    for index, edit in enumerate(normalized_edits):
        old_text = edit["oldText"]
        occurrences = _count_occurrences(normalized_content, old_text)
        if occurrences == 0:
            raise ToolInputError(_not_found_error(path, index, len(normalized_edits)))
        if occurrences > 1:
            raise ToolInputError(_duplicate_error(path, index, len(normalized_edits), occurrences))
        start = normalized_content.index(old_text)
        matches.append((start, start + len(old_text), edit["newText"]))
# ...
    _validate_non_overlapping(matches)
    new_content = normalized_content
    for start, end, new_text in sorted(matches, reverse=True):
        new_content = f"{new_content[:start]}{new_text}{new_content[end:]}"
    if new_content == normalized_content:
        raise ToolInputError(_no_change_error(path, len(normalized_edits)))

The same decisions, one by one.

Every edit is counted and located in the content that came in, before anything is replaced, and the first one that fails raises for the whole call. The empty oldText is refused in a pass of its own, above the one shown.
Overlap is decided on the sorted spans, with the same comparison yours makes, so edits that touch are legal there too (src/tau_coding/tools.py:1105-1111).
The replacements run over the spans in reverse, for the reason your cell showed.
A call that produces identical content is an error, not a quiet success (src/tau_coding/tools.py:1158-1165).
The tool checks the shape of the model's arguments where you check it, and names the entry: edits must be a non-empty list (src/tau_coding/tools.py:1084-1090), and each item an object with two strings (src/tau_coding/tools.py:1092-1100).
The model is told the rule before it calls, in the description and again in the guidelines that go into the system prompt: each oldText must match a unique region of the original file, and nearby changes should be merged into one edit rather than sent as overlapping ones (src/tau_coding/tools.py:538-543, src/tau_coding/tools.py:552-554).
Tau's tests are the two you would write first: a call whose second edit cannot be found leaves the file exactly as it was (tests/test_coding_tools.py:321-340), and two identical lines are refused with the count in the message (tests/test_coding_tools.py:343-356).

Where the wording differs. Tau writes every refusal twice: once for a call carrying a single edit ("Could not find the exact text in <path>") and once for a call carrying several ("Could not find edits[1] in <path>"), and both carry the file's name because the function is given the path for that purpose (src/tau_coding/tools.py:1128-1137). Yours keeps only the indexed form and apply_edits never sees a path, so the tool names the file and the refusals name the edit. It is half the code, the model is always told which edit failed, and a test can ask for edits[1] without a special case.

What Tau adds. Four things, all of them the real world arriving.

Line endings and byte-order marks. The file is stripped of a leading BOM and normalised to \n before anything is matched, then put back the way it was found (src/tau_coding/tools.py:915-920, src/tau_coding/tools.py:1124-1125). Without it, a model that quotes a Windows file back at you with plain newlines matches nothing, and every edit is refused for a reason that is invisible on screen. Your Workspace keeps the bytes it was given and no file in this course has a CRLF line in it, so nothing here turns on it.
Argument shapes the model got wrong but meant. Before the shape is checked, edits that arrived as a JSON string is parsed, and a top-level oldText/newText pair — the older, single-edit form of this tool — is folded into the list (src/tau_coding/tools.py:1062-1081). [general] Tool arguments are generated text, and a model that has seen a hundred variants of this tool will sometimes send you one of the others.
A lock per file. Reading, applying and writing happen inside an asyncio.Lock held for that path, so two edit calls to one file cannot interleave (src/tau_coding/tools.py:512-521, src/tau_coding/tools.py:1202-1214). Yours cannot have that problem yet, because your tools run one at a time inside a for loop; lesson 15 is where that stops being obviously true.
A diff for the screen. The result the model gets is one sentence; the diff, a patch and the first changed line go into details, which is never sent (src/tau_coding/tools.py:523-534). That is lesson 6's argument about the two audiences, in the tool that most obviously has both.

Where yours is weaker. Your edit can only be as good as the read in front of it: the model quotes back text it was shown, and it was shown the first 2,000 lines. Ask it to change something on line 2,500 and every edit will be refused as not found until it pages down, which it will only do if it reads your notice. [general] Neither of you checks that the file on disk is still the file the model read; between the read and the edit, another process, another agent or a person with an editor may have changed it, and an oldText that still matches will be applied to a file nobody looked at. And nothing here knows what the text means: an edit that matches exactly once and produces code that does not parse is applied, cheerfully, and the next bash call is where you find out.

src/tau_coding/tools.py:923-953 · pinned to commit 9fe6a71 · view on GitHub

One more case

The model asks you to rename a helper that is called on forty lines of one file, and sends forty edits, one per call site. Your tool refuses the call, and would refuse every one of those edits for the same reason: the text each of them names is in the file forty times. The model has nowhere to go. In a sentence or two: what do you add, and what does your answer cost?

Your rule was never "the text must be unique in the world". It was "each edit names one place". Which of the fixes you can think of keeps that promise, and which one quietly hands the choice back to the tool?

Three answers, and they are not equally good. Widen each edit until it is unique — the model's own move from the last cell — which keeps the rule and costs forty edits of two or three lines each. Add a replaceAll flag to an edit, which is honest about giving up uniqueness for that one edit and should say in the result how many places it changed, so the count arrives somewhere the model can see it. Or give the agent a different tool for this shape of job, which is what bash and a careful command already are. What you must not do is make the plain edit quietly change the first match: that is the five-call cell near the top of this page, and it is a wrong line in somebody's file with a success message on top.

Common answers, and what each one misses

"Change all forty: that is obviously what it meant." Probably, and "probably" is doing the work. One of the forty may be in a comment, a docstring or a string literal that a rename should not touch, and the tool cannot tell which.
"Let the model send forty edits in one call." That is the answer, once each one is unique — and it is why the all-or-nothing rule matters at this size. Forty edits, one refusal, nothing written, one message naming edits[i]: the model fixes that one and sends the lot again.
"Make it a regular expression." Then the argument is a program, the model has to get it exactly right with no way to test it, and a mistake matches text nobody has read. Exact text is chosen precisely because it is the thing the model can copy rather than compose.

You hit

a one-line fix that cost 8,525 output tokens and quietly cut a 3,000-line file down to the 2,000 lines the model had been shown

You built

apply_edits(content, edits) and make_edit_tool(ws): exact text, exactly one match, every edit measured against the content the model read, applied from the back, all of them or none

The principle

an edit names one place in a file the model cannot see, so it must match exactly, match once, and be found in the file the model read

Your harness now

lesson 6's file, plus two functions in region 2.

error_message
str_arg
int_arg
tool_specs
run_tool
truncate_head
truncate_tail
make_read_tool
make_write_tool
apply_edits
make_edit_tool
make_bash_tool
run_agent
context_for_model

Your answers

Back on the path

Nothing after this page needs it. Next is Checkpoint A: six runs from lessons 1 to 6, broken in six different ways, with nothing on the page saying which lesson each one comes from.

Previous 06 The firehose Next Checkpoint A Transcript doctor Six runs went wrong, in six different ways. Which idea fixes each one?