When a coding agent changes a file, the language model is only one part of the event. A surrounding program supplies tools, executes requests, returns results and carries the conversation forward. That program is often called the harness.
Pi provides a concrete way to inspect that relationship. This companion uses a fictional failing test to show where observations, edits and verification belong. It also distinguishes what Pi’s documentation, accessed on October 17, 2026, promises from safety features a reader might assume every agent includes.
Separate a proposed action from its execution
Pi’s project overview describes a minimal, extensible agent harness. The important architectural point is that a model’s text is not itself a filesystem operation. Tools implemented by the surrounding software perform the work.
In Pi’s agent-loop implementation, assistant responses are examined for tool calls. Executed tool results are appended to the working context before the loop continues. This is source-code evidence for an observation-and-action cycle, not a benchmark proving that the cycle will solve a particular programming task.
Imagine a fictional test that expects a login error to say “Email is required,” while the application returns “Missing email.” The agent needs more than those two strings. It must establish which behavior is intended, locate the relevant code and check whether another part of the interface depends on the wording.
Follow the evidence through one repair
A useful record of that fictional repair has five stages:
- Locate: identify the test that reports the wording mismatch.
- Read: inspect the relevant implementation and establish which wording the requirement calls for.
- Change: edit the behavior or test for a stated reason, rather than simply making the strings match.
- Check: run the appropriate verification after the edit and retain its result.
- Review: compare the actual diff and result with the intended behavior.
The agent may loop back when a check reveals new information. Each conclusion should still point to an observation.
Suppose the first search returns three files. A final response saying “fixed the login bug” skips the evidence needed to evaluate the work. The useful questions are which file was changed, why that behavior was correct and what command demonstrated it afterward.
A passing test also needs interpretation. Changing the expected string to match broken behavior could make the test pass without satisfying the requirement. For this invented repair, decide the intended behavior first. Then compare the diff and the test result against that decision.
Use the interface as a record
Pi’s terminal usage guide describes a transcript of prompts, tool calls, results and errors. Tool output can be expanded, and sessions can be continued or explored through session commands.
For the fictional repair, use that visible history to answer a short chain of questions. Did the tool run the command in the intended project? Did the output refer to the intended test? Was the successful run before or after the edit? A plausible final explanation cannot substitute for those details.
Keep the scope proportionate. A one-line behavior change should have an understandable reason for every additional file it touches. A large diff is not automatically wrong, but unexplained extra changes make review harder. Treat the diff as the record of what the tool actually changed, rather than a picture of what the model intended to change.
Do not assume Pi asks before every command
This is where a small harness can expose an important limit. Pi’s security guidance explicitly says that it does not request approval before every tool call. It can read, change and execute files with the permissions of the account running it.
The same guidance distinguishes watching the transcript from a security boundary. Operating-system or virtualization restrictions can limit access; a visible conversation alone does not. Project trust affects which resources load, but does not make all resulting actions safe.
For the hypothetical login task, write down what the process can actually reach before running unfamiliar work: the project files, other mounted folders and available network access. That is a question about the execution environment. A model’s promise to stay within scope is a different kind of evidence. The agent security guide examines how permitted actions and untrusted material can interact.
This article is not a setup guide and does not ask you to install or run Pi. The practical reading habit is to locate the documented boundary instead of inferring one from an approval button shown in another product.
A saved session and current context differ
Pi’s compaction reference describes summarizing older messages and rebuilding the model’s working context from that summary and retained recent messages. Branch summaries support navigation between parts of a session.
A summary necessarily changes the representation of earlier work. For the fictional repair, a useful continuation note would preserve the intended login behavior, the file changed, the exact check result and the remaining uncertainty. “Login fixed” loses too much information to guide a later decision.
If the agent repeats an abandoned approach after a long session, inspect what context it has retained. The complete saved history and the information included in the next model request are not interchangeable. Diagnose the missing fact before assuming a different model would solve the problem.
Skills supply instructions, with their own limits
Pi’s skills documentation describes advertising available skills by name and description, then reading their full instructions when relevant. It also acknowledges that a model can miss an applicable skill.
For the test example, project-specific guidance might explain the correct command or the convention for error messages. Confirm that the relevant instructions were loaded and are current. A skill containing executable scripts can affect the environment, so its contents belong in the trust assessment too.
The result is a useful way to evaluate a coding agent without relying on the demo’s fluency. Follow the requirement into an observation, the observation into a proposed action, the action into a diff and the diff into a meaningful check. Then ask what permissions and remembered context surrounded that sequence. Those are the parts of the harness that make a repair understandable and a failure diagnosable.
