The model is not the product. The harness is.
Why most of the real engineering in an AI system lives in the code around the model, and what that code has to do.
·2 min read·harness engineering, agents
Every AI demo looks the same from the outside: a text box, a model call, a nice answer. And every team that tries to put that demo in front of real users discovers the same thing. The model call is about five percent of the work.
The other ninety-five percent is what I call the harness: everything wrapped around the model that turns a clever autocomplete into a system you can trust.
What a harness actually does
A harness is not one thing. It is a set of responsibilities that someone has to own:
- Context assembly. Deciding what the model sees: instructions, retrieved documents, conversation state, tool results. Too little and it guesses. Too much and it gets confused and expensive.
- Control flow. Loops, retries, timeouts, and the decision of when to stop. An agent that never stops is a bug that bills you.
- Tool routing. Which tools exist, what arguments are allowed, what happens when a tool fails or returns garbage.
- State and memory. What carries over between steps and sessions, and what must be forgotten.
- Guardrails. Permissions, input and output validation, and the human approval step before anything irreversible.
- Observability. Traces of every step, so that when something goes wrong you can see why instead of re-running and hoping.
- Evaluation. A way to know, with numbers, whether today’s change made things better or worse.
None of these are model problems. They are software engineering problems, which is good news, because software engineers already know how to solve them.
Why this matters for your career
The industry spent two years treating “AI engineer” as “person who writes prompts”. That framing is already dead. The engineers getting hired, and the teams shipping AI that survives contact with users, are the ones who treat the model as an unreliable dependency and engineer around it.
If you have a few years of backend or full-stack experience, you are closer to this than you think. Retries, idempotency, validation, observability and testing are skills you already have. The harness is where they pay off again.
A simple test
Take any AI feature you’ve built and ask three questions:
- If the model returns something malformed, what happens?
- If I change the prompt tomorrow, how do I know I didn’t break something?
- If a user tries to make it do something it shouldn’t, what stops it?
If the honest answer to any of these is “nothing” or “I’d notice”, you have a demo, not a product. The good news is that each one has a well-understood engineering answer, and that is exactly what I’ll be writing about here.