The AI-Assisted Engineer
What changes when producing an implementation gets cheaper
AI can make it inexpensive to produce a candidate implementation. Whether that reduces the cost of a finished change depends on what happens around it: understanding the repository, specifying behaviour, reviewing the result, testing it and operating it afterwards.
The distinction matters. Producing more code is not the same outcome as delivering more useful software. When generation becomes faster, the unresolved work can move into integration, review or verification. In some tasks, those costs consume the apparent saving.
The useful question is therefore where AI changes the work in front of you, and what evidence would show an improvement.
The productivity claim needs a setting
Research does not support a single productivity multiplier for software development. A study across three companies and 4,867 developers reported a 26.08% increase in completed tasks with an AI coding assistant. METR's early-2025 study of experienced developers working on familiar open-source repositories found tasks took 19% longer when AI was allowed.[1][2]
These studies used different tools, populations and measures. They are not directly interchangeable estimates. METR's February 2026 follow-up suggested improvement with newer tools but warned that selection effects and difficulties measuring concurrent work made the size of the effect unreliable.[3]
The practical conclusion is to measure complete work in the relevant setting. A fast first draft can be valuable. It is not, on its own, evidence of a faster accepted change.
Specification gives the work a boundary
A useful specification identifies the behaviour that matters: inputs and outputs, invariants, failure handling, interfaces, security boundaries and acceptance examples. It should also say what the change does not need to solve.
The amount of detail should match the uncertainty and consequences. A wording change needs little ceremony. A packet parser or a change to privileged operations needs much more precise constraints. For an exploratory prototype, the specification may primarily identify the question the prototype is intended to answer.
A model can help discover ambiguities and propose alternatives. It can challenge a requirement. What it cannot supply by itself is authority to decide which business consequence is acceptable. Somebody still has to own that decision and check the assumptions behind it.
One useful question before generation is: what would make this implementation unacceptable even if the demonstration looked good? The answer often exposes a contract that the original request omitted.
Plausible explanations need checking too
NetWatch's September review recorded traffic displayed under pid:0 and proposed that the sandbox might be preventing process attribution. That was a plausible explanation worth investigating. The later 0.30.0 changelog records a different finding: missing attribution had been formatted as PID zero. The displayed identity was misleading.[4]
The distinction is small in code and substantial in diagnosis. A missing value and a known process identifier carry different meanings. Treating the label as evidence about the operating system sends the investigation in the wrong direction.
This is a documented correction, not evidence that AI introduced the defect. It illustrates the standard an AI-assisted investigation needs to meet: distinguish the observation from the proposed explanation, identify a check that can separate competing causes, and revise the explanation when the result disagrees.
The same applies to a model's confident description of unfamiliar code. A coherent explanation is a hypothesis until the relevant paths and behaviour support it.
Three kinds of convincing mistake
Candidate implementations can fail in several ways while still looking reasonable.
The wrong problem. A formatter turns an unknown process into a plausible identifier, satisfying a request to fill an empty column while changing the meaning of the data.
The wrong contract. A function is internally consistent but violates an assumption elsewhere: time units, event ordering, cancellation, error propagation or the meaning of an absent value.
The wrong context. A local solution duplicates an existing capability, bypasses the project's preferred abstraction or introduces a dependency whose operating cost was deliberately avoided.
None of these is unique to generated code. Review can catch all three when the reviewer has the relevant context. The concern is that a large volume of plausible output can exceed the attention available to examine it.
NetWatch's 0.30.1 changelog gives another concrete example. Histories from multiple interfaces had been aligned at their oldest entries even though they all ended at the present. A newer interface could therefore appear to have carried traffic before it existed.[5] A graph could render successfully while contradicting the measurement it claimed to show.
For that case, the important check is temporal alignment against a small known history. Rendering an attractive chart does not establish the meaning of its horizontal axis.
A loop with independent checks
Specify. State the intended behaviour and constraints. Include at least one example that distinguishes the desired result from a plausible wrong one.
Generate. Produce a change small enough to understand. Ask for the assumptions and affected interfaces, then verify those against the repository. A persuasive explanation does not remove that step.
Review and verify. Inspect the design and diff, run the checks appropriate to the change, and exercise the user-visible path. Derive expected results from the requirement, an independent reference or a deliberately constructed example. If the same mistaken interpretation supplies both implementation and expected output, passing tests may only demonstrate agreement.
Measure. For a performance claim, record the workload, environment and comparison. For a productivity claim, include review, rework and integration effort alongside generation time. Compare similar classes of task and keep quality visible.
Refine. Feed the findings back into the specification and durable checks. If the failure revealed an ambiguous contract, resolve that contract so the next change does not inherit the same ambiguity.
These checks overlap, which is useful. NIST's Secure Software Development Framework treats source review or analysis and executable testing as distinct practices.[6] Neither compilation nor a successful run establishes every property of a program. Choose evidence proportionate to the consequence of being wrong.
Architecture makes review manageable
Clear interfaces, narrow responsibilities and replaceable dependencies make it easier to bound a change. They also make it easier to state what a generated implementation is allowed to affect.
A collector, for example, should be testable with controlled observations rather than requiring a live network for every check. A reporting component should not silently gain permission to modify the host. Those boundaries let a reviewer concentrate on the change while retaining confidence about the surrounding system.
DORA's 2025 research presents AI as amplifying existing organisational strengths and weaknesses.[7] The engineering implication is to invest in the delivery system around generation. Adding coding capacity to a queue that is already constrained by review may simply lengthen the queue.
That does not mean every project needs an elaborate platform before using AI. Start with the constraints and checks the next meaningful change requires. Expand them when the work exposes a recurring gap.
Use cheaper experiments carefully
AI can help explore an interface, compare implementation approaches or explain an unfamiliar module. The useful output of an experiment may be a decision to discard the implementation.
NetWatch's design bundle makes that separation concrete: it contains review findings, HTML mockups and rendered views, while its handover states that the displayed scenario uses synthetic data.[8] Those artefacts can make a proposed interaction inspectable before implementation. They do not establish that the runtime behaviour works, and the bundle does not provide a measured AI productivity result.
Keep the question for an experiment explicit. Are you checking whether a screen is understandable, whether an algorithm meets a bound, or whether an integration is possible? A prototype that answers one of those questions should not acquire production status by accident.
Keep the maintenance cost in view
An implementation that becomes cheap to generate still needs to be understood, changed and operated. Before adding another abstraction, ask which recurring difficulty it removes and what it adds to the next diagnosis.
A single caller is not automatically evidence of a bad abstraction; it may isolate a volatile dependency or a security boundary. Equally, a sophisticated design is not justified merely because it was easy to generate. Judge it by the problem it resolves.
The engineer's responsibility is to turn available tools into a result that holds up under scrutiny. That includes choosing the work, defining the constraints, testing the explanation and accepting the consequences. AI can help throughout that process. The benefit is established when useful changes arrive with acceptable total effort and quality, not when the code arrives quickly.
Sources
Zheyuan Cui, Mert Demirer, Sonia Jaffe, Leon Musolff, Sida Peng and Tobias Salz, “The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers”, June 2025. Reported pooled completed-task increase: 26.08%, standard error 10.3%. ↩︎
Joel Becker, Nate Rush, Beth Barnes and David Rein, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity”, 10 July 2025. Sixteen developers, 246 tasks; a result for that setting and period. ↩︎
Joel Becker, Nate Rush, Tom Cunningham, David Rein and Khalid Mahamud, “We are Changing our Developer Productivity Experiment Design”, 24 February 2026. ↩︎
NetWatch,
review/REVIEW.mdandHANDOVER.md, 3 September 2026;CHANGELOG.md, release 0.30.0, 5 September 2026. See NetWatch evidence: attribution diagnosis. ↩︎NetWatch,
CHANGELOG.md, release 0.30.1, 7 September 2026. See NetWatch evidence: graph semantics. ↩︎Murugiah Souppaya, Karen Scarfone and Donna Dodson, Secure Software Development Framework (SSDF), Version 1.1, NIST SP 800-218, February 2022, practices PW.7 and PW.8. Cited as established development guidance. ↩︎
DORA, State of AI-assisted Software Development, 2025, report overview. Organisational evidence, not a controlled estimate for a specific tool or task. ↩︎
NetWatch,
netwatch-next/HANDOVER.md, 3 September 2026, supplied design bundle. See NetWatch evidence: design handover. ↩︎