LangChain's State of Agent Engineering report, published June 12, 2026, surveyed 1,340 people building AI agents between November 18 and December 2, 2025, and the headline number is a good one: 57.3% now have an agent running in production. Read past the headline and the report's own numbers cut the other way. Quality, not cost, not latency, is the single most common reason a project stalls before reaching production, cited by about a third of respondents. At the same time, 29.5% of everyone surveyed runs no evaluation on what their agent actually produces. Those two findings sit in the same report, a few pages apart, describing what looks like the same population from two different angles.
What the report actually measured
The 1,340 responses skew toward the people who'd know: engineers, product managers, and technical leaders, 63% from technology companies, another 16% from financial services and healthcare combined. Company size ran the full range, 49% under 100 people, down to 9% at 10,000 or more. Production adoption tracked size closely: 50% of teams at organizations under 100 people had an agent in production, rising to 67% at organizations of 10,000 or more. That's not a downloadable dataset the way a Census or BLS release is, it's the publisher's own report page rather than a raw file, but it's LangChain's own survey of its own practitioner base, not a vendor blog summarizing someone else's numbers.
Everyone traces what the agent does. Far fewer grade it.
The report draws a clean line between two different disciplines that get talked about as though they're the same thing. Observability, logging what an agent did and in what order, is close to universal: 89% of respondents have implemented some form of it, rising to 94% among teams with agents already in production, and 71.5% of those production teams have full step-by-step tracing. Evaluation, scoring whether what the agent produced was actually correct or useful, lags well behind. Offline evaluation, testing against a fixed set of examples, sits at 52.4%. Online evaluation, scoring real production traffic as it happens, sits at 37.3% overall and 44.8% among teams already in production. Tracing tells you what the agent did. It doesn't tell you whether what it did was right. LangChain's own framing of the gap: "observability is more widely adopted, agent evals are still catching up."
Quality is the top blocker, and it's also the thing least measured
Ask the same 1,340 people what's stopping their agent from reaching production, and about a third name quality, meaning accuracy, relevance, consistency, and tone, ahead of latency at 20% and, among larger enterprises with 2,000 or more employees, security at 24.9%. The report doesn't publish a cross-tab of who named quality and who also runs no evals, so that specific overlap isn't something the data proves directly. But read the two findings side by side and the shape is hard to miss: the most commonly cited reason an agent doesn't ship is a property nearly a third of the same survey population has no mechanism to measure at all. A team that can't put a number on "is this good" is not well positioned to argue it's good enough to launch, and it's not well positioned to notice when a prompt change or model upgrade makes it worse.
What this costs a team that already shipped
This isn't only a pre-launch problem. It shows up after an agent is live, when a beta customer reports the assistant gave a wrong answer, and an engineer opens the trace, sees exactly what the agent called and in what order, and still can't say whether that failure was a one-off or the start of a drift that started three deploys ago. Tracing answers "what happened." It doesn't answer "is this happening more than it used to." Without a baseline evaluation set run against every prompt or model change, a team is choosing between shipping updates blind and freezing the agent in place out of caution, and neither of those is a plan a VP of Engineering wants to defend in a postmortem. Our look at Datadog's own production telemetry found a similar pattern from the infrastructure side: most reported AI failures traced back to capacity and rate limits, not the model being wrong, which makes evaluation the piece that actually answers the question infrastructure monitoring can't.
A worked example
Picture a 30-person, Series B logistics-software company that shipped an internal agent nine months ago to triage incoming carrier disputes, routing each one to the right queue and drafting a first response. It works well enough in the demo that leadership approved expanding its scope to draft the actual resolution, not just the routing. Two weeks after the expanded rollout, a support lead flags that resolutions on a specific dispute type read as confident but are wrong about the contract terms roughly one time in six. Nobody can say whether that rate is new, because the team has full tracing on every agent call and no evaluation set that scores resolution accuracy against a known-correct answer. The fix that actually stops the bleeding isn't a better prompt tried at random, it's building the evaluation set first: a few hundred labeled examples of correct and incorrect resolutions, run automatically against every change, so "one time in six" becomes a number the team tracks instead of a complaint it reacts to.
Where this fits, and where it doesn't
An evaluation harness earns its cost on an agent whose output is subjective, variable, or high-stakes enough that a wrong answer costs more than the harness itself, customer-facing drafting, contract or compliance judgment calls, anything a support team would otherwise QA by hand. It's the wrong first investment for an agent doing narrow, deterministic work with a checkable right answer, extracting a field from a structured document, say, where a simple pass-fail validation check catches most of what a full evaluation harness would, at a fraction of the build. The honest sequencing is observability first because you need to see what's happening, evaluation second once the agent's job is subjective enough that "happened" and "was correct" can diverge, and only then a wider rollout.
If your team already has an agent in production and the trace logs to prove it, but no number for whether last week's change made it better or worse, that gap is exactly what an evaluation harness closes, and it's a smaller, faster build than most teams expect once someone who's built one before scopes it. It pairs with the broader pattern in why AI agents fail in production and with how long an agent can be trusted to run unsupervised, covered in our look at METR's own time-horizon data. Our Silicon Valley team scopes this kind of production-engineering work under build your team; talk to us about what an eval harness would need to cover for the specific agent you're trying to expand.
Sources
- LangChain: State of Agent Engineering (published June 12, 2026; survey fielded November 18 to December 2, 2025, 1,340 respondents)