Papaya now supports Claude Managed Agents! If your agents run on Anthropic’s managed platform, or soon will, you can send every session to Papaya and get a ranked list of what to fix for quality, latency, and cost. Setup takes a few minutes in the Claude Console and needs no code changes.
The most common question we get from teams making the move is a fair one: if Anthropic runs the agent for us, do we still have our traces, and is there still anything to improve? The answer to both is yes. This article explains what Claude Managed Agents give you out of the box, what Papaya adds on top, and how to connect the two.
What Claude Managed Agents are
Claude Managed Agents is Anthropic’s hosted way to run agents. You define the agent once: its model, system prompt, tools, skills, and MCP servers. Anthropic stores that definition with versions, runs the agent loop for you, and gives each session its own sandbox where the agent’s tools run, such as bash, file edits, and code execution. You start sessions and stream their events, but you no longer write or host the loop yourself. For many teams that removes a lot of infrastructure.
What Anthropic gives you, and where Papaya comes in
Anthropic gives you good ways to see what your agents did. You can open any session in the Claude Console and read it step by step. And with trace export, every session can go over OpenTelemetry to a destination you choose, with the system prompt, user messages, the agent’s replies, and every tool call and result.
So you have your traces. The hard part is reading them. Once your agent handles thousands of sessions a week, nobody on the team can go through them all, and the questions that matter are about all of them together: what are people actually using the agent for, which of those jobs keep failing, why, and what should we change first? That is the part Papaya does.
What Papaya does with your sessions
Papaya is an agent improvement platform. You send it your agent’s traces, and it works out how the agent behaves, where it fails, and what to change, without you configuring anything. Here is what that looks like once your Managed Agents sessions arrive.
Your sessions become business use cases
A production agent does many jobs at once, and each job succeeds or fails for its own reasons. So the first thing Papaya does is sort your sessions into the business use cases they served. You see how much traffic each use case takes, how well each one works, and which ones need attention. You do not tag anything. Papaya reads the conversations and works it out.
This is the part you will not find anywhere else. Generic dashboards tell you an error rate. Papaya tells you that cancellations are failing while bookings are fine, which is something you can act on.
Improvements, ranked by impact
Within each use case, Papaya compares the sessions that went well with the ones that did not, using 200+ research-backed analyses. What it finds becomes a list of improvements: a tool that keeps failing and gets retried, a step where the agent skips checking its own work, a system prompt rule the agent keeps misreading, a large model doing an easy step, context that grows until the agent loses track. Each improvement comes with a priority, its expected impact on quality, latency, and cost, the change to make, and the sessions that show the problem.
Papaya also finds user signals: things people try to do with your agent that it cannot do today, how often they come up, and how those conversations ended. These are not bugs in the agent. They are decisions for your product team about what to build next.
A trace viewer people can read
When you want to see a session for yourself, Papaya’s summary view shows the run as a goal and a few plain-language steps, with the step where it went wrong marked. The detail view shows the full session underneath: every message, and every tool call with its request and response. A product manager can read it as easily as the engineer who built the agent.
Fixes come to you
Papaya posts new high-impact improvements to the Slack channels you choose, with the problem, the fix, and the evidence. If you connect the repository where your agent’s definition lives, Papaya can open a pull request with the change, so your part is reviewing and approving it.
Why this fits managed agents especially well
When Anthropic runs the loop, the code you own gets smaller. What is left is the agent’s definition: the system prompt, the tools and their descriptions, the skills and MCP servers it can use, and the model and effort you choose. Those are exactly the things Papaya’s improvements change. A recommendation like “this tool description makes the agent call it with the wrong ID” or “this use case does not need the largest model” maps straight onto a field in your agent definition.
It also matters for cost and speed. Managed sessions can run long, with many tool calls in one sandbox. Papaya shows where that time and those tokens go, use case by use case, and which changes would cut them without hurting quality.
Reading sessions yourself vs. Papaya
| Question | Reading sessions yourself | With Papaya |
|---|---|---|
| What happened in this session? | Open it and read it step by step. | A summary view of the run, with the failing step marked, plus the full detail. |
| What do people use the agent for? | Read enough sessions to work it out. | Every session sorted into business use cases, with traffic and quality for each. |
| Which failures keep happening? | Spot them in the sessions you happen to read. | Repeated failures grouped, with how often each happens and the sessions behind it. |
| What should we change first? | Weigh the problems you found by hand. | Improvements to the prompt, tools and model choices, ranked by impact on quality, latency, and cost. |
| What are users missing? | Notice the requests the agent turned down. | User signals: requests the agent cannot handle today, with how those conversations ended. |
| How do we hear about it? | Someone has to go and look. | Slack alerts for new improvements, and pull requests when your repository is connected. |
Setup: a few minutes, no code changes
Connecting is two short steps, one in Papaya and one in the Claude Console:
- Create a Papaya SDK key. In the Papaya app, open SDK Keys, create a key for your agent, and set Capture to Full.
- Add Papaya as a telemetry destination. In the Claude Console, open Settings → OpenTelemetry, add a Custom destination with Papaya’s endpoint and your key, choose Full content, and pick the workspaces to send.
Click Test, run one short session, and you are connected. From then on every new session reaches Papaya as it happens. If you already have sessions from before, we can pull those through the Claude API too. The full steps, with the endpoint and header to copy, are in the Claude Managed Agents setup guide.
Try it on your own sessions
Papaya’s free plan covers one agent and 1,000 traces a month, enough to see what it finds in a real workflow. Follow the setup guide, or book a demo and we will set it up with you.
Frequently asked questions
Does Papaya support Claude Managed Agents?
Yes. Claude Managed Agents can export every session over OpenTelemetry, and you add Papaya as the destination in the Claude Console. Papaya then sorts your sessions into use cases and finds improvements and user signals, the same way it does for agents you host yourself.
Can I still get traces when Anthropic runs the agent?
Yes. Trace export sends each session to a destination you choose, including the system prompt, user messages, the agent’s replies, and its tool calls and results. You can read single sessions in the Claude Console, and send all of them to Papaya for analysis.
Do I need to change my agent’s code to use Papaya?
No. Setup happens in the Papaya app and the Claude Console: create a Papaya SDK key, then add Papaya as an OpenTelemetry destination with Full content. Your agents keep running as they are.
What does Papaya add on top of the Claude Console?
Papaya reads all of your sessions together, which nobody on a team has time to do by hand. It finds the business use cases your agent serves, groups the failures that keep repeating, and recommends ranked changes to the prompt, tools and model choices, with the sessions that show each problem.
What about sessions that ran before I connected Papaya?
Trace export sends new sessions only. We can pull your earlier sessions through the Claude API, so the first analysis can start from your existing history.
Related
Keep reading
Agent optimization
How to Turn Agent Traces into Improvements
Most teams collect traces and never act on them. The agent improvement loop that turns traces into ranked, shipped, and verified fixes, and what those fixes usually look like.
Article
Agent Optimization vs. Observability: Why Watching Isn't Fixing
Your dashboard saw the bad run and did nothing. The gap isn't between having data and not — it's between seeing what happened and knowing what to change.
Agent reliability
How to Detect AI Agent Failures Automatically from Traces
Most agent failures never throw an error. A practical procedure for finding them in production traces, and an honest look at what automated detection can and cannot do.

