Docs / How it works

How Papaya works

Papaya reads your agent’s production traces, and its code if you connect it, finds the changes that will improve quality, latency, and cost, and helps you ship them. Here is what happens at each step.

From traces to fixes

Three steps from raw traces to a fix you can merge.

Wrap your LLM calls and then spend your time building the new features customers want, not on agent maintenance.

Step 01

Connect to your traces

Using the Papaya SDK, you can wrap any LLM or agent client with a single line of code. Alternatively, connect Papaya directly to your observability tool or share a dataset in any format. Papaya will automatically detect the shape of the data. Connecting your code repository is optional, but it improves the results.

Step 02

Papaya analyzes and finds improvements

Papaya runs 200+ research-backed analyses against your traces. It identifies the improvements that matter most and ranks them by expected impact.

Outcome & Trajectory
Prompt & Planning
Context, Memory & Retrieval
Tool Execution Reliability
Cost & Model Routing
Evidence-Backed Validation
Step 03

Review and implement suggestions.

Understand the exact runs that are producing a recommendation and then choose to implement. Automatic alerts in your tool of choice when a new optimization is found.

  • The exact run that produced the finding
  • The prompt, tools, and context as the model saw them
  • Estimated impact, risk, and confidence
Step 01 in detail

What you can connect

Start with whichever is easiest. Traces are all Papaya needs for a first analysis.

  1. Papaya SDK

    Wrap any LLM or agent client with one line of code. Supports OpenAI, Anthropic, Vercel AI SDK, Bedrock, and Gemini.

  2. Fetch wrapper

    Built your own agent loop? The fetch wrapper works with any model provider.

  3. Your observability tool

    Connect Papaya to the tool that already collects your traces.

  4. A dataset in any format

    Share an export for a one-off analysis, even a raw one. Papaya detects its shape.

  5. Your code repositoryOptional

    Optional, but it improves the results: findings tie to the code behind them, and fixes arrive as pull requests.

Step 02 in detail

What Papaya looks for

Papaya runs 200+ research-backed analyses across six areas of agent behavior, then ranks what it finds by expected impact on quality, latency, and cost.

Read the research behind the analyses
Analysis run200+ checks
  • Outcome & Trajectory
  • Prompt & Planning
  • Context, Memory & Retrieval
  • Tool Execution Reliability
  • Cost & Model Routing
  • Evidence-Backed Validation
Findings ranked by expected impact
Step 03 in detail

What every recommendation includes

P1Recommendation · Example

Context bloat

The same conversation history is sent again on every turn.

Affected runs
18.0% of runsView runs →
What the model saw
PromptToolsContext
Expected impact
−$2.1K/mo costConfidence: high
Proposed fix
Summarize earlier turns instead of resending them Open pull request
  1. The runs behind it

    The exact runs that produce the finding, and how many runs it affects.

  2. What the model saw

    The prompt, tools, and context as the model saw them in those runs.

  3. Expected impact

    Estimated impact on quality, latency, and cost, with risk and confidence.

  4. A fix you can ship

    A proposed change you can review, delivered as a pull request or a Slack alert.

After you ship

Papaya keeps watching

Shipping a fix starts the next loop, not the end of one.

  1. 01

    You ship the fix

    You choose which fixes to ship. Papaya can open a pull request for each one you approve.

  2. 02

    Impact is measured

    As new traces come in, Papaya shows what the fix changed for quality, latency, and cost.

  3. 03

    Drift is caught

    Live alerts when quality metrics drift, before a customer escalates.

  4. 04

    Findings get sharper

    Every fix and every new run feeds the next analysis. Nothing starts from zero.

Get started

Start with one workflow.
Your first analysis is on us.

Share traces from one workflow, in any format, and we’ll walk you through the ranked fixes. Questions first? The docs FAQ covers setup, data, and models.