# Papaya

> Papaya is continuous optimization for AI agents. Papaya analyzes AI agent traces and code to improve quality, cut latency, and reduce cost. It reviews prompts, context, tools, sub-agents, caching, and more.

## Key facts

- Continuous Optimization for AI Agents: Papaya analyzes production traces, finds the changes that improve agent quality and latency and reduce cost, ranks them by expected impact, and helps teams ship them.
- Connect with a one-line SDK wrapper around any LLM or agent client, an observability-tool integration, or a dataset export in any format.
- Trace collection is asynchronous, so Papaya adds no latency to an agent's request path.
- Every change is human-in-the-loop; Papaya can open pull requests for approved fixes and send Slack alerts.
- Pricing (never per seat): Developer: Free — 1 agent, up to 1,000 traces / month; Business: $500/month — Up to 3 agents, 50,000 traces / month; Enterprise: Contact us — Custom.
- Backed by Y Combinator: https://www.ycombinator.com/companies/papaya
- LinkedIn: https://www.linkedin.com/company/papayaai

## Product

- [Papaya](https://papaya.fyi/): Papaya analyzes AI agent traces and code to improve quality, cut latency, and reduce cost. It reviews prompts, context, tools, sub-agents, caching, and more. (updated 2026-10-19)
- [Pricing](https://papaya.fyi/pricing): Papaya pricing scales with the agents you monitor and the traces you send — never per seat. Start free, upgrade for automated PR creation and team workflows.
- [Research](https://papaya.fyi/research): Explore the research behind Papaya's 200+ optimization checks for AI agents, including context economy, tool quality, verification, and retry loops, plus FAQs. (updated 2026-10-14)
- [Docs](https://papaya.fyi/docs): Setup guides, answers, and support for Papaya — connecting your data, understanding findings, shipping fixes, metrics, and account management. (updated 2026-10-15)
- [How Papaya Works](https://papaya.fyi/docs/how-it-works): How Papaya turns your agent’s traces, and optionally its code, into ranked fixes for quality, latency, and cost, from connecting data to shipping the fix. (updated 2026-10-14)
- [Connect Your Code](https://papaya.fyi/docs/connect-your-code): Connect your agent code to Papaya with two GitHub apps: read-only Code Analysis, and Code Changes for pull requests. Prerequisites, install steps, and access. (updated 2026-10-15)
- [Change or Remove Code Access](https://papaya.fyi/docs/manage-code-access): How to change which GitHub repositories Papaya Code Analysis or Papaya Code Changes can use, pause their access, or uninstall either app from your organization or account. (updated 2026-10-15)
- [Contact](https://papaya.fyi/contact): Talk to Papaya about improving your AI agents. Book a demo and we'll work with you on your first agent to find fixes for quality, latency, and cost.

## Articles

- [Articles](https://papaya.fyi/articles): In-depth articles on agent optimization — context economy, tool quality, verification design, and getting production LLM agents to improve.
- [A trace shouldn’t be a wall of text](https://papaya.fyi/articles/a-trace-shouldnt-be-a-wall-of-text): Reading and comparing hundreds of agent traces puts demands on our attention. A trace viewer should help us follow what happened and decide where to look next. (updated 2026-10-21)
- [Agent Optimization vs. Observability: Why Watching Isn't Fixing](https://papaya.fyi/articles/agent-optimization-vs-observability): Observability records what your agent did. Optimization clusters failures, quantifies the fix, and ships it. Why watching isn't fixing. (updated 2026-06-22)
- [How to Evaluate AI Agents in Production: A Practical Framework](https://papaya.fyi/articles/how-to-evaluate-ai-agents-in-production): Learn how to evaluate production AI agents across outcomes, trajectories, critical steps, reliability, cost, and latency—with examples and rollout steps. (updated 2026-07-14)
- [Context Bloat: Too Much Context Makes Agents Worse](https://papaya.fyi/articles/ai-agent-context-bloat): Why oversized context makes AI agents slower, costlier, and less reliable — and how to reduce it without making the agent forget what matters. (updated 2026-07-24)
- [From Traces to Decisions: AI Observability Data Pipeline](https://papaya.fyi/articles/from-traces-to-decisions-ai-observability-data-pipeline): Learn how to turn heterogeneous agent traces into normalized, enriched, decision-ready observability data across providers and frameworks. (updated 2026-07-15)
- [How to Detect AI Agent Failures Automatically from Traces](https://papaya.fyi/articles/detect-ai-agent-failures-from-traces): A step-by-step method for detecting AI agent failures automatically from production traces, combining deterministic checks, LLM judges, and cohort comparison. (updated 2026-09-17)
- [AI Agent Failure Patterns: A Field Guide from Production Traces](https://papaya.fyi/articles/ai-agent-failure-patterns): Ten failure patterns that show up in production AI agent traces, how to detect each one automatically, and the fix that usually works. (updated 2026-09-23)
- [LLM-as-a-Judge for AI Agents: How to Build a Judge You Can Trust](https://papaya.fyi/articles/llm-as-a-judge-for-ai-agents): What research says about LLM judges, why agents are harder to judge than answers, and a seven-step method to build and calibrate a judge for agent traces. (updated 2026-09-29)
- [Langfuse Alternatives (2026): 7 Tools Compared](https://papaya.fyi/articles/langfuse-alternatives): Langfuse alternatives compared for 2026: Papaya, LangSmith, Braintrust, Arize Phoenix, Opik, Laminar, and Datadog, with pricing, licenses, and when to stay. (updated 2026-10-23)
- [LangSmith Alternatives (2026): 7 Tools Compared](https://papaya.fyi/articles/langsmith-alternatives): LangSmith alternatives compared for 2026: Papaya, Langfuse, Braintrust, Arize Phoenix, Opik, Latitude, and Laminar, with pricing, licenses, and LangSmith Engine. (updated 2026-10-23)
- [Arize Phoenix Alternatives (2026): 7 Tools Compared](https://papaya.fyi/articles/arize-phoenix-alternatives): Compare Arize Phoenix alternatives in 2026: Papaya, Langfuse, Opik, MLflow, Laminar, LangSmith, and Braintrust, plus Phoenix versus Arize AX. (updated 2026-10-23)
- [Braintrust Alternatives (2026): 7 Tools Compared](https://papaya.fyi/articles/braintrust-alternatives): Braintrust alternatives compared for 2026: Papaya, Langfuse, LangSmith, Arize Phoenix, Opik, Confident AI, and Latitude, with pricing, licenses, and when to stay. (updated 2026-10-23)
- [Datadog LLM Observability Alternatives (2026): 8 Tools Compared](https://papaya.fyi/articles/datadog-llm-observability-alternatives): A comparison of Datadog LLM Observability (now Agent Observability) alternatives in 2026, with per-span pricing, a worked cost example, retention, and when to stay on Datadog. (updated 2026-10-21)
- [Best AI Agent Observability Tools in 2026](https://papaya.fyi/articles/best-ai-agent-observability-tools): Compare Papaya and nine AI agent observability tools by tracing, evaluation, recurring failure detection, pricing, and production workflow fit. (updated 2026-10-23)
- [Best AI Agent Evaluation Platforms for Startups (2026)](https://papaya.fyi/articles/best-ai-agent-evaluation-platforms-for-startups): Best AI agent evaluation platforms for startups in 2026: Papaya, Braintrust, Langfuse, LangSmith, DeepEval, Opik, and Arize, with free tiers, pricing, and startup programs. (updated 2026-10-23)
- [How to Debug AI Agents in Production](https://papaya.fyi/articles/how-to-debug-ai-agents-in-production): A step-by-step process for debugging AI agents in production: find the first wrong decision, check context and tools, confirm the pattern, and verify the fix. (updated 2026-09-08)
- [How to Turn Agent Traces into Improvements](https://papaya.fyi/articles/turn-agent-traces-into-improvements): Turn production AI agent traces into improvements with a repeatable loop: segment by use case, find what separates failures, quantify impact, ship, and verify. (updated 2026-09-25)
- [How to Evaluate Multi-Step AI Agents Without Labeled Data](https://papaya.fyi/articles/evaluate-ai-agents-without-labeled-data): Six signals for evaluating multi-step AI agents without labeled data, from final-state checks and user behavior to reference-free judges, pass^k, and cohorts. (updated 2026-09-11)

## Blog

- [Blog](https://papaya.fyi/blog): Papaya updates and essays on improving production AI agents: context economy, tool quality, verification design, evaluation, and continuous optimization.
- [The Easiest Platform to Analyze and Improve AI Agents](https://papaya.fyi/blog/easiest-platform-for-agent-trace-analysis): Papaya needs one SDK wrapper and no configuration. It reads any trace format, sorts runs into use cases, and surfaces improvements and user signals in a trace viewer people can read. (updated 2026-10-27)
- [Observability is not enough.](https://papaya.fyi/blog/observability-is-not-enough): Teams are buying observability, but it seems that what they need is prediction and action. (updated 2026-06-12)
- [Product Updates - June 16](https://papaya.fyi/blog/product-updates-june-16): More ways to connect your data, basic observability in the Papaya app, and more from this week's releases. (updated 2026-06-16)

## Authors

- [Macy Mody](https://papaya.fyi/authors/macy-mody): Macy Mody writes about production LLM agents, agent observability, and AI agent optimization at Papaya.
- [Niyaz Puzhikkunnath](https://papaya.fyi/authors/niyaz-puzhikkunnath): Niyaz Puzhikkunnath writes about evaluating and improving production AI agents at Papaya.
- [Faiz Vadakkumpadath](https://papaya.fyi/authors/faiz-vadakkumpadath): Faiz Vadakkumpadath writes about AI observability, trace pipelines, and data engineering for production agents at Papaya.

## Legal

- [Privacy Policy](https://papaya.fyi/privacy-policy): Learn how Papaya collects, uses, shares, retains, and protects personal information across our website and AI agent optimization services.
- [Terms of Service](https://papaya.fyi/terms-of-service): Review the terms that govern access to and use of Papaya's AI agent optimization platform, related software, tools, and support services.

## FAQ

### How is Papaya different from LangSmith, Langfuse, or Braintrust?

Those tools show you what happened in each run. Papaya picks up from there: it groups your traces into use cases by what users are trying to do, finds the problems that repeat across thousands of runs, and hands you a ranked list of fixes with the runs behind each one.

You don't need to replace the tool you use today. Papaya works alongside it and can import your traces from it.

### What kinds of problems does Papaya find? Is it just prompts?

No. Papaya runs 200+ analyses across prompts, context and memory, retrieval, tool calls, sub-agents, prompt caching, and model choice. It merges overlapping findings into a short list of fixes, ranked by their expected effect on quality, latency, and cost.

### How do I connect my agent, and how much work is it?

Wrap your LLM client with the Papaya SDK, which supports OpenAI, Anthropic, Vercel AI SDK, Bedrock, and Gemini, or use the fetch wrapper for a custom agent loop. Traces are sent asynchronously, so Papaya never slows down your agent.

Setup takes a few minutes, and you don't have to do it by hand: paste our setup prompt into Claude Code or Codex and your coding agent does the integration for you.

### What data does Papaya need, and how much?

Full traces work best: the system prompt, user messages, model responses, tool calls with their results, and any sub-agent steps, plus timestamps and model names so Papaya can work out latency and cost. Partial exports still work, but the fixes get less specific.

About 100 traces from one agent is enough for a first analysis.

### Why should I trust a recommendation?

Every recommendation links to the runs it came from, highlights the exact spot in each trace, and includes trace IDs you can look up in your own logs. The evidence is quoted from your traces, not written by a model.

Each one also shows how many runs it affects and what fixing it is worth, so you can check it before you act.

### How does Papaya know whether a run succeeded?

For each use case, Papaya builds an LLM judge from your traces, so you don't have to write a rubric. The judge checks each step along the way, such as tool calls, sub-agent hand-offs, and errors, and then whether the run as a whole achieved what the user wanted.

You can edit any judge, bring your existing evals, or add user feedback such as thumbs up and down.

### Do I have to define my business use cases?

No. Papaya reads your traces and groups them into use cases by what users are trying to do, then analyzes each use case on its own. If you already label your traces, it can use your labels instead, and you can rename or merge use cases.

### Do you need access to my code?

[Connecting your code](https://papaya.fyi/docs/connect-your-code) makes a big difference. Papaya can pinpoint exactly what to change, down to the prompt or line of code, and open a pull request with the fix for you to review. It also finds whole classes of problems that traces alone don't show, such as misconfigured settings or broken prompt caching.

So code access isn't required, and Papaya finds plenty from traces alone, but connecting your code greatly improves the results.

### What kinds of agents does Papaya work with?

Any agent that does repetitive business tasks is a good candidate. Papaya compares many runs of the same task to learn what works and what doesn't, and because the agent runs again and again, every gain in quality, latency, and cost compounds.

There's no limit on the kind of agent. Papaya can read traces from any agent, including proprietary and custom-built ones: multi-step agents with sub-agents and hundreds of tool calls, chat and voice agents, and batch pipelines.

Besides the SDK, Papaya accepts OpenTelemetry data and exports from the observability tool you already use.

### How do you handle sensitive data and PII?

Papaya automatically scrubs common personal data, such as email addresses and phone numbers, before trace data is stored, and you can strip fields or leave out specific customers in your own code before anything is sent.

All data is encrypted in transit and at rest, and access to it is restricted and logged.

### Is Papaya SOC 2 compliant? Can it be self-hosted?

Papaya has completed its SOC 2 Type 2 audit, and the final report is on its way. You can request our security documentation, data processing agreement, and subprocessor list from the [Trust Center](https://trust.papaya.fyi/). We sign NDAs, and BAAs for healthcare data.

Papaya runs as a hosted service today. If you need a self-hosted deployment or data kept in a specific region, talk to us.

### How much does Papaya cost? Is there a free plan?

Yes, there's a free Developer plan, and your first analysis is on us: share traces from one agent in any format and we'll walk you through the results.

Paid plans are priced by trace volume, never per seat. See the [pricing page](https://papaya.fyi/pricing) for current plans and limits.