LangSmith vs Langfuse vs Braintrust: LLM Tracing 2026

Key takeaways

• Tracing is not evaluation. A trace records what happened during a run; an evaluation applies a scorer, rubric, assertion, or human judgment to an output or trace.
• Offline and online evals solve different jobs. Offline evals test a controlled dataset before release. Online evals score sampled or selected production traffic after release.
• OpenTelemetry improves trace portability, not full-platform portability. Spans can fan out through an OTel collector, but datasets, evaluators, annotations, dashboards, and prompt workflows still use vendor-specific models.
• Open source is not a binary row. Langfuse's repository applies MIT Expat to most code while reserving separate licensing for named enterprise directories. Confirm the exact features and license terms you plan to run.
• CI gating is configured, not magical. Each platform documents a way to run experiments or evals in CI. Your workflow still needs an explicit threshold and a failing exit or check before it can block a merge or deployment.
• Pricing, retention, hosting, and compliance are procurement inputs. Public plan details change, and enterprise controls can depend on contract and deployment mode. Recheck the linked official pages before purchase.

Quick decision matrix

Start with the workload, not the logo

A useful proof of concept should contain more than a single LLM call. Instrument one representative workflow with retrieval, at least one tool call, a retry or error path, and sensitive fields that must be masked. Then run the same small evaluation set through each candidate.

Measure:

1. time to a readable parent-child trace;
2. effort to attach user, session, model, token, cost, and version metadata;
3. support for your Python, TypeScript, agent framework, or raw API path;
4. dataset and scorer authoring ergonomics;
5. how a failed offline score appears in a pull request;
6. how production traces become reviewed examples or datasets;
7. export, retention, masking, and access-control behavior;
8. cost at your expected span, score, storage, and seat volume.

That test preserves the page's three-provider selection job. If the next question is whether to buy a hosted service rather than which tracing platform fits, use APIScout's AI eval API and service shortlist as the separate service-selection step.

Tracing vs offline evals vs online evals

Tracing: reconstruct a run

Tracing captures the execution path: prompts, model calls, retrieval, tools, latency, token use, metadata, errors, and nested spans. It answers questions such as "which tool returned the bad record?" or "where did latency enter this agent run?"

A trace can contain feedback or scores, but collecting a trace does not establish that the answer was correct. It provides the evidence an evaluator or reviewer can inspect.

Offline evals: test before release

Offline evaluation runs a version of the application against a controlled dataset. The dataset may contain expected values, reference answers, edge cases, or human-reviewed examples. Scorers can be deterministic code, model-based judges, similarity measures, or human labels.

This is the right layer for regression tests and candidate comparisons. A release check should define the dataset version, scorer version, baseline, tolerated variance, and failure threshold. Without those controls, a green average score can hide a serious failure on a critical slice.

Online evals: monitor production behavior

Online evaluation scores production traces after they are created. It is useful for drift, safety or policy classifications, topic discovery, user-feedback correlation, and selecting failures for human review. Because online scoring can add model cost, teams commonly sample traffic or apply rules to selected traces.

Online scoring is monitoring evidence, not a substitute for pre-release tests. A practical loop is: detect a production failure, add the case to a reviewed dataset, reproduce it offline, fix it, and then monitor the same class after release.

OpenTelemetry portability: what actually moves

OpenTelemetry gives teams a shared vocabulary for traces and spans, plus collectors that can route telemetry to more than one backend. LangSmith documents OTel tracing and collector fan-out. Langfuse accepts OTel-shaped instrumentation through its observability stack. Braintrust documents an OTel integration for sending spans into its platform.

That makes a dual-write proof of concept possible. It does not make every platform object interchangeable. Before calling a setup portable, inventory what would need migration:

• span names and attribute conventions;
• masking or redaction rules;
• datasets and examples;
• scorer code and model prompts;
• annotations and reviewer queues;
• dashboards and alerts;
• prompt versions and experiment history;
• user/session identifiers and retention policies.

Treat the collector and semantic conventions as the portability seam. Treat each platform's evaluation and workflow objects as an adapter that may require migration work.

LangSmith: strongest when LangChain is already the application seam

LangSmith's official observability path covers instrumenting applications, investigating traces, and monitoring production behavior. Its evaluation documentation separates offline tests from online evaluators and supports dataset-backed experiments, regression tests, and production feedback loops.

For LangChain and LangGraph users, automatic or low-friction instrumentation is a practical advantage. Teams outside that ecosystem can use LangSmith SDK instrumentation or OpenTelemetry, so "LangChain-only" would be inaccurate. The real question is whether the organization wants LangSmith's trace and evaluation objects to sit alongside its LangChain/LangGraph development model.

LangSmith's public pricing page listed a one-seat Developer plan with a base monthly trace allowance and a paid Plus plan priced per seat with a larger base allowance when this guide was rechecked. It also distinguished shorter-lived base traces from longer-retained extended traces. Treat those numbers and retention windows as a dated snapshot, not a permanent quote; verify current usage units, overages, seats, and retention before modeling cost.

Enterprise documentation describes hybrid and fully self-hosted options. Availability, architecture, support, and security controls should be confirmed with the vendor for the intended region and data class.

Choose LangSmith when:

• LangChain or LangGraph already defines much of the application workflow;
• the team wants observability and evaluation in the same product model;
• dataset experiments and online evaluators match the release process;
• the procurement team accepts the current seat, usage, and retention structure.

Validate carefully: non-LangChain instrumentation effort, high-volume trace economics, long-term retention cost, data export, and the exact enterprise deployment contract.

Langfuse: strongest when open-source control leads the decision

Langfuse documents tracing, prompt and model metadata, sessions, users, scores, and production analysis in its observability layer. Its evaluation workflow connects production traces, datasets, experiments, and manual or automated evaluators, with both offline and online paths. It also documents experiments in CI/CD, including a GitHub Actions workflow that can gate changes before production.

The license needs precise wording. The repository license says content outside specified enterprise directories is available under MIT Expat, while named enterprise directories use a separate license. That is materially different from claiming every feature is MIT. Review the repository license and the self-hosting feature matrix for the version and capabilities you intend to operate.

Langfuse offers a free cloud entry point, paid self-service tiers, and enterprise options; its public pricing model uses included usage units plus overage pricing on relevant plans. Self-hosting can reduce vendor dependence, but it does not eliminate costs: the documented stack includes application services and data infrastructure that your team must secure, back up, scale, and upgrade.

Langfuse documents configurable data-retention behavior and publishes security and compliance information. Confirm cloud-region availability, retention rules, SSO, audit controls, support, and any BAA or contractual requirement rather than inferring suitability from a badge.

Choose Langfuse when:

• self-hosting or open-source inspectability is a primary requirement;
• the application spans several frameworks or uses raw model APIs;
• OTel-oriented instrumentation fits the existing telemetry strategy;
• the team is prepared to own the operational surface it chooses to self-host.

Validate carefully: which capabilities use enterprise licensing, infrastructure sizing, upgrade and backup procedures, cloud usage-unit economics, and evaluator behavior at production volume.

Braintrust: strongest when evals drive the operating loop

Braintrust's evaluation documentation explicitly separates offline experiments from online scoring. It describes running evals from code or UI, tracking experiment history, scoring production traces, and automating evaluation runs in CI with GitHub Actions or the bt eval CLI. That makes the eval workflow easy to evaluate as a coherent system.

The important qualification is that Braintrust does not automatically block every deployment simply because an eval exists. A team must wire the CI job, comparison, and threshold into branch or deployment policy. LangSmith and Langfuse also document CI/evaluation paths, so "only Braintrust can gate CI" is no longer supportable.

Braintrust's public pricing page listed a $0 Starter plan and a $249/month Pro plan when rechecked, with usage allowances and overages expressed in processed data and scores. Recalculate against current pricing because score volume, model calls, trace data, and retention can move the total beyond the base plan.

Braintrust is also no longer accurately described as managed-only. Its security documentation describes SaaS, BYOC, and self-hosted deployment options, configurable retention policies, data export, and compliance claims. Deployment mode changes who operates the data plane and which controls are available, so validate the exact architecture and contract.

Choose Braintrust when:

• datasets, scorers, experiments, and production scoring are the main team workflow;
• pull-request evals are part of the release discipline;
• reviewers need production failures to flow back into datasets;
• its usage and deployment model fits the expected volume and security posture.

Validate carefully: trace-only ergonomics, score and model-call cost, CI failure semantics, export fidelity, and the operational responsibilities of BYOC or self-hosting.

Pricing, retention, security, and compliance checklist

Do not compare only the headline monthly price. Request or model all of the following:

Vendor security pages are first-party statements, not an independent audit. Ask for the current report or trust-center evidence under the vendor's access process and map it to your own threat model.

A practical evaluation sequence

1. Instrument one workflow in all three tools. Keep span names and core attributes consistent.
2. Create one shared reviewed dataset. Include ordinary cases, failures, and a critical slice.
3. Implement one deterministic scorer and one model-based scorer. Version both.
4. Run an offline experiment. Record runtime, score variance, debugging experience, and CI output.
5. Create one production scoring rule. Use sampling and estimate monthly scoring cost.
6. Test a real failure loop. Promote a bad production trace into the dataset, patch the application, and prove the offline regression catches it.
7. Export the project. Verify what leaves cleanly and what remains platform-specific.
8. Price the measured workload. Use observed spans, data size, scores, model calls, seats, and retention—not a guessed request count.

If implementation is the next job, keep it separate from vendor selection: PkgPulse covers JavaScript packages for AI-agent evals, while StarterPick covers SaaS observability and eval architecture. For a learning path before adopting the workflow, CourseFacts maintains an AI-agent evaluation course guide.

Final recommendation

• Pick LangSmith for the most direct LangChain/LangGraph-aligned observability and evaluation workflow.
• Pick Langfuse when open-source deployment control and framework-neutral tracing carry the most weight.
• Pick Braintrust when eval datasets, experiments, production scoring, and CI checks are the center of the team's operating model.

Those are editorial fit judgments based on the workflow above, not vendor-certified rankings. Re-run the proof of concept when pricing, deployment mode, data requirements, or application architecture changes.