How the execution-quality numbers actually work
(there's no single score)

This is the written, formula-level companion to the interactive explainer "What execution quality actually costs you", for readers who want the mechanics, not just the chart.

The promise, and where you actually land

Every DEX aggregator shows you a quote before you trade: "swap this, get that." Think of the quote as a promise pointing at an amount, call it quoted_out. Then the trade actually executes, and you land somewhere else, at realized_out. Almost always a little short of the promise, sometimes exactly on it, occasionally not at all (the trade reverts and you land nowhere).

The distance between the promise and where you land, in basis points, is the one number every aggregator would rather you not look at too closely. ClearTrace's whole job is measuring that distance and then asking a second question most tools skip: what caused it?

The comparison is not new and we are not renaming it. In trading it is implementation shortfall, the gap between the decision price and the achieved price, and the obligation it serves is best execution. This repo calls its own instance quoted-vs-realized, which is the term used throughout the source linked at the foot of this page. What is specific here is the instrument: the sampler asks an aggregator for a real quote, then replays that aggregator's own swap calldata in a fork simulation and reads the output the route produces. The promise is recorded from the aggregator's API and the outcome is observed from a simulated execution of its own route, rather than inferred from a price oracle. That holds for the output gap and the components it decomposes into. It does not hold for gas_bps or all_in_cost_bps: the gas units come from the simulation, but valuing them in USD uses a price feed, and the size cohorts are sized the same way.

Two numbers that look like "the score" and aren't

There are exactly two single-number "quality scores" anywhere in ClearTrace's code, and both are narrower than they sound:

• slippage_score (public API): 100.0 - median_abs_gap_bps, clamped to [0, 100]. The field description says plainly: "This is a slippage-only metric; MEV exposure and revert rates are reported separately." It is narrower still than that sounds — see what median_abs_gap_bps is actually measuring. (That field was called avg_slippage_bps until 2026-09-25. The old name is still served with the same value until 2027-03-31, but it is deprecated, because it says average and the number is a median.)
• execution_quality_score (the Dune-side leaderboard query): 100 - LEAST(100, median_slippage_bps). Same idea, same limitation: it's slippage rebased to a 0–100 scale, nothing more.

Neither number knows whether the trade reverted. Neither knows whether MEV took a cut. Neither knows whether the aggregator routed through an off-chain market maker instead of an on-chain pool. If you only ever saw one of those two numbers, you'd think Odos (among the cheapest fills in the whole sample) was the best aggregator to route through, and you'd be routing into a venue whose quotes fail to realize on roughly one trade in seven overall, and nearly one in five at the smallest trade size. A single score would have hidden the most important fact about it. So ClearTrace doesn't publish one.

The actual formula: decomposing a quote's gap

What we compute instead is a decomposition: the same total gap, broken into the pieces that caused it, all still in basis points so they're comparable across trade sizes and tokens.

output_gap_bps  = (quoted_out - realized_out) / quoted_out * 1e4
fee_bps         = explicit_fee_out / quoted_out * 1e4
mev_bps         = mev_toll_out     / quoted_out * 1e4
slippage_bps    = output_gap_bps - fee_bps - mev_bps
all_in_cost_bps = output_gap_bps + gas_bps

Reading it symbol by symbol:

• quoted_out: what the aggregator promised. This is the reference base for everything else; every term below is "bps of the promise."
• realized_out: what you actually received. On a perfect fill, this equals quoted_out and the gap is zero.
• output_gap_bps: the total shortfall between promise and reality, in basis points (× 1e4 converts a fraction to bps).
• explicit_fee_out → fee_bps: the slice the aggregator openly took as a fee (including any "positive slippage" it kept for itself rather than passing back to you).
• mev_toll_out → mev_bps: a reserved slot for the slice a sandwich attacker or similar extracted. It is 0.0 on every sampler row, and structurally must be: these fills come from a fork simulation, a fork has no adversary in it, so there is no toll to observe. Nothing is hidden by the zero, but nothing is measured by it either. See the note below.
• slippage_bps: whatever is left over after fees and MEV are subtracted out. This is the residual: the part of the gap nobody can point to a specific cause for. It's defined as a remainder on purpose, so the three components always add back up to the total, nothing gets double-counted or lost. Because mev_bps is always zero here, in practice this is the gap minus the explicit fee, and it carries no MEV term at all.
• gas_bps: the gas the trade cost, expressed as bps of trade size (a separate line, since gas is paid in the native token, not skimmed from the output).
• all_in_cost_bps: the number that actually answers "what did this trade cost me": the output gap plus gas. It is null whenever gas could not be valued, rather than quietly falling back to the gap alone.

Coverage: the decomposition is unavailable on more than a third of rows. detect_output_skim returns no detection whenever the taker was paid straight out of a pool, and an unmeasurable fee is written as null rather than as 0.0, so fee_bps is null and slippage_bps is null with it. That is 39,306 of the 109,692 committed rows carrying a measured gap, 35.8%, across all eleven venues. Those rows still carry output_gap_bps: the total shortfall is measured, only its split into fee and residual is not. Read any statement that a gap was "decomposed into fee and slippage" against that share. The rule the module enforces is that null is unmeasured and 0.0 is a finding; until 2026-09-16 this argument defaulted to 0.0, which pushed an unmeasurable fee's whole gap into the residual. Source: app/quote_decomposition.py.

Correction, 2026-08-21. Between 2026-06-23 and 2026-08-21 the sampler wrote gas_bps = 0.0 on every row instead of leaving it unset, so all_in_cost_bps was identical to output_gap_bps and this page's "the output gap plus gas" described a number with no gas in it. The formula above was always the intent; the pipeline was not implementing it. Samples now carry a gas_valuation column recording how gas was priced, either live (read at capture) or reconstructed-hourly (recovered afterwards from the block's base fee), and null where it could not be priced at all. On Base, Arbitrum and Optimism the figure covers L2 execution only and excludes the L1 data fee, so it is a lower bound there. Method and caveats: docs/METRICS.md.

Correction, 2026-09-15. Until today the mev_toll_out line above read "the slice a sandwich attacker or similar extracted, detected from on-chain trace data", which described a measurement this page has never carried. mev_bps is 0.0 on all 114,533 decomposed rows, and no caller anywhere in the pipeline populates mev_toll_out. Unlike the gas_bps zero corrected above, this one is not a pipeline defect: the sampler measures fills on a fork simulation, a fork contains no adversary, and a sandwich toll that cannot occur cannot be observed. The number is right and the description was wrong. The consequence that matters is a comparison, not a value. This page's slippage_bps is MEV-free by construction, while a realized on-chain slippage figure measured from settled trades contains whatever MEV was extracted from them. The two are different quantities and must not be read against each other. Our sibling page on the execution verifier has carried the fork-has-no-adversary caveat since it was published; this page did not, and the two disagreed in public until now. Registry entry: app/metric_findings.json.

One deliberate exclusion: reverted trades are never forced into this formula. If the trade fails, nothing was decomposed, because nothing happened: no fee, no MEV, no slippage, just gas spent on a transaction that went nowhere. Reverts get counted separately, as a rate across a batch of samples, not smuggled into a per-fill bps number. That distinction is why the companion interactive page reports "fails to realize" as its own axis instead of folding it into cost.

In the current sample the three characters this produces are stark. Odos posts among the cheapest realized fills of any aggregator (a near-zero median cost) but roughly 14.5% of its fork-verifiable quotes fail to realize overall, rising to 19.9% at the $1k size tier. KyberSwap shows a third character again: 86% of its 448 sampled $1M routes carry at least one off-chain RFQ leg (quotes a fork can't replay, so they're read on a routing axis rather than counted as failures), and its genuine on-chain revert rate stays low. Read that as one venue's number, not a ranking — the cross-venue RFQ comparison was withdrawn on 2026-09-08 as a detection artifact, because the sampler could only parse a route out of KyberSwap's quote format. OpenOcean almost never fails (0.4%) but its median realized cost ex-gas, about 11.5 bps, runs an order of magnitude above the near-zero-cost routers. Three different axes, three different stories, and averaging them into one number would erase all three.

The second formula: when a number is allowed to be cited

Decomposition answers what a gap is made of. A separate rule answers a different question: is this number stable enough to act on?

rated = (realized_samples >= 30) AND (span_days >= 7)

A cell, one (aggregator, pair, size) combination, only earns the label "rated" once it clears both bars: at least 30 samples that actually realized (not reverts), spread across at least 7 calendar days. Both conditions matter, not just sample count: thirty samples taken in one afternoon are highly correlated (same market minute, same gas conditions) and don't tell you how the aggregator behaves across varied conditions. Until a cell clears both bars, it's labeled "preliminary" and is shown, but never ranked against rated cells.

This maps to a maturity ladder we state plainly to buyers: first rated cells at roughly one week of continuous sampling, a credible preliminary scorecard at about 30 days, evidence-grade (the bar for citing a number in a compliance or best-execution context) at around 90 days of continuous sampling, strengthening every month after that.

Three things you can use this week

If you're a trading desk or compliance function evaluating whether to trust an execution-quality number, from ClearTrace or anyone else:

1. Ask for the decomposition, not the headline number. A single blended bps figure can hide a reliability problem behind a good median. Odos's near-zero median cost tells you nothing about its 14.5% failure-to-realize rate unless you ask for both.
2. Check the confidence label before you cite a number as evidence. "Preliminary" and "rated" mean different things, and the difference (30 samples, 7-day span) is exactly the kind of methodology detail an auditor or regulator will ask about under a best-execution requirement. A number without a confidence label attached isn't evidence, it's a claim.
3. Match the metric to the decision, not the marketing headline. Cheapest fill, most reliable, and best-routed-at-size are three different questions with three different winners in this sample. The "best aggregator" answer changes with the trade you're actually making. Pick the axis that matches it, not whichever number the aggregator chose to put on their homepage.

What median_abs_gap_bps is actually measuring

Both scores above are built on one on-chain number: the median absolute gap between the minute-oracle USD value of the token you received and of the token you gave. Absolute, and a median. Those two choices do three things that are easy to miss, and we would rather state them than have someone discover them in a ranking.

• It scores integrator fees as execution cost. A fee charged inside the fill sits in exactly the same place execution cost sits, and neither on-chain source table carries a fee-recipient column — so on this metric the two cannot be told apart. A venue whose flow arrives through fee-taking front-ends reads worse than one whose flow does not, at identical execution. We have root-caused this on real venues more than once.
• It discards direction. Because the gap is absolute, a venue whose takers systematically receive more than oracle value reads the same as one whose takers lose. A systematic gain against the oracle is not execution quality either — it usually means the population being measured is not a user trade.
• It has a noise floor. For a gap distribution centred near zero, the median of the absolute value is exactly the 75th percentile of the signed gap. So a venue whose true cost is below the minute oracle's own dispersion does not publish a small cost — it publishes that dispersion. Low single-digit figures are ranking venues by how quietly their token pairs are priced, not by execution, and should not be read against a venue carrying a genuine two-sided cost.

What we do about it. We disclose rather than exclude. Dropping the fee-bearing fills would mean deciding which of a venue's real costs count, on a rule we cannot falsify — and every one of those fills is a trade that really happened at that price. The same choice is made on the sampler side, where venue fees we have established are disclosed in a curated registry rather than subtracted. The defect is registered in the repo as exec-slippage-includes-integrator-fees-and-drops-sign-2026-09-09.

Methodology & sources

Data snapshot: ClearTrace quote sampler, Ethereum, Jun 23 – Jul 20 2026 window, 23,352 samples, 7 aggregators, 4 size cohorts ($1k / $10k / $100k / $1M). Figures match the companion interactive page and trace to the pinned snapshot research_drafts/snapshots/quote_samples_2026-07-20.csv (frozen 2026-07-20 so the numbers stay reproducible as the live seed keeps growing). Formulas and thresholds trace to the ClearTrace repo: