Learnings
What a Programmatic Scorecard Should Actually Measure
Collin Davis
Chief Product Officer
Most dealers get handed a media report that is full of true statements and answers none of their questions.
Impressions delivered: 1.2 million. Video completion rate: 94%. Unique households reached: 84,000. Every number accurate and auditable — and a general manager reading it has no idea whether the money did anything.
This isn't a reporting failure. It's a category error. Those metrics describe what we did. The question is what changed. Not the same question, and no amount of delivery detail turns one into the other. Here's what a scorecard has to answer instead.
Question 1: Did the right households see it enough times to matter?
Reach and frequency get reported as separate numbers, which hides the thing that matters — how they're distributed. Advertising effect isn't linear in exposures: two exposures is practically indistinguishable from zero, but somewhere around four to six over a few weeks, an impression becomes a memory. Above a dozen, you're mostly paying to annoy people who already got the message.
So “84,000 households reached, average frequency 3.2” can describe two very different months: 84,000 households at roughly 3 exposures each, almost all under the threshold — or 30,000 at 6–7 exposures and 54,000 at 1, where a third of the audience is genuinely reached and the rest wasted.
The second is better and the average hides it. A scorecard should report the frequency distribution, not the mean — what share of reached households landed in the range where exposure actually does something. And “the right households” matters too: reaching 84,000 households in a DMA where your store draws from three ZIP codes is a large number describing a small amount of relevant delivery.
Question 2: Did behavior change here, versus somewhere comparable where we didn't run?
This is the question almost no dealer report attempts, and it's the only one that establishes causality.
The standard approach is to look at what happened during the flight and attribute the good parts to the media. Sales were up 8% in September, we ran streaming in September, therefore streaming produced the lift. That's not measurement — it's coincidence with a narrative attached, and it fails in both directions: crediting media for a strong market, blaming media for a soft one.
The fix is comparison. Hold out a comparable market — similar demographics, competitive set, seasonality — and don't run there. Or stagger launch timing across markets so each one is its own before-and-after with a concurrent control. Then the question becomes answerable: did the market where we ran move differently from the market where we didn't?
That's harder to set up and it costs something — a holdout market is a market you're not advertising in. It's also the difference between knowing and believing. Most dealers have never been offered the option, which is a fair criticism of the industry rather than of the dealers.
Question 3: Does it show up in the dealership's own system of record?
This is where most attribution quietly falls apart. The ad platform knows what it delivered, and may know a household later visited your site or lot. What it doesn't know is what happened after — whether the appointment showed, the deal closed, or what the gross was.
Platform-reported conversions describe an event the platform can see. The dealership's DMS and CRM describe an event the dealership got paid for. Those two numbers will not match, and the second one is the real one.
A scorecard worth the name reconciles to the dealership's system of record — not “the platform reported 340 conversions” but “of the units sold this month, here's what we can associate with exposure, measured where the sale was actually recorded.”
That reconciliation is unglamorous and produces smaller numbers than platform reporting does — and it is the strongest signal that a partner is measuring rather than marketing. One whose reported conversions are always larger than your DMS shows isn't measuring your business.
Question 4: Would it survive one hard follow-up from a GM?
The practical test. Take any number on the report and ask the obvious skeptical question:
“How do we know these units wouldn't have sold anyway?”
“This says 94% completion — on what kind of screen, and does anyone actually watch it?”
“You reported 340 conversions and my CRM shows 60. Which is it?”
“If I cut this in half next month, what specifically gets worse?”
A report that absorbs those questions is a scorecard. One where the answers are “it's a proxy” or “directionally” is a delivery summary in a nicer template — and the last question is the most revealing: if nobody can say what would get worse, nothing on the report is load-bearing, and that's the report that gets cut on the 18th, correctly.
What's theater
Worth naming directly, because a scorecard is defined as much by what it excludes:
Impressions delivered. Confirms the media ran. Says nothing about effect — a billing check, not a result.
Completion rate on non-skippable inventory. If the format doesn't allow skipping, a 94% completion rate mostly measures whether the ad loaded, not engagement.
Any metric with a name invented by the vendor reporting it. If a term appears nowhere else in the industry, ask what standard metric it's built from and why that one isn't shown instead.
Click-through rate on streaming TV. People do not click their television. It measures a small, unrepresentative slice of behavior and treats it as the point.
Cost per impression as a headline. A cheaper way to reach the wrong households at insufficient frequency is a worse outcome, not a better price.
Why program-level reporting doesn't solve this
If you're in a manufacturer digital program, it's tempting to treat that program-level reporting as the measurement question answered. It isn't — structurally, not as a criticism. It exists to compare providers on a consistent basis, which means using metrics standardized across every vendor: delivery, completion, reach. Legitimate purpose, serves the program well.
What it can't do is tell one dealer whether their money moved units off their lot — it has no access to that dealer's system of record and no comparable holdout.
So you need both, and only one is provided. The program report is a league table. The scorecard is your P&L. Nobody builds the second one as a byproduct of the first — closing that gap is worth asking your provider about directly.
What to ask for
If you want to move from a delivery summary to a scorecard, four asks:
Frequency distribution, not average frequency. What share of reached households hit the effective range?
A holdout or staggered-launch design to compare against. If your partner says it's not possible, ask why — harder, not impossible.
Reconciliation to your DMS/CRM, with the gap between platform-reported and system-of-record numbers shown, not smoothed over.
One sentence on what would get worse if you cut it in half. If nobody can write it, you've learned something useful.
None of these require a new platform. They require a partner willing to report a smaller, truer number instead of a larger, safer one.
Want to see what this looks like against your own numbers? Start a conversation →
Evaluating a CTV vendor right now?
Download the 15-Point CTV Vendor Field Guide — the scorecard multi-location operators use to tell a real streaming TV partner from a media buy with a logo on it.