How we measure whether AI search is actually working for you
AI search is new enough that a lot of agencies measure it with numbers that look impressive and mean nothing. We'd rather tell you what we track, what we ignore, and why — because if you can't measure it, you're paying on faith, and this brand doesn't run on faith.
What we actually watch is whether AI assistants recommend you when a real Chantilly customer asks a real buying question. We run the prompts your customers would run — "who's the best [your service] near Chantilly VA," "reliable [your trade] around the Dulles corridor," "[your service] open now near Route 50" — across the assistants people here actually use, and we log whether you're named, how you're described, and who's named alongside you. That last part matters: knowing you appear next to two specific competitors tells us exactly who to out-position.
We also track the sources those assistants lean on when they answer. AI tools pull from places they trust — your Google profile, established directories, review sites, and the pages on your own site that answer questions plainly. When your name starts showing up in those source pockets, recommendations follow. Watching the sources is often an earlier, more honest signal than the recommendation itself, because it moves before the visible answer does.
And we tie every bit of this back to whether the phone actually rings. An AI mention that never turns into a Chantilly call or a filled-out form is a trophy, not a result. So alongside the prompt testing, we watch the leading indicators that matter — direction requests and calls on your Google profile, and whether the assistants describe you accurately enough that a customer who acts on the recommendation reaches the right business. Being recommended but listed with the wrong hours or the wrong service is worse than not being named at all.
Here's what we deliberately ignore. We don't report "AI impressions" or invented reach numbers — no assistant hands out reliable impression counts, so anyone quoting them is guessing and dressing it up. We ignore a single lucky mention that doesn't repeat when we run the prompt again, because a one-off isn't a ranking, it's noise. And we don't celebrate being mentioned in an answer to a question no Chantilly customer would ever ask. The vanity metrics we skip:
- "AI impressions" or reach figures no assistant actually publishes.
- One-off mentions that vanish when the same prompt is re-run.
- Getting named for informational questions that don't lead to a call or a job.
- Being cited on national queries with no Chantilly or Northern Virginia intent.
The scoreboard that counts is narrower and harder to fake: for the buying questions your Chantilly customers ask, are you recommended, is that recommendation stable when we check again, and is your business described accurately when it appears? We report those plainly, month over month, with the exact prompts we tested so you can run them yourself. It's the same discipline behind our core SEO work — measure what turns into customers, and quietly throw out the rest.