How we tell whether AI search is actually working for you
AI search is new enough that a lot of "measurement" is theater. The honest problem is that ChatGPT, Google's AI overviews, and the rest don't hand out a rankings dashboard the way classic search did. So the temptation is to report numbers that look impressive and mean nothing. We'd rather tell you what we watch and, just as important, what we deliberately ignore.
What we actually track starts with the question that matters: when someone asks an AI "who's a good roofer in Mechanicsville?" or "best HVAC company near Atlee," does your business get named? We run those real prompts on a schedule across the major tools, log whether you appear, in what position on the list, and how the AI describes you. Movement from "not mentioned" to "mentioned" to "mentioned first" is the whole game. It is slower and less flashy than a traffic chart, but it is the thing that puts you on the short list a customer never scrolls past.
The second real signal is accuracy. AI tools that mention you will also characterize you — your services, your area, sometimes your hours. When they get it right, that came from clear, consistent information we made easy to find. When they get it wrong or hedge, that tells us exactly what to shore up. Watching the description improve over time is a genuine measure of progress.
Third, we watch referral traffic and calls that trace back to AI surfaces as those sources become more visible in analytics. It is early and imperfect, but a rising trickle of "how did you hear about us" answers that point to an AI assistant is real evidence, not a vanity chart.
Here is what we refuse to sell you as a win:
- Raw "impressions" on AI overviews, generic organic traffic we pin on AI without a traceable path, or a single lucky prompt cherry-picked once and screenshotted forever — none of those prove a Mechanicsville customer found you and called.
The reason the honest version matters here specifically: because Mechanicsville is wired so tightly to Richmond, AI tools asked about the town frequently default to metro businesses and skip the genuinely local ones. So a vanity metric can look fine while the AI is quietly recommending a company inside the city line instead of you. Measuring the right thing — are you named, are you named accurately, for the Hanover-area prompts your customers really use — is the only way to know you are winning the ground that actually sends you work. Everything else is a number that feels good on a report and does nothing in the parking lot at Rutland Commons.