How we measure whether AI search is actually working
AI search is new enough that a lot of people measuring it are measuring the wrong things. Since almost no Blacksburg competitor is even trying yet, the temptation is to celebrate anything that looks like motion. We do not. Here is what we actually track, and just as importantly, what we ignore.
The real metric is presence in answers. We run the questions your Blacksburg customers actually ask — "best breakfast in downtown Blacksburg," "who fixes furnaces near Virginia Tech," "a trustworthy plumber in Blacksburg" — across ChatGPT, Google's AI Overviews, and the assistants people use, and we log whether your business gets named, how it is described, and whether the facts are right. That last part matters: an AI that recommends you but lists a wrong phone number or old hours is a leak, not a win. We track corrections landing as carefully as we track mentions appearing.
We also watch the sources these systems lean on. AI answers pull from a specific set of places it trusts — your own site, your Google Business Profile, established directories, and local mentions. So we measure whether your facts are clean and consistent across those sources, because that consistency is the lever that actually moves whether an AI is willing to name you. When your presence on the trusted sources improves, the answers follow, and we can show you the before-and-after in the AI's own words.
The honest complication is attribution. AI referral traffic is still messy to track — a lot of people ask an AI, get your name, and then call you or type your name into Google directly, so the credit never shows up cleanly in analytics. Rather than pretend a dashboard number captures it, we watch the leading indicators we can verify and we ask your new customers how they found you. In a market this AI-native, "ChatGPT told me about you" is already a real answer, and it is worth counting even when the software cannot.
Here are the vanity metrics we refuse to sell as results:
- Raw counts of AI mentions with no check on whether the facts are correct or the description is flattering
- Generic AI traffic in analytics that cannot be tied to anything you can act on
- One-off screenshots of a good answer, cherry-picked and never re-tested to see if it holds
- Any promise of a guaranteed spot, since no one controls what these models say from one update to the next
What you get instead is a plain report a few times through the engagement: the exact questions we tested, what the AI said about you before, what it says now, and what we changed to move it. It is a newer discipline with fewer guarantees than classic SEO, and we would rather tell you that than dress up a number. If you want to see what AI says about your Blacksburg business today, an AI-search audit is where we start.