How we measure whether AI search is actually working in Galax
AI search is new enough that a lot of people selling it lean on numbers that look impressive and mean nothing. Before I take on this work for a Galax business, I tell you exactly what I will measure and what I will ignore — because the whole point is knowing whether it moved the needle, not admiring a chart. The measurement that matters is simple to state: when someone asks ChatGPT, Google's AI Overviews, or Perplexity a real question your customer would ask — "where should I eat before the Old Fiddlers' Convention," "who does HVAC near Galax," "best coffee in downtown Galax" — does the answer name you? That is the scoreboard. Everything else is supporting evidence.
So we start by writing down the actual questions. Not keywords — questions, in the words a Fiddlers' Convention visitor or a Grayson County homeowner would really type. Then I test each one across the major assistants and record the starting truth, which is usually humbling: for most Galax businesses, the AI either recommends a competitor, names a chain, or gets your details wrong. That baseline is the honest before-picture. From there, progress is measured the same way, on the same questions, over time — did we go from unmentioned, to mentioned, to recommended by name with the right phone number and hours attached.
Here is what we track, and what we throw out:
- We track — whether you are named in the answer, whether you are recommended versus just listed, whether the details the AI states about you are correct, how often you appear across the set of real questions, and whether Google's AI Overview cites a page from your site. We ignore — raw "AI visibility scores" from dashboards, generic impression counts, and any single-number grade that cannot be traced back to a specific question a real Galax customer would ask.
The reason the vanity metrics get ignored is that they cannot be verified and they do not correspond to a customer decision. A dashboard telling you your "AI presence is 72" is unfalsifiable. An assistant naming your barbecue joint when someone asks where to eat downtown is a customer walking through your door. The second one you can check yourself, in thirty seconds, on your own phone — and that is exactly the test I want you able to run.
What makes this genuinely measurable in Galax rather than guesswork is the size of the market. There are only so many real questions a visitor or a local would ask about your category here, so the full list is knowable and testable — unlike a big city where the question space is endless. We can cover it, watch it move, and know for certain whether the work landed. If you want to see your real starting point, the AI search overview explains the approach, and when you are ready I will test what the assistants say about your category in Galax right now and show you the honest baseline before we change a thing.