How we actually measure whether AI search is working
AI search is new enough that the industry is full of people selling it with numbers that sound impressive and mean nothing. Since you cannot log into ChatGPT and see a dashboard of where you rank, measurement here is genuinely harder than regular SEO — so it matters even more that we agree up front on what counts as working and what is just noise. Here is exactly how I judge it for a Hillsville business, and what I refuse to bill you excitement over.
The real measure is simple and it is a test we can both run: does the tool name you? I take the actual questions your customers ask — "who's a good mason near Hillsville, Virginia," "emergency tree service Carroll County," "best plumber near Fancy Gap" — and I ask them across ChatGPT, Google's AI answers, and the other assistants, on a schedule. Then I track three things over time: whether you get mentioned at all, whether you show up in the first handful of names instead of buried, and whether the details it repeats about you — your service, your area, your phone — are correct. A business that goes from never mentioned to reliably named in the answer is the whole win, and it is something you can verify yourself.
Underneath that, I watch the inputs these systems actually pull from, because those are the levers we control. Is your structured data clean and complete so the machine can parse exactly what you do and where. Is your Google profile and your review base strong, since the assistants lean on that trust the same way people do. Is your site written in plain, direct answers to real questions rather than marketing fog. When those inputs improve, the mentions follow — so tracking them tells us the work is landing before the answers even catch up.
Now the vanity metrics I ignore, because you should not pay attention to them either:
- Raw "AI traffic" numbers with no context — a spike in visits means nothing if the tool is describing you wrong or sending people who were never going to call; I care whether you are named correctly, not whether a counter went up.
- Screenshots of a single good answer, cherry-picked on a lucky day — these systems vary their responses, so one flattering result proves nothing without the repeat testing that shows a real pattern.
- A "visibility score" from some tool that grades you against the whole internet — in a rural market like Carroll County the only question that matters is whether you win the specific local questions your actual customers ask, not an abstract number.
The honest part is that this is early, the tools shift monthly, and I will tell you plainly when a result is a genuine gain versus a fluke. Because AI search runs on the same foundation, most of the measurable progress shows up in your local SEO too, which means your money is rarely wasted even while the AI side matures. Tell me the questions your customers ask and I will show you where you stand today in a written proposal.