How we measure whether AI Search is actually working for you
AI Search is the newest thing we do, which makes it the easiest thing for an agency to fake with impressive-sounding numbers that mean nothing. So before we start, it is worth being direct about how we measure whether ChatGPT, Google's AI Overviews, Perplexity, and the rest are actually sending qualified Harrisonburg customers your way — and which numbers we deliberately ignore because they only exist to look good on a report.
The metric that matters most is straightforward: when someone in the Shenandoah Valley asks an AI assistant a real buying question — "who does emergency plumbing in Harrisonburg," "best roofer near JMU," "drain cleaning in Rockingham County" — does your business get named, and is the information about you correct? We test this on a recurring cadence across the major AI tools, log whether you appear, and track whether that presence is trending up. Getting named in the answer is the whole game. Being absent from it is the problem we are solving.
The second real metric is the quality and accuracy of what the AI says about you when it does mention you. It is not enough to be cited. If an assistant recommends you but lists your old hours, the wrong service area, or a phone number from two locations ago, that citation costs you the call. So we monitor the substance, not just the mention, and we shape the source material — your site content, your structured data, your profile — so the AI has clean, current, unambiguous facts to pull from.
The third metric is the one that pays your bills: referral traffic and conversions that originate from AI tools. We tag and watch the visitors arriving from these assistants and, more importantly, whether they call, fill out a form, or book. A hundred AI mentions that never turn into a Harrisonburg homeowner picking up the phone is a vanity result. We are tracking the path all the way to a real lead.
Here is what we deliberately do not report as a win:
- Raw "AI impressions" or made-up visibility scores with no tie to a real question a customer would ask
- Being mentioned for irrelevant or out-of-area queries that will never produce a Rockingham County customer
- One-time appearances that do not hold up when we re-test the same prompt a week later
- Any single-number "AI score" presented without the underlying prompts and results you can check yourself
The reason this discipline matters is that AI Search moves fast and the temptation to dazzle with unverifiable metrics is real. We would rather show you the actual prompts we tested, the actual answers you appeared in, and the actual leads that followed — even when the honest number is smaller than a flashier one — than hand you a dashboard that looks great and means nothing. If it is not moving real Harrisonburg customers toward you, we are not going to pretend it counts.