How We Actually Measure Whether AI Search Is Working For You
AI search is the newest channel, which means it is also the one where the most nonsense gets sold. Plenty of people will happily charge you to "optimize for ChatGPT" and then report metrics that look impressive and mean nothing. Here is how we actually judge whether the work is paying off for a Winchester business — and which numbers we deliberately ignore.
The metric that matters most is simple: do you show up in the answer when a real person asks an AI a real buying question? So we test it the way your customer would. We run the prompts a Winchester homeowner or business owner would actually type — "who does emergency plumbing near Old Town Winchester," "best [your trade] in Frederick County VA," "I need [your service] near Stephens City" — across ChatGPT, Google's AI answers, Perplexity, and the rest. We track whether you appear, whether you are named accurately, and whether the AI links or points to you. That presence, checked on a repeating schedule, is the scoreboard.
The second thing we measure is the quality of the mention. Being named is good; being named correctly is what wins the job. We watch for whether the AI describes what you actually do, cites your correct service area, and pulls your real phone number and hours rather than stale data from an old listing. A confident but wrong AI answer costs you customers, so we treat accuracy as a metric, not an afterthought — and when the AI gets it wrong, that tells us exactly which source data to go fix.
The third measure is the one your accountant cares about: referral traffic and leads that trace back to AI tools. As these platforms increasingly send clicks, we watch your analytics for visits arriving from ChatGPT, Perplexity, and AI-powered search, and we tie those to form fills and calls wherever the tracking allows. It is early-stage and the volume is still smaller than classic Google, but the trend line is the signal — and it is climbing.
Now the vanity metrics we ignore. We do not report "AI visibility scores" from tools that invent a number with no methodology behind it. We do not celebrate you appearing for prompts no human would ever type. We do not count a mention in an answer to "list marketing agencies in Virginia" as a win when nobody buying your service asks that. And we flatly refuse to chase impression counts on platforms that do not even expose reliable impression data — reporting a made-up number is worse than reporting none.
- What we track: real-prompt presence across AI engines, accuracy of what the AI says about you, and measurable referral traffic and leads from AI sources.
The honest picture is that AI search is a growing slice, not the whole pie yet — so we treat it as one measured channel alongside your core SEO, not a magic replacement. Measured that way, you always know whether it is earning its place. If you want to see how you currently show up when the AI is asked about your trade in Winchester, get in touch and we will run the prompts and show you.