How we measure whether AI search is actually working for you
AI search is new enough that a lot of what gets sold around it is smoke. The fair question from any Poquoson owner is: how do we know it's working, and how do I keep you honest about it? Here's exactly what we track — and, just as important, what we ignore.
The real metric is simple: are you the business named when someone asks an AI a buying question about Poquoson? We test it directly. On a set schedule we ask ChatGPT, Google's AI overviews, and the assistants your neighbors actually use the questions a real customer would type — "who's a good [your trade] in Poquoson," "best [service] near Poquoson VA," the emergency and comparison phrasings — and we record whether you show up, where you rank in the answer, and how accurately you're described. That transcript over time is the scoreboard. Being named for three relevant questions this month versus zero last month is a real, checkable result.
We also watch whether the AIs describe you correctly — right services, right area, right phone number for 23662, right hours. An AI that recommends you but sends the customer to a wrong number or a closed-down detail is a bug we fix, and it's the kind of thing a vanity-metrics approach never even checks.
Then there's the outcome that pays your bills: calls and form fills from people who found you through an AI answer. We ask new customers how they found you, we watch for the traffic and call patterns that follow an AI mention, and we treat a real inquiry as worth more than any dashboard number. In a market where a huge share of residents are engineers and base personnel who already ask an AI by default, those referrals are early but real, and they're the point.
Here's what we deliberately ignore. We don't chase raw "impressions" or how many times your name theoretically appeared, because in AI answers that's unmeasurable and unfalsifiable — a perfect vanity metric. We ignore follower counts and generic "reach" numbers that have nothing to do with whether an AI recommends you. We don't report on being cited for questions no customer would ever ask. And we don't count a mention buried below three competitors as a win.
- Tracked: whether you're named, and how high, for real buying questions in Poquoson
- Tracked: accuracy of your listing in AI answers — services, area, phone, hours
- Tracked: actual calls and inquiries that trace back to AI discovery
- Ignored: impressions, reach, follower counts, and citations for questions nobody asks
The whole approach only works because it's checkable. You get the actual question-and-answer transcripts, not a made-up score, so you can judge for yourself whether AI tools are starting to point Poquoson customers your way. If you want to see how the AIs answer for your business right now, that's the first thing I'll send back.