Guides

Does ChatGPT recommend your business?

Nobody can tell you with certainty what the ChatGPT app said to a particular person this morning. Here is what can be measured instead, and why it is still worth knowing.

Why the exact question cannot be answered

The consumer apps are not measurable from the outside. Answers vary by account, by location, by memory of past chats and by which model happens to serve the request. Scraping them is against their terms, and a tool that did it would be giving you numbers it had no right to collect and no way to reproduce. ReplyScore does not scrape consumer AI apps and does not scrape Google results pages.

What is measured instead

Questions are asked through official APIs: Anthropic, and Perplexity Sonar. Those are the same underlying models that answer in the products people use, asked the same questions a customer would ask, without your business name in them. Google AI Overviews are checked separately through DataForSEO when that check is switched on, and an overview that is not shown is recorded as exactly that rather than folded into the other numbers.

Why an API answer is still informative

An API answer is not a screenshot of somebody's phone, and it is not claimed to be. What it does show is whether a model, asked plainly who provides this service in this place, has anything to say about you. If it does not, that is consistent across accounts and locations in a way a personalised app answer never is. The signal is weaker than a promise and far stronger than a guess.

What a yes or a no is worth

A mention on one question on one day is a single observation. A pattern across twenty unbranded questions is worth paying attention to. Neither is a ranking, because there is no ranking to report: assistants do not publish positions, and a tool that hands you a position number for ChatGPT has invented it. The report states which engines were asked, on what date, with what questions, and what came back.

What to be suspicious of

Be suspicious of any tool that will not name the assistants it tests. Be suspicious of a score with no methodology page behind it. Be suspicious of a percentage uplift promised in advance, because the model, your competitors and the rest of the web all change between two measurements, and nobody can hold those still. If a vendor cannot tell you what it asked and what came back, it is selling confidence rather than measurement.