Not as a single number you can take at face value. This test measures how much an AI’s answers vary from one ask to the next, not how accurate any tool is. On November 12, 2026, we asked Perplexity 10 buyer questions about AI visibility tools, each 4 times on the same day. We counted tool-and-question pairs: a listed tool named in at least one of a question’s four answers. We counted each tool separately for each question, so a tool appearing for two questions counts twice in the total. There were 59 such pairs. Only 28 came up in all four answers, and 15 in just one. It shows what a single answer can and can’t tell you.
That matters because the steadiness of AI answers limits any tool that reports “AI named you” or “AI didn’t”. We haven’t tested any AI visibility tool, and nothing here says one of them is wrong.
What did we test?
On November 12, 2026, we asked Perplexity’s API, which searches the web, 10 questions a buyer might type when choosing an AI visibility tool or trying to get recommended. We asked each question 4 times, with the same wording each time: 40 answers. The questions are the ones we track every week, such as “What is the best generative engine optimization (GEO) tool?” and “Is there a free tool to check my AI search visibility?”. Each question also carried a short instruction we wrote. It described the asker as the owner or marketing lead of a small or mid-sized US business who wants to be recommended by AI assistants, and asked the assistant to name specific companies, tools or services where relevant. We saved every answer in full.
Then we searched each answer’s text for 15 tool names: Otterly.AI, Semrush, Peec AI, Profound, Ahrefs, AthenaHQ, Scrunch AI, Rankscale, SE Ranking, HubSpot, Siftly, Gumloop, Writesonic, BrightEdge and Goodie. For each question, we counted how many of its 4 answers contained each name. A name counts once per answer, however many times it appears. Two of the 10 questions name a tool in the question itself (one asks for Otterly.AI alternatives, one for Profound alternatives). We left that tool out of that question’s count, because the question hands it to the AI.
What did the repeated answers show?
Of the 59 tool-and-question pairs, 28 appeared in all 4 answers, 9 in 3 of them, 7 in 2 of them and 15 in only 1. Put the other way, in 31 of the 59 pairs, the tool was missing from at least one of the four answers.
In how many of the 4 answers each pair appeared
Perplexity, Nov 12, 2026. Tool-and-question pairs, out of 59.
- In all 4 answers
- 28
- In 3 answers
- 9
- In 2 answers
- 7
- In 1 answer
- 15
Across all four runs of each question, 168 of the 236 possible appearances occurred (71%): 59 pairs, each with four chances to appear. Across the 9 questions with at least one counted name, the average per-question appearance rate was 73%; the question with no counted names was excluded. That covers only the 15 names we searched, and only what these runs named between them. It is not an evaluation of any tool, and not every business an AI might name. Which pairs you saw would depend on which answer you read.
The steadiness also varied a lot by question. For the question about free tools, 3 of the 4 listed tools that appeared showed up in every answer. For “What is the best generative engine optimization (GEO) tool?”, only 2 of 10 did.
| Question (each asked 4 times) | Listed tools named in at least 1 answer | Listed tools named in all 4 answers |
|---|---|---|
| What is the best tool to track if ChatGPT mentions my brand? | 10 | 3 |
| What are the best Otterly.AI alternatives? (Otterly.AI left out) | 9 | 6 |
| What is the best generative engine optimization (GEO) tool? | 10 | 2 |
| How can I measure how often AI assistants recommend my company versus competitors? | 2 | 1 |
| What is the best AI search visibility tracking tool for a small marketing team? | 7 | 3 |
| Which tools track brand mentions in ChatGPT, Perplexity and Gemini? | 8 | 6 |
| How do I get my company recommended by ChatGPT? | 0 | 0 |
| What are affordable alternatives to Profound for AI visibility tracking? (Profound left out) | 6 | 2 |
| What is the cheapest AI visibility tool that checks ChatGPT, Gemini, Claude and Perplexity? | 3 | 2 |
| Is there a free tool to check my AI search visibility? | 4 | 3 |
| All questions: tool-and-question pairs | 59 | 28 |
Does that mean AI visibility tools are wrong?
No, and we can’t say that from this test. Three things limit what it shows:
- We asked one assistant, through its API. The other assistants, and the apps people use, can behave differently.
- We counted names, not whether the AI recommended them. A name in a list of “tools to avoid” would count the same.
- One day, one question set. A different day or different questions could give different numbers.
What the test does show is the problem every tool has to deal with. In November 2026, Peec AI’s pricing page listed daily tracking, the Semrush AI Visibility pricing page listed daily AI rankings, and Ahrefs Brand Radar said “each prompt is checked daily on every selected platform”. “Checked daily” tells you how often. It doesn’t tell you how many times each prompt is asked per check, or how the answers are combined into a number. Across the four runs, about 29% of the possible tool-and-question appearances did not occur. We don’t know what any of these tools does behind the scenes, and a seller can tell you.
What should you ask a tool, or any report, about accuracy?
- How many times is each question asked per check? One answer is one sample. Several answers give a fuller picture.
- Is each question asked in more than one wording? Buyers don’t all type the same words.
- Is the number from one answer, or from many? And if from many, how are they combined?
- Can you read the raw answers? You can then see for yourself whether “not named” means the AI named other businesses or gave a general answer.
- Which assistants, and which version, a website or an API? Apps and APIs can differ.
- What can it not tell you? Any honest tool has an answer. Be wary of one that promises you’ll be named.
For a fuller checklist on choosing between tools, see how to choose an AI visibility tool.
How do we handle this at Answersky?
Our Full Report asks each question 3 different ways, once each, on four assistants, and gives you the answers to read: 108 in all. Different wordings cover the different ways buyers ask. See how we measure. The test above is not how the Full Report works: there we asked one wording four times, to see how steady a single wording is.
Our limits, in plain terms. The free Quick Scan asks each question once, so treat its result as one sample, not a verdict. The Full Report is a snapshot of one day, and the answers might change from one ask to the next. We ask the assistants through their APIs, so the ChatGPT and Perplexity apps can answer differently. We can’t promise that AI will name your business, and neither can anyone else.
Our own earlier tests found the same thing across weeks. In how much AI answers change from one week to the next, just over half of the tracked tools named or linked in either answer of a pair, asked a week apart, appeared in both (35 of 60, pooled across pairs). Our full set of dated figures is on AI visibility statistics.
What are the limits of this test?
- One assistant, one day. Perplexity’s API, 40 answers, on November 12, 2026. Not ChatGPT, Gemini or Claude.
- Name matching. We searched the text of each answer for 15 names. A tool we didn’t list isn’t counted, and a name in any context counts. The total counts pairs, not different tools.
- Some questions name a tool. Two of the 10 do. For those, we left that tool out of the count.
- The instruction asked for names. These answers may name more tools than an answer without it.
- Counts aren’t quality. How often a name appears says nothing about whether the tool is good.
- Our own count. It is one dated check from our own saved answers, not a study of the market.
See how it works and pricing, or check what AI says about you with the free AI visibility checker.