How to Choose an AI Visibility Tool: 10-Question Checklist
Choosing an AI visibility tool? Ten questions on ranges, noise rules, readable answers, engines and question sets, plus a 0 to 2 score sheet.
- ai visibility
- buying guide
- geo tools
- measurement
Before you buy an AI visibility tool, ask whether its numbers survive the fact that AI answers change every run: does each score show a range, is there a written rule for real change, and can you read every answer? SparkToro found a less than 1 in 100 chance that ChatGPT repeats a brand list, so a tool without those answers sells you noise. (For a side-by-side of named tools, see AI visibility tools compared; this checklist is about how to judge any of them.)
Why does the choice of tool decide what you will believe?
An AI visibility tool does not just report your visibility, it decides which movements you will treat as real. Two tools fed the same questions can disagree sharply. Paul Dyer, CEO of the agency /prompt, told Digiday (1 May 2026): "If you use three different tools and give them the same prompts, you get three different answers."
Picture an agency that pitches a client on a 12-point gain after the first month. In month two the gain is gone, and nothing the agency did explains either move. The client does not blame the AI; they blame the agency. (Invented example.) The checklist below is built to stop that conversation before it happens. Nodpick is the AI visibility tracker we build, so we hold it to the same list: the last column shows its answers, each taken from a public page.
Rand Fishkin of SparkToro, after 600 volunteers ran 12 prompts a combined 2,961 times (published 27 January 2026; the study was run with Gumshoe.ai, an AI-tracking startup, which Fishkin names as a possible conflict of interest), put the demand bluntly: "stop throwing money at AI tracking products that don't provide stats-backed, publicly-reviewable research." Here is what that means as questions you can ask in a demo.
Which 10 questions should you ask an AI visibility tool?
These ten questions test whether an AI visibility tool measures your brand or just samples noise. Each comes from a documented problem with AI answers, and each has an answer you can check on the vendor's own pages before you pay. The last column shows how Nodpick answers, taken from its published method and pricing.
| # | Ask the vendor | Why it matters | Nodpick's published answer |
|---|---|---|---|
| 1 | Does every score show a range and the number of answers behind it? | One answer is a draw, not a reading | Every score with a 95 % range and n, for example "42 (range 31 to 53, n=60)" |
| 2 | What exact rule makes a change "real", and what happens when my mention flips? | Without a rule, every bounce looks like news | Four published conditions: enough data, paired test at 1 %, at least 5 points, more than twice the normal swing; on a flip it asks again right away and counts both answers |
| 3 | Does it tell "recommended" from "just mentioned"? | A name in a list is not a pick | Every answer that names you is sorted into recommended, just mentioned or negative; citing your site is counted separately; recommendation rate shown with a 95 % range |
| 4 | Can I read every answer and the sources it cites? | A number you cannot audit is a claim | Every answer one click away, with its cited sources and its group (recommended, just mentioned or negative) |
| 5 | Which engines are in the base price? | Add-on engines change the real cost | Same 6 core engines on every plan: ChatGPT, Google AI Overviews, Google AI Mode, Gemini, Perplexity, Claude. Microsoft Copilot is not measured |
| 6 | How many different questions, and how many answers a week? | Breadth cuts noise more than reruns | 25, 70 or 220 questions in total; 150, 420 or 2,200 answers a week |
| 7 | Do the tracked questions name my brand? | Branded or hand-picked questions flatter you | Scored questions never name your brand; one branded question is kept out of the score |
| 8 | Does it ask in my buyers' country and language? | Local answers name local providers | 7 languages, 16 European markets plus the US |
| 9 | What happens when Google shows no AI Overview? | Counting a missing box as zero punishes you | Questions without one are not counted; coverage is shown |
| 10 | Does every recommendation say how well it is proven? | Advice without evidence is guesswork | Grades A (documented), B (studied), C (untested), each with a source |
Questions 1 to 4 decide whether you can trust and read the number. Questions 5 to 8 decide whether the number covers your buyers. Questions 9 and 10 decide whether the number leads to sensible work. The next sections explain the three that most buyers skip.
Why should a tool show a range instead of a single score?
A single AI visibility score without a range hides how much of it is luck. Sistrix, analysing 10,826 daily measurement series (published 27 October 2026), found a typical deviation of ±34 points from the true value with one prompt, ±9.5 with 20 prompts and ±4.2 with 100. In Sistrix's own example, a brand truly named in 25 % of answers measures anywhere between 16 and 35 % with 20 prompts on one day.
A tool that prints a bare number invites you to act on a 4-point move (our post on why scores jump around shows how often that happens with nothing changing). A tool that prints the range next to it tells you the move sits inside the noise. Nodpick's method page explains how its range is built: questions vary as a whole, because engines answering the same question tend to move together, so the range does not look tighter than it is. One plain explanation like that on a method page tells you more about a tool's honesty than its dashboard design.
If ranges answer "how sure is this number?", the next question is "who picked the questions?".
Why do hand-picked questions flatter your brand?
Questions you choose yourself drift toward topics where your brand already appears, so they overstate your visibility against competitors. Sistrix compared brands' own prompt sets with neutral ones: the own brand scored better in three of four cases, "63 bis 80 Prozent" depending on the competitors compared, where chance would give about half. Sistrix calls it "kein Messfehler, sondern ein Auswahleffekt" (not a measurement error but a selection effect).
What to ask: who writes the questions, do any name your brand, and do they cover the category, the problem, alternatives and comparisons? In Nodpick this is the first setup step: it suggests questions from your category, country and language across exactly those four angles, you edit them or write your own, and it never puts your brand name into a scored question. For a full method of building the set, see the guide on tracking AI visibility and the post on tracking brand mentions in ChatGPT.
A clean, wide question set still leaves the hardest buyer question open: what do I do with the result?
Does the tool tell you what to do, and how sure that advice is?
Most GEO advice has never been tested for your kind of brand, and a good AI visibility tool says so. Ryan Mason of Markacy told Digiday: "It's just a benchmarker in my mind." The fix for that complaint is not more advice. It is advice that carries its evidence.
| Evidence grade | Meaning | Example source |
|---|---|---|
| A: documented | The platform documents it | OpenAI and Google crawler and AI features documentation |
| B: studied | Independent research found an effect | GEO study by Aggarwal et al. (first version 16 November 2023), which reports visibility gains "by up to 40%" in generative engine responses |
| C: untested | Neither documentation nor a study shows an effect | For example llms.txt, where no study shows an effect yet |
Nodpick attaches one of these grades to every recommendation and links the source. Only an A or B fix can lead a report as the biggest fix, so an untested trick never tops your to-do list. The method page lists every grade and source.
One limit applies to every tool on this list: monitoring shows what assistants say, it does not change the answer. Answer monitoring and referral analytics are also separate signals. A tool that reads answers cannot see visits, and your analytics cannot see the answers that nobody clicked, so plan to use both and do not expect either to stand in for the other.
How do you choose between AI visibility tools? A scoring sheet
Score each tool 0, 1 or 2 on the ten questions: 0 when the vendor cannot answer, 1 when the answer is only verbal or behind a demo, 2 when it is written on a public page you can link. This is our own heuristic, not an industry standard, but it rewards the thing that matters most: claims you can check.
A tool scoring under 10 of 20 is likely to give you numbers you cannot defend to a client or a manager. A tool at 16 or more gives you numbers, the answers behind them and a reason for every fix. One more check before you sign: compare the plan's credits or limits with the number of answers you will actually review. A plan that covers 2,200 answers a week is wasted if your team reads 150, and a plan that covers 150 is too small for a 70-question set. Multiply your questions by the engines you need, then pick the plan whose limit matches that number. Run the same scoring on Nodpick using its published method and its plans; every answer in the table above comes from a page you can read.
The common objection is "can't I just ask ChatGPT myself?" You can, and for a first look it is useful. The cost shows up in week three: a 20-question set on 6 engines is 120 answers a week to log, compare question by question and test for noise, before you have read a single source. The other objection, "is any of this worth paying for if clicks from AI are small?", has a data answer in whether GEO is worth it: click numbers miss the readers who see your name and decide without a visit.
Nodpick is built to answer all ten questions with a link, not a promise: ranges on every score, a written noise rule, every answer readable, six engines on every plan, and evidence grades on every fix.
Check your own brand
A good way to test the list is on your own brand. The free AI visibility checker asks ChatGPT, Perplexity, Gemini and Claude five buyer questions written from your website and shows whether they name you, who they name instead and which sources they cite, with a range around the score. It opens on launch day, and you can join the waitlist on that page to get one email when Nodpick opens. Plans for the weekly version are on the pricing page.
Common follow-up questions
Is a free AI visibility checker enough to choose a tool?
A free check is a useful first look, but it usually asks each question once, so its ranges are wide. Nodpick's free check asks 5 scored buyer questions plus 1 about your brand, on ChatGPT, Perplexity, Gemini and Claude, with no card, and marks its ranges as early samples. Use a free check to decide whether to track, not to judge a change.
Should an AI visibility tool track prompts daily or weekly?
Not for the visibility level. Sistrix found that 20 prompts tracked over seven days reach ±7.9 points, barely different from a single day at ±9.5, while 140 different prompts on one day reach ±3.6. Daily runs help spot sudden events; more questions give a truer level.
Can an AI visibility tool make ChatGPT recommend me?
No, and no tool can. A tool can show where you stand, who is named instead, which sources the assistants cite and which fixes have the strongest evidence. Whether an assistant names you depends on what the web says about your brand.
Can you compare AI visibility scores between tools?
No. Tools differ in question sets, engines, countries, formulas and how they treat missing answers, so the same brand gets different scores. Compare a tool's score only with itself over time, and use the ten questions above to judge which tool's trend you can trust.
Sources
- SparkToro, "NEW Research: AIs are highly inconsistent when recommending brands or products" (27 January 2026), https://sparktoro.com/blog/new-research-ais-are-highly-inconsistent-when-recommending-brands-or-products-marketers-should-take-care-when-tracking-ai-visibility/, Last checked: 1 November 2026
- Sistrix, "Breite schlägt Wiederholung: Wie man AI-Sichtbarkeit verlässlich misst" (27 October 2026), https://www.sistrix.de/news/breite-schlaegt-wiederholung-wie-man-ai-sichtbarkeit-verlaesslich-misst/, Last checked: 1 November 2026
- Digiday, "Marketers question expensive AI visibility tools as inconsistent results fuel skepticism" (1 May 2026), https://digiday.com/marketing/marketers-question-expensive-ai-visibility-tools-as-inconsistent-results-fuel-skepticism/, Last checked: 1 November 2026
- Aggarwal et al., "GEO: Generative Engine Optimization", arXiv:2311.09735, https://arxiv.org/abs/2311.09735, Last checked: 1 November 2026
- Nodpick, How we measure AI visibility, https://nodpick.com/method, Last checked: 1 November 2026
- Nodpick, Pricing (plans and FAQ), https://nodpick.com/pricing, Last checked: 1 November 2026
- Nodpick, Home page, https://nodpick.com/, Last checked: 1 November 2026
Checked 1 November 2026.