seo & ai search · playbook
LLM SEO checker: how to track your visibility in ChatGPT and AI Overviews
How an LLM SEO checker works: what to measure in ChatGPT and AI Overviews, a free manual method, the tool categories worth paying for, and the limits of any tracker.
Key takeaways
- An LLM SEO checker runs a fixed set of prompts through AI assistants and records whether your brand is mentioned, whether your pages are cited, and how you compare with competitors.
- Track four numbers per platform: mention rate, citation rate, share of voice and the sources cited instead of you.
- Search Console counts AI Overviews and AI Mode inside normal Web search data and adds an impressions-only Generative AI report; Bing Webmaster Tools has a dedicated AI Performance report for Copilot citations.
- AI answers vary between runs and users, so sample each prompt several times and read trends over weeks, not single checks.
- Start with a free manual check of 20 to 30 prompts, then buy a tool only when you need daily tracking, more platforms or competitor benchmarks.
An LLM SEO checker measures how often AI assistants such as ChatGPT, Google AI Overviews, AI Mode, Perplexity, Gemini and Copilot mention your brand or cite your pages when people ask questions in your category. It works by running a fixed list of prompts, capturing the answers, and counting mentions, citations and competitor appearances over time. You can do this manually with a spreadsheet, or use a tracking tool such as Ahrefs Brand Radar or the Semrush AI Visibility Toolkit when you need scale. No tool can see every user’s answer, so treat every number as a sample, not a rank.
What does an LLM SEO checker actually measure?
Classic rank tracking asks “what position is my URL for this keyword?” AI answers have no stable positions, so the useful metrics are different.
| Metric | Definition | Why it matters |
|---|---|---|
| Mention rate | % of tracked prompt runs where the answer names your brand | The closest thing to “ranking” in an AI answer |
| Citation rate | % of runs where one of your URLs appears as a source or link | Drives referral clicks and signals the model trusts your page |
| Share of voice | Your mentions divided by mentions of you plus named competitors, across the same prompts | Normalizes for prompt volume and shows relative position |
| Cited sources | Which domains and URLs the answers cite, for you and competitors | Tells you where to earn coverage (review sites, listicles, Reddit, docs) |
| Position in the answer | First recommendation, one of several, or a passing mention | A first-named mention is worth more than a footnote |
| Accuracy and sentiment | Whether the description, pricing and positioning are correct and favorable | Wrong facts in AI answers become sales objections |
Mentions and citations are different things. ChatGPT can recommend you without linking to your site, and AI Overviews can cite your page while recommending a competitor. Track both.
Which prompts should you track?
Your prompt set is the keyword list of AI visibility. A bad prompt set produces confident but useless numbers.
Build 20 to 50 prompts across these buckets:
| Bucket | Example prompt | Share of set |
|---|---|---|
| Category discovery | “What are the best product analytics tools for B2B SaaS?” | 30-40% |
| Problem or use case | “How do I track feature adoption in a SaaS app?” | 20-30% |
| Comparison | “Mixpanel vs Amplitude vs Acme for a 50-person startup” | 15-20% |
| Alternatives | “Alternatives to Mixpanel for B2B” | 10-15% |
| Brand facts | “How much does Acme cost?” “Does Acme support EU data residency?” | 10% |
Rules for the set:
- Write prompts the way buyers ask, as full questions with context (“for a 20-person team,” “under $500 a month”), not as two-word keywords.
- Source them from real questions: sales call notes, support tickets, Search Console queries phrased as questions, Reddit threads and People Also Ask boxes.
- Keep the wording frozen. If you change the prompt, you start a new time series.
- Tag each prompt by bucket, funnel stage and product line so you can see where you are weak.
- Add your Google queries separately. AI Overviews trigger on search queries, so track them as keywords, not as chat prompts.
How do you check your LLM visibility manually?
A manual check is free, takes two to three hours for 25 prompts, and is the best way to learn what the tools are measuring before you pay for one.
- Set up a clean environment. Use a logged-out or fresh account where possible, turn off memory and custom instructions in ChatGPT, and use a private browser window for Google. Note your country; results vary by location.
- Create a sheet with columns: date, platform, prompt, run number, brand mentioned (Y/N), position (1st, listed, passing), your URL cited (Y/N), competitors mentioned, domains cited, factual errors.
- Run each prompt three times per platform. Use a new chat each time. Answers vary between runs, and one run is anecdote.
- Check ChatGPT with search on. Answers that use web search show source links; those are your citations. Answers from model knowledge alone show what the model “believes” about you without retrieval.
- Check Google. Search the query, expand the AI Overview if one appears, and note the linked sources. Repeat in AI Mode.
- Check Perplexity, Gemini and Copilot for the prompts that matter most. Perplexity shows sources with its answers, which makes citation tracking easy.
- Compute the four core numbers per platform: mention rate, citation rate, share of voice versus your top three competitors, and the top 10 domains cited.
- Repeat monthly with identical prompts and conditions.
What to do with the results:
- Cited domains you don’t control (G2, Reddit, a “best tools” listicle) are your outreach and PR list. See the GEO playbook.
- Prompts where you are absent but rank in Google usually point to extractability problems. See the AEO / GEO content optimization guide.
- Prompts where you are absent everywhere mean you lack a page that answers the question.
- Factual errors need a clear, crawlable source of truth on your own site and corrected third-party listings.
What free data do Google, Bing and OpenAI give you?
Before buying anything, use the first-party data that exists. It does not show prompts, but it shows real traffic.
| Source | What it shows | What it does not show |
|---|---|---|
| Google Search Console | Clicks and impressions from AI Overviews and AI Mode, counted within the normal Performance report under the “Web” search type; a separate Generative AI performance report (available to all sites since August 31, 2026) isolates AI Overviews and AI Mode impressions by page, country, device and date | Clicks or queries for AI features alone; the Generative AI report is impressions-only |
| Bing Webmaster Tools AI Performance | Launched in public preview in February 2026: citation counts for your pages across Microsoft Copilot, Bing AI summaries and some partner integrations, cited pages, trends and sampled “grounding queries”; June 2026 preview additions include intents, topics, citation share and compare | Clicks from AI answers; ChatGPT and other non-Microsoft assistants |
| Google Analytics (or any analytics) | ChatGPT adds utm_source=chatgpt.com to referral links from ChatGPT search, per OpenAI’s publisher FAQ; Perplexity, Gemini and Copilot referrals show up by referrer domain | Answers where you were mentioned but not clicked |
| Server logs | Fetches by OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-User and other AI agents | Whether a fetch turned into a citation |
A simple GA4 setup: create a custom channel group or exploration filter that matches source values such as chatgpt.com, perplexity.ai, gemini.google.com and copilot.microsoft.com, and report AI referrals as their own channel.
What kinds of LLM SEO tools exist?
The market is young and changes monthly, so evaluate by category and check each vendor’s current platform coverage before buying. Verify pricing on vendor sites; it varies by prompt count, platforms and check frequency.
1. SEO suites with AI visibility modules
Best if you already pay for the suite and want AI data next to keyword and backlink data.
- Ahrefs Brand Radar tracks brand mentions and citations across AI platforms including AI Overviews, AI Mode, ChatGPT, Perplexity, Gemini and Copilot, with competitor benchmarking and custom prompt tracking. Its distinguishing feature is a large prebuilt index of prompts, so you can see visibility for prompts you didn’t think to add.
- Semrush AI Visibility Toolkit reports an AI Visibility Score, share of voice and sentiment in its Brand Performance report, prompt research with topic-level volume and difficulty, competitor research for prompts where rivals are cited and you are not, and daily prompt tracking on platforms including ChatGPT and Google AI Mode.
2. Dedicated AI visibility trackers
Best if AI search is a primary channel and you need daily tracking across many assistants, prompt-level history and URL-level citation reports. Examples include Profound, Peec AI and Otterly.AI. Coverage, refresh frequency and whether they query the consumer apps or the APIs differ by vendor; ask.
3. First-party webmaster reports
Free and based on real users rather than sampled prompts: Search Console (AI features inside Web search, plus the Generative AI report) and Bing Webmaster Tools AI Performance. Use these as your ground truth for traffic and citations on Google and Microsoft surfaces.
4. Readiness auditors
These don’t track answers; they check whether your site can be cited: crawler access in robots.txt, server-rendered content, structured data, answer-first structure and entity clarity. Run one when tracking shows low visibility and you need to know why. The technical checklist in GEO technical implementation covers the same ground manually.
How to choose
| You need | Choose |
|---|---|
| A baseline and a learning exercise | Manual check, free |
| AI data alongside existing SEO workflows | Suite module (Ahrefs, Semrush) |
| Daily, multi-assistant tracking with history | Dedicated tracker |
| Real traffic and citation counts | Search Console, Bing Webmaster Tools, analytics |
| To know why you aren’t cited | Readiness audit |
Questions to ask any vendor:
- Which assistants and which versions or modes (logged-out web app, API, search on or off)?
- How many runs per prompt per check, and how often?
- Which countries and languages?
- Do they report mentions and citations separately, and at URL level?
- Can you export raw answers, not just scores?
What are the limits of AI visibility tracking?
Every LLM SEO checker, including your own spreadsheet, has the same structural limits.
- Non-determinism. The same prompt can produce different brands, orders and sources from one run to the next. Single checks are noise. Sample multiple runs and watch multi-week trends.
- Personalization and memory. ChatGPT memory, custom instructions, signed-in Google history and location change answers. Trackers usually query a clean or logged-out context, which is not what a returning user sees.
- API vs. app differences. Some tools query model APIs rather than consumer apps. API answers can differ from what users see in the app, especially for web search behavior.
- Model and feature churn. A model update can move your numbers overnight without anything changing on your site. Annotate model launches on your trend charts.
- No volume data for prompts. Unlike keywords, there is no public data on how often a given prompt is asked. Tool “prompt volumes” are estimates; treat them as directional.
- AI Overviews are query-dependent. Many queries show no AI Overview at all, and whether one appears can vary between checks.
- Mentions are not traffic. A recommendation without a click still influences buyers, but it won’t show in analytics. Pair visibility data with self-reported attribution (“How did you hear about us?” with an AI assistant option).
Decision rule: act on a change only when it persists for two or more consecutive checks across multiple runs.
How often should you check, and what is a good score?
- Monthly is enough for most B2B teams doing a manual check.
- Weekly or daily tracking makes sense when you are actively shipping GEO changes and want to see what moved.
- There is no universal benchmark for a “good” mention rate. Compare yourself to the competitors in the same prompt set, and aim to raise share of voice in the bucket closest to revenue, usually category discovery and comparisons.
Fix the gaps, not the dashboard
Tracking only matters if it produces a work list: pages to create, pages to restructure, third-party sources to earn and crawler rules to fix. In GTM Hacker, the GEO Visibility Auditor audits how ready your site is to be cited by ChatGPT, Perplexity, Claude and Google AI and files the gaps as tasks on your SEO board, where you approve what gets worked on. Pair it with a monthly prompt check so you can see whether the fixes changed what the assistants say.