Back to Blog

Inside Our Query Engine: How We Test 100+ AI Conversations About Your Business

Brightwill Team·2026-03-20

When someone asks ChatGPT “What's the best sushi restaurant in Miami?”, the AI has to decide which businesses to name. That single response is one data point. To understand how AI engines actually treat your business, you need hundreds of data points across different question types, phrasings, and platforms. That's what our query engine does.

This article explains exactly how we generate and run those queries - the business profiling step, the 10 query types we test, how we adapt queries to your business type, and what happens when the queries hit real AI engines.

brightwill.ai/report
Brightwill query evidence showing real AI conversations
Every query is captured with the full AI response - see exactly what each engine says about your business.

Why One Question Isn't Enough

AI responses vary significantly based on how a question is phrased. Asking “best Italian restaurant in Chicago” produces a different response than “where should I take my parents for a nice dinner in Chicago?” - even though both might be relevant to the same restaurant. A single query gives you a snapshot. A comprehensive audit gives you a statistical picture.

Our query bank contains 53 templates per business category across 10 query types. For a comprehensive audit, each template is sent to all three AI engines - ChatGPT, Claude, and Gemini - producing over 150 individual query-response pairs per audit. The free audit runs 5 queries through ChatGPT only, giving you a quick directional read.

Step 1: Understanding Your Business

Before we generate a single query, we need to understand your business at a granular level. Knowing you're a “restaurant” isn't enough - we need to know you're a “Japanese/Sushi restaurant” that specializes in omakase and sake pairings.

Our profiler uses Gemini 2.5 Flash with Google Search grounding to research your business in real time. It takes your business name, category, and location (or product description for digital businesses), searches the web, and returns a structured profile in about 2–4 seconds.

What the Profiler Returns

Subcategory
Your specific niche within the broader category
e.g. Japanese/Sushi
Specialties
3-5 things you're known for
e.g. omakase, sashimi, sake pairings
Search Terms
3-5 terms people use to find businesses like yours
e.g. sushi restaurant, Japanese food
Subcategory Plural
Natural language plural for query templates
e.g. sushi restaurants

This profiling step is what makes our queries specific rather than generic. Instead of asking AI “best restaurants in Miami,” we can ask “best sushi restaurants in Miami” and “best omakase experience in Miami” - the kinds of questions your actual customers ask.

For digital businesses (SaaS, ecommerce, creators), the profiler adapts its approach. Instead of location-based context, it identifies your product niche, target audience, and competitive landscape. If the profiler can't find enough information about your business online, it falls back to your broader category - which itself is a signal worth knowing.

Step 2: Generating the Right Questions

We test 10 distinct query types. Each type probes a different dimension of how AI engines perceive your business. Some test whether AI discovers you organically. Others test whether AI knows accurate information about you when asked directly.

Discoverygeneric
Does AI recommend you when someone searches your category?
"Best sushi restaurants in Miami"
Subcategory Discoverygeneric
Does AI find you for niche-specific searches?
"Best omakase experience in Miami"
Directdirect
Does AI know about you when asked by name?
"Tell me about Joe's Sushi in Miami"
Comparisongeneric
How do you stack up against competitors?
"What are the main differences between the top sushi restaurants in Miami?"
Use Casegeneric
Are you recommended for specific scenarios?
"Where should I take a date for sushi in Miami?"
Reviewsdirect
What does AI say about your reputation?
"What are people saying about Joe's Sushi?"
Specificsdirect
Does AI know your services, pricing, and details?
"What does Joe's Sushi specialize in?"
Source Probingdirect
Which sources does AI cite when discussing you?
"Where can I find reviews for Joe's Sushi?"
Verificationdirect
Does AI have accurate, current facts?
"Is Joe's Sushi still open?"
Fact Verificationdirect
Probe for phone, hours, website with source citations
"What is Joe's Sushi's phone number? Where did you find this?"

The Generic vs Direct Split

About 60% of our queries are generic - they don't mention your business name. These test organic discovery: whether AI recommends you when someone is looking for what you offer but hasn't heard of you yet. This is where the real growth opportunity lies.

The remaining 40% mention your business by name. These test brand awareness: whether AI knows accurate information about you and whether it speaks positively when someone asks directly. Both metrics appear separately in your report as Organic Discovery Rate and Brand Awareness Rate.

Step 3: Adapting to Your Business Type

Not all businesses operate the same way, and our queries reflect that. We classify businesses into three scopes, each with tailored query behavior:

Local
Restaurant, Gym, Salon, HVAC, Dental, Legal, Real Estate
Location-centric queries. AI recommendations here are tied to geography.
{marketContext}"in Miami"
Digital
SaaS, E-commerce, Online Course, Creator, Telehealth
Product and audience-centric queries. No location, focused on use cases and features.
{marketContext}"for startups"
Hybrid
Agency, Consulting, Freelancer, Nonprofit
Both location and product context. Gets the widest range of query types.
{marketContext}"in NYC for enterprise"

Our query templates use placeholders like {businessName}, {marketContext}, and {subcategoryPlural} that get filled in with your specific business data. A template like “Best {subcategoryPlural} {marketContext}” becomes “Best sushi restaurants in Miami” for a local sushi restaurant, or “Best project management tools for startups” for a SaaS product.

How It All Runs

The execution differs between our two tiers:

Free Audit
15-25 seconds
+1 AI engine (ChatGPT)
+5 queries run sequentially
+Business profiling (~2-4s)
+Recommendation probability calculated
+Results cached for 24 hours
Comprehensive Audit
5-15 minutes
+3 AI engines in parallel
+48 queries per engine (140+ total)
+Business profiling (~2-4s)
+Full GEO Score (0-100) computed
+Always runs fresh (no cache)

For the comprehensive audit, the three AI engines run in parallel. ChatGPT (GPT-4.1-mini), Claude (Claude Sonnet 4), and Gemini (Gemini 2.5 Flash) each process the same set of queries independently. This cross-platform testing reveals where your visibility differs between engines - a business might be well-known to ChatGPT but invisible to Gemini, or vice versa.

Each individual query has a 30-second timeout with one automatic retry. If a query fails after the retry, it's recorded as a failed execution rather than silently dropped. The audit continues with the remaining queries.

What Happens Next

Every AI response goes through our multi-layer extraction pipeline that turns unstructured AI text into structured data - whether you were mentioned, how you were characterized, which competitors appeared, and what sources the AI cited. That structured data then feeds into the AI Visibility Score calculation and your personalized action plan.

The query engine is the foundation. The quality and breadth of the questions we ask directly determines the accuracy of the picture we can paint of your AI visibility. With 407 templates across 16 business categories and 3 scopes, we test the specific conversations that matter for your business - not generic queries that miss the nuances of your market.

Run a free audit to see 5 of these queries in action against ChatGPT.

Continue Reading

Methodology6 min read

AI Visibility Score: How We Measure If AI Recommends Your Business

A transparent look at our methodology — how we track visibility, position, and sentiment across 150+ real AI conversations to calculate your AI Visibility Score (0-100), using organic-only scoring and source attribution.

Read more
Methodology8 min read

How We Analyze Every AI Response: Our Multi-Layer Extraction Pipeline

How we turn raw AI conversations into structured data — extracting mentions, sentiment, competitor rankings, and source citations from every response.

Read more
Methodology9 min read

From Data to Action: How We Build Your Personalized GEO Action Plan

How we turn audit data into 15-30 specific optimization actions — from pre-computed gap analysis to opportunity scoring and the earned vs owned media split.

Read more