There is no Search Console for AI shopping. Shopify will not send you a weekly report that says "you appeared in 412 assistant answers and lost the sale to a competitor 38 times." So most merchants have no idea whether agentic commerce is sending them nothing, or quietly sending sales they are not even attributing. They are flying blind on a channel that is growing every quarter.
The good news is that you can check it yourself, and the basic version takes about ten minutes. This is how to tell whether you are winning or losing in Shopify's Global Catalog, what to record, and how to read what you find.
What winning actually means
Before you measure, define the scoreboard. Winning in the catalog is not one thing, it is five, and each one fails differently.
- You appear for category searches, not just your brand. Showing up for your own name is table stakes. The customers you do not have yet search by category, like "organic dog treats" or "merino base layer," and that is where visibility turns into growth.
- Your product is clustered correctly. Shopify groups identical products from different sellers under one Universal Product ID. If yours is mis-clustered or stranded, you are not even in the comparison.
- You are the chosen offer. When several stores sell the same product, the assistant shows one. Being in the cluster but never chosen is its own kind of losing. The mechanics are in the agentic buy box.
- The details are accurate. The price, the category, the attributes the assistant repeats all have to match reality. A confident wrong answer loses the sale and the trust.
- The product is buyable. If it is out of stock, not shipping to the shopper, or not eligible for a clean checkout, the assistant routes around you.
Hold those five in mind, because every check below maps back to one of them.
The ten-minute check: ask the assistants
The fastest way in is the same surface your customers use. Open ChatGPT, Gemini, and the Shop app, and ask the questions a new customer would ask. The rule is simple: never search your brand name. Search the way someone who has never heard of you shops.
For each query, write down three things:
- Did you appear at all?
- Where in the list were you?
- Which competitors showed up, and were they above you?
Run eight to ten of these. Use real category terms, add the qualifiers customers actually use ("under $40," "ships to Canada," "for sensitive skin"), and keep the list so you can run the exact same set next month. Within an afternoon you will have a clearer read on your agentic visibility than almost any store in your category has bothered to get.
The systematic check: sample the catalog
The assistant test shows you the customer's-eye view. To understand why, you sample the catalog itself. It is queryable without an API key, which is exactly how assistants and tools read it, so a fixed list of category and intent searches gives you a repeatable read on presence, position, and the competitor set behind each query. This is the layer where a tool earns its keep, because running the same queries by hand every month gets old fast.
Either way, the output is the same scoreboard.
The six signals to record
| Signal | What it tells you | If it is bad, look at |
|---|---|---|
| Presence rate | Share of your target queries where you appear at all | Agentic channel turned on, category, syndication |
| Brand-vs-category gap | You appear for your name but not the category | Category and attribute data |
| Average position | Where you land when you do appear | Ranking signals: data richness, ratings |
| Buy-box win rate | When clustered, are you the shown offer | Price, shipping coverage, data completeness |
| Competitor set | Who keeps beating you, and on what | Their price, ratings, attribute depth |
| Accuracy and freshness | Do the shown details match your admin | Inference errors, stale price, mis-category |
The two most revealing are the brand-vs-category gap and accuracy. Nearly every store I check appears for its own name and disappears for its category, which is the single most common agentic-discovery problem. I wrote up that exact pattern in why AI shopping shows your brand but skips your category.
How to read what you find
The scoreboard points you at a cause. Here is the quick decision tree.
- Absent almost everywhere. Either you are not in the catalog or you are mis-categorized out of the searches that matter. Confirm agentic discovery is on under Sales channels then Agentic, then check your product categories. The full triage is in why your products don't show up in AI shopping.
- Present for your brand, absent for the category. A category and attribute gap. This is a data problem, and it is fixable. Start with the product fields that decide your AI shopping rank.
- Present but ranking low. You passed the filters but lost on ranking. Usually thinner data or weaker ratings than the stores above you. Enrich the attributes and get real reviews onto the product pages.
- Present but the details are wrong. Either Shopify inferred an attribute incorrectly or your price or stock is stale. Inference has no confidence score, so a wrong guess looks identical to a fact. State the value yourself to override the guess.
- In the cluster but never chosen. A buy-box loss, which comes down to price, shipping coverage, or data completeness against the seller who keeps winning.
Make it a habit, not a one-off
A single check tells you where you stand today. The trend tells you whether you are gaining or slipping, and the trend is the part that actually drives decisions. Catalog data shifts as your inventory, pricing, and competitors move, so the only way to manage it is to measure it on a schedule.
Keep a fixed list of category queries, run it monthly, and log the six signals each time. When a number moves, you will know which lever to pull because you already know which signal maps to which fix. This is rank tracking for agentic commerce, and right now doing it at all puts you ahead of almost every competitor in your category.
Start with the data side
The measurement above tells you where you stand. The fixes almost always come back to product-data quality, which is the part you can audit right now.
Our free Shopify AI-readiness checker scores the product and structured data the catalog reads and hands you the specific gaps, ranked by impact. Pair it with the Shopify Global Catalog guide for the full picture of how discovery works.
More in the AI and agentic commerce library.

Comments
Every comment here comes from a verified email. Write yours, confirm from your inbox, and it's live.
Loading comments…