How to measure which unanswered shopper questions cost conversions

Unanswered shopper questions cost conversions when they appear at a high-intent decision point, concern a product the shopper is considering, and are followed by weaker engagement or purchase behavior than comparable sessions. Measure that chain directly: classify each question and answer outcome, join it to product and funnel events, estimate the revenue at risk, then validate the biggest opportunities with a controlled test.

Start with question outcomes, not conversation counts

Conversation volume tells you how often shoppers seek help. It does not tell you whether a question blocked a purchase, was answered successfully, or came from a low-intent visitor.

Create a record for every question with five fields:

Field	What to capture
Question	The shopper’s original wording, preserved before categorisation
Context	Product or category viewed, device, traffic source, and session timestamp
Intent	Fit, size, compatibility, ingredients, use case, shipping, returns, comparison, or product discovery
Outcome	Answered, partially answered, unanswered, escalated, abandoned, or resolved after a recommendation
Commercial result	Product view, comparison click, add to cart, checkout start, purchase, revenue, and later return or exchange where available

Define “unanswered” before reporting it. A message should count as unanswered if the assistant gives no answer, signals uncertainty, cannot find supporting product information, or hands the shopper to support without resolving the buying question. A technically generated reply is not the same as a useful answer.

Keep question themes separate from individual messages. “Will this fit a 16-inch laptop?” and “What size laptop fits?” may represent one compatibility gap, while a single shopper asking three follow-ups may represent one unresolved decision. Report both message-level volume and shopper-level incidence so repeated prompts do not inflate the signal.

Join unanswered questions to the purchase funnel

The strongest evidence comes from connecting each question outcome to what the shopper did next. Track the question event alongside product engagement and standard ecommerce events such as view_item, add_to_cart, begin_checkout, and purchase; Google documents these events as the basis for measuring ecommerce behavior and purchase funnels.

Use a consistent attribution window. For example, assign a purchase to the question session when it occurs before purchase, and keep a separate view of purchases that happen later. The exact window should fit your buying cycle; the key is to choose it before comparing themes.

A useful question-level dataset looks like this:

session_id
question_id
question_theme
product_id
question_timestamp
answer_status
answer_confidence
recommendation_clicked
product_view_after_question
add_to_cart_after_question
begin_checkout_after_question
purchase_after_question
order_value

Measure the following for each theme, product, and category:

Unanswered-question rate: unanswered shoppers divided by shoppers who asked that theme.
Resolution rate: shoppers who received a satisfactory answer, clicked a relevant recommendation, or reached a human resolution.
Post-question engagement: product views, comparison clicks, and add-to-cart actions after the question.
Purchase rate: purchasers divided by shoppers who asked the theme.
Revenue per question session: attributed revenue divided by question sessions, with zero-purchase sessions included.
Drop-off rate: sessions that end after the unanswered question without an add-to-cart or checkout event.

Do not treat the last metric as proof that the question caused the exit. It identifies where to investigate. Google’s funnel exploration documentation describes funnels as a way to visualise success and failure at each step and find inefficient or abandoned journeys; it does not turn a funnel drop-off into causal evidence.

Estimate the conversion and revenue opportunity

Rank unanswered questions by expected commercial impact, not by volume alone. A theme asked 20 times on a high-value product may deserve attention before a theme asked 500 times by casual browsers.

One practical opportunity estimate is:

opportunity = eligible unanswered shoppers
              × benchmark conversion rate gap
              × average order value

Use a benchmark that matches the question’s context. Compare unanswered sessions with one of these groups, in order of preference:

Resolved sessions with the same question theme and product context. This is usually more informative than comparing all assistant users with all site visitors.
Sessions that saw the same product and funnel stage but did not ask a question. This helps separate question-related friction from general product or traffic differences.
A pre-launch or historical baseline. Use this when the available question sample is too small, but label the result as directional.

The gap is calculated as the purchase rate of the comparison group minus the purchase rate of unanswered-question sessions. Multiply by the number of eligible unanswered shoppers and average order value to create a scenario, not a booked-loss figure.

Show a range rather than false precision. A conservative case might use the lower end of the observed gap; an upside case can use the full gap. Include sample size, date range, product mix, traffic source, and whether repeat purchasers are included. This makes the estimate reviewable by finance and ecommerce teams.

Separate correlation from causation with a test

A lower conversion rate after an unanswered question is a warning signal, not proof that the missing answer caused the sale to disappear. Shoppers with difficult questions may already be less certain, may be comparing competitors, or may be buying more expensive products.

To prove incremental conversion, test the fix. Group a recurring question theme into a treatment and control experience:

Control: the current product page or assistant behavior.
Treatment: a verified answer, clearer product content, a comparison module, or a guided recommendation that addresses the question.
Primary outcome: purchase rate or revenue per eligible session.
Secondary outcomes: add-to-cart rate, checkout start, answer resolution, support handoff, and returns.
Guardrails: margin, refund or exchange rate, support contacts, and page performance.

Randomise at the shopper or session level and keep the assignment stable where possible. Include only shoppers exposed to the relevant product or question context. Do not stop the test because a few early purchases look promising; set the decision rule and test duration in advance with your experimentation team.

If randomisation is not available, use a matched comparison by product, device, traffic source, and funnel stage, and call the result an association. A before-and-after improvement can support a rollout decision, but it is weaker proof because promotions, seasonality, inventory, and media mix may have changed at the same time.

Turn the result into a prioritized question backlog

The best next action depends on why the question went unanswered. A conversion problem is not always an assistant problem.

Finding	Likely action
High volume, high unanswered rate, clear product fact missing	Improve product data, product-page copy, or an FAQ block
Low volume, large purchase-rate gap	Investigate the high-intent journey and test a prominent answer
Answer given, but low add-to-cart rate	Check answer accuracy, recommendation relevance, price, and availability
Frequent comparison questions	Add comparison content or guided product selection
Shipping, returns, or location questions	Surface policy and next-step information earlier
Question requires judgement or exception handling	Define a human escalation path and measure resolution time
Theme linked to returns or exchanges	Fix expectation-setting before purchase, not only post-purchase support

Review the backlog by product, category, intent, and outcome. Anagram’s published framework recommends that same breakdown and lists product-page exits, recommendation clicks, add-to-cart actions, and purchases as signals in a closed measurement loop. Its Site Agent is positioned to answer product questions and guide recommendations while its insights show what shoppers ask and where the buying experience has friction. That makes it relevant when the team wants the question capture and the on-site intervention in one workflow, but the conversion proof still comes from your ecommerce events and tests.

See Anagram’s shopper-question measurement framework for the question, outcome, and merchandising-loop view, and the Anagram homepage for its Site Agent and shopper-insight product context.

The dashboard your ecommerce team should review

A useful dashboard answers which questions are unanswered, how many high-intent shoppers encounter them, what those shoppers do next, and whether a fix changes purchases.

Include these views:

Theme ranking: unanswered shoppers, unanswered rate, purchase rate, revenue per session, and estimated opportunity.
Journey view: events before and after the question, including product engagement, add to cart, checkout, and purchase.
Segment view: product, category, device, new versus returning shopper, traffic source, and geography where sample size allows.
Answer-quality view: answer status, confidence or escalation, recommendation clicks, and unresolved follow-ups.
Experiment view: treatment versus control lift, uncertainty, guardrails, and the date range used.

Add the number of eligible shoppers to every rate. A 50% purchase rate from two shoppers should never outrank a 4% rate from thousands without showing the sample size.

The final measure is not “how many conversations did the assistant have?” It is: which unresolved decision did shoppers face, what commercial behavior followed, what was the expected value of fixing it, and did the fix produce incremental purchases without damaging the customer experience?