“Will this fit?”, “Where is my parcel?” and “The product injured me” can all arrive in the same chat box. They require different evidence, urgency and authority. A system that handles them as variations of one friendly-response task may sound consistent while making the wrong decision.
The commercial design question is which help the assistant can actually complete, and how the remaining problem reaches a person who can act.
Start with the source and the consequence
Illustrative conversations: All products, customers, systems and exchanges below are fictional teaching examples. They do not describe an Aright deployment, tested system or actual complaint. Forma D2 has the same supplied facts as our product-copy example: two dividers per pack, 32–47 cm contact span, 8 cm body height, at least 8.5 cm clear internal height, and flat, parallel, unobstructed contact faces. No load or injury assessment is supplied.
| Customer question | Information needed | Useful assistant action | Decision it must not invent |
|---|---|---|---|
| A 49 cm internal span, with 9 cm height | Current product conditions and the customer's stated measurements | Explain that the span fails the stated requirement | Claim compatibility because another dimension fits |
| A parcel is late and a refund is requested | Authenticated order, promised route, current events and applicable remedy facts | Explain the missing access and route the case through verified order help | Invent a carrier status, refund decision or delivery date |
| A divider broke and someone was injured | A prompt human safety response and appropriate help, not an ordinary product FAQ | Stop routine troubleshooting and initiate the defined safety handoff | Diagnose injury, reassure that continued use is safe, or wait for a normal analytics review |
The distinction is not whether the model can produce plausible words. It is whether those words complete an appropriate task with the information and authority available.
Conversation one: give the bounded answer
Customer: My drawer measures 49 cm between the inside contact faces and has 9 cm clear height. Can I use D2?
Assistant: No. The stated contact span for D2 is 32–47 cm, so 49 cm does not meet that requirement. The 9 cm height does not cancel the span mismatch. Do not choose D2 for this drawer on these measurements.
Customer: Could I angle it so it reaches?
Assistant: The supplied conditions require flat, parallel contact faces. I have no product evidence supporting that alternative installation, so I cannot recommend it. A different suitable product or a product-specific assessment is needed.
This answer is useful without a human repeating the specification. It also does not make every unsupported situation merely “probably fine.” If the customer had supplied only the cabinet's external width, the proper answer would instead identify the missing internal measurements. Known mismatch and missing information are different branches.
The response should be tied to the current product source. If a conflicting instruction appears, stop the compatibility answer until the conflict is resolved. A saved response must not outrank a corrected specification just because it is easier to retrieve.
Conversation two: do not perform an order lookup in prose
In this scenario the assistant has public product information but no authenticated order access. A verified account-based order-help route exists and is staffed; that is an explicit fictional assumption, not a feature claim.
Customer: My parcel is two days later than the date I expected. Has it been lost? Can you refund me?
Assistant: I cannot see or verify your order or carrier record in this chat, so I cannot tell you that it is lost or decide the refund. Please use the store's verified order-help route from your account. It lets the team check the promised date, current shipment information and the appropriate next action. Do not post payment-card details here.
Customer: The last assistant told me it would arrive tomorrow.
Assistant: I cannot confirm that date. Include that earlier message in the order-help case so the team can examine the promise as well as the shipment. The status and remedy need a verified response, not another estimated date from this chat.
The answer does not settle the customer's rights or ask them to wait indefinitely. It makes the access gap explicit and gives a usable next step. The business must keep that step available and follow up appropriately. If the route fails, the support design needs a working alternative; repeating the same link is not escalation.
If an authenticated integration is later added, reassess its exact capabilities. Read access to tracking does not itself grant permission to cancel an order, disclose another customer's data or issue a payment. The answerable portion can expand without giving the assistant every operational authority.
Conversation three: interrupt the normal flow
Customer: One divider broke and cut my hand. Can I keep using the other one?
Assistant: Stop using the product. I cannot assess the injury or confirm that another piece is safe to use. If you need urgent medical help, contact your local emergency service rather than waiting for this chat. This needs the store's human safety-response process, not ordinary installation advice.
The assistant should not request a demonstration of the failure, suggest another installation or continue a satisfaction survey. It should explain the established safety contact route and preserve the report for authorized staff. Collect only what the defined process needs; injury details and photographs are not material for unrestricted analysis or public chat.
Where a real handoff is supported, report success only after the receiving queue confirms it. In a fictional confirmed-handoff continuation, the message could be:
Assistant, after the safety queue acknowledges receipt: Your report has been accepted into the human safety queue. That acknowledgement is not an injury assessment or a promise that a particular remedy is approved. If you have not received the promised human contact, use the store's verified safety contact route; do not rely on this chat for urgent help.
Without that acknowledgement, say the handoff is unconfirmed and provide the actual alternative. A generated case number is not a received case. The human team must determine the appropriate product, customer and market response, including any applicable reporting duties; the chat should not delay it.
Evaluate the handoff as part of the answer
NIST's Generative AI Profile highlights human–AI interaction risks, including over-reliance. A polished reply can make a customer or operator believe more has happened than actually has.
Test therefore whether the correct source was used, whether the system separated known facts from missing access, and whether the receiving team actually got the case. Include ambiguous messages and failures of the handoff route. A system that answers routine fit questions accurately but buries an injury report in an ordinary queue has not passed because its average response score is high.
Do not optimize the pilot around cases “deflected” from staff alone. Look at correct resolution, repeat contacts, time to an appropriate person, abandoned handoffs and important cases missed. A customer who stops asking because help is inaccessible is not a successful automated resolution.
The practical release choice is selective: answer the bounded product question; move the order decision to verified order help; interrupt routine automation for the safety report. If the source, access or human response route is missing, narrow the assistant's task until it can offer an honest and useful next step.