Handling questions the assistant should refuse

There is a temptation to treat every refusal as a failure. It shows up in the metrics as an unanswered question, and unanswered looks bad. That framing produces systems that answer things they should not, and a confidently wrong answer about a refund policy costs more than a refusal ever will.

The four categories that should always refuse

1. Account specific questions

"How many credits do I have left" cannot be answered by a system with no authenticated access to that account. Attempting it either fabricates or leaks. Route to login or to a human.

2. Commitments

Anything that would constitute a promise: custom pricing, contractual terms, delivery dates, whether you will build a feature. These need a person with authority, and a system that improvises them creates obligations the company did not agree to.

3. Legal, medical, financial advice

Even when your content touches these areas, applying them to someone's specific situation is a different act from describing them. State the general position, decline the application, point to a person.

4. Anything outside the indexed content

The broadest category and the one that matters most day to day. If no source covers it, say so. This is the boundary the whole design rests on.

Making refusal happen reliably

Instructing a model not to answer outside its context helps and is not sufficient on its own. Two structural mechanisms do more work:

A retrieval threshold. If the best matching chunk scores below a floor, do not pass context at all and return the refusal path directly. The model never gets the chance to over-reach because it never receives weak context to over-reach from.

Output constraints. Requiring the answer to cite which source it came from makes unsupported claims structurally awkward to produce and easy to detect afterwards.

Refusal wording matters

A refusal is still an interaction and it can be helpful or unhelpful. Three components make the difference:

• State plainly that the answer is not available, without apologising at length.
• Say what is available, if anything nearby is.
• Offer the next step, which is usually a person or a specific page.

"I do not have information about enterprise contract terms. I can tell you what is in the published plans, or connect you with someone who can discuss contracts." That is a useful refusal. It closes one door and opens two.

Tuning the threshold

Set the refusal threshold too high and the assistant is useless. Too low and it invents. There is no universal correct value, and it depends on your content density and embedding model.

The empirical approach: take fifty real questions, half of which you know are covered by your content and half of which are not. Sweep the threshold and find the point where covered questions are answered and uncovered ones are refused. Then set it slightly toward refusal, because on a public marketing site the cost of a wrong answer is higher than the cost of an unnecessary escalation.