Skip to content
steelabs

Guide

Build a support assistant using company documents

Permission-aware retrieval, source-backed answers, difficult-question evaluations and an escalation path that survives stale knowledge and failed dependencies.

Published 7 min read6 chapters

Contents

  1. 01Prepare knowledge people can maintain
  2. 02Enforce permissions before generation
  3. 03A citation check is useful but limited
  4. 04Test questions that a demo avoids
  5. 05Design a fallback people can use
  6. 06Cost and maintain the whole knowledge system

Build a support assistant using company documents

A company-document assistant is a knowledge system with a generated interface. Its usefulness depends on source quality, retrieval and access controls before it depends on conversational polish. Begin with a specific support task and approved knowledge, rather than indexing every folder the integration account can reach.

The goal is an answer a user can verify or a useful handover when the assistant cannot answer. A confident invented policy is a worse outcome than a clear limitation. This guide describes an implementation pattern, not a claim that a demo has reduced a client’s support workload.

Chapter 01

Prepare knowledge people can maintain

Select documents with a named owner, intended audience, effective date and deletion rule. Resolve duplicate or conflicting policies before indexing. Preserve document identifiers and section boundaries so a retrieved passage can be traced to a visible source.

For a refund assistant, provide the approved policy and product-specific exceptions. Keep internal escalation notes separate from customer-visible documents. Decide whether historical policy versions are relevant and how the assistant should distinguish a current purchase from an old one.

  • Extract text carefully from tables and scanned documents; missing structure can change the meaning.
  • Track source version and last successful ingestion, not merely the date the index was created.
  • Propagate access changes and deletion to indexes, caches and derived records.

Chapter 02

Enforce permissions before generation

  1. 01Authenticate the user and resolve permitted knowledge collections on the server.
  2. 02Retrieve only passages the user may access, with source IDs and version information.
  3. 03Check whether the retrieved evidence is sufficient for the question.
  4. 04Generate a concise answer from that evidence, allowing abstention.
  5. 05Validate cited IDs, then render permitted source links and a human escalation route.

Never retrieve across all customers and ask the model to hide the wrong records. Access filtering must constrain the retrieval result. A later citation filter cannot undo disclosure if forbidden content already reached the model or logs.

Check retrieval and generation separately. If the right policy is missing from the context, adjusting answer wording will not repair the search problem. If retrieval is correct but the answer adds an unsupported exception, the generation or validation layer needs attention.

Chapter 03

A citation check is useful but limited

Illustrative citation boundary; tested locally for missing and unknown source IDs, not semantic accuracy.
function grounded(answer, permittedSourceIds) {
  if (!answer || answer.kind !== 'answer') return false;
  return typeof answer.text === 'string' &&
    answer.text.trim().length > 0 &&
    Array.isArray(answer.citations) &&
    answer.citations.length > 0 &&
    answer.citations.every(id => permittedSourceIds.has(id));
}

This check prevents references to unknown IDs. It cannot prove that the cited passage supports the answer. Evaluate claim support separately, inspect quotations and provide a route to the source. A real system also validates output length, schema, links and safe rendering.

For action requests, keep answering and acting distinct. An assistant may describe a refund policy without permission to issue a refund. Any write action needs a server-side authorisation decision, an agreed approval path and duplicate-request protection.

Chapter 04

Test questions that a demo avoids

Build an evaluation set from approved support questions, with expected sources and answer constraints labelled by the knowledge owner. Hold back a set from prompt tuning. Include answerable questions, missing information, ambiguous wording, conflicting policies, an unavailable source and requests from the wrong user role.

Add adversarial documents that tell the assistant to ignore its task or reveal other records. Treat those instructions as content. Verify that permission isolation still holds independently of the model response. Prompt wording is only one layer; source access and tool permissions do the enforceable work.

Score retrieval relevance, supported claims, appropriate abstention, source accessibility and escalation quality. Measure latency and cost on the same examples. An overall answer-quality percentage can hide a serious permission failure, so review that category separately.

Chapter 05

Design a fallback people can use

When evidence is inadequate, say what is missing and offer the next useful step. Preserve the original support channel. A timeout should not strand the user in a spinner, and an unavailable model should not prevent ordinary search or contacting support.

Start with a small audience and monitored scope. Review sampled answers under an approved data policy, collect feedback categories and have an owner for disputed answers. Do not make reviewers read every transcript without considering the personal data those transcripts may contain.

A provider’s training policy is not a complete privacy assessment. Confirm retention, storage, region, subprocessors and deletion for the selected product and account. The API and the provider’s consumer chat product can have different terms and controls.

Chapter 06

Cost and maintain the whole knowledge system

Estimate requests per month, average retrieved context, generated output, retries and evaluations. Add document parsing, indexing, hosting and support-review effort. Limit retrieval volume and request size, and alert on unusual usage without assuming a billing dashboard alone is an enforcement boundary.

Re-index changed sources and test deletions. Re-run the held-out set after a model, prompt or retrieval change. Check expired credentials and broken source links. A knowledge assistant needs someone responsible for policy freshness, not just an engineer responsible for uptime.

See product AI integration and the maintenance checklist. An enquiry can begin with the intended audience and a sample of approved documents.

Source handling references checked 11 October 2026: OWASP: prompt injection and OpenAI: API data controls. The permission and citation examples are illustrative controls, not a guarantee of correct answers.

Going deeper

The individual pieces

Practical companion resources

Contact

Turn the guide into a scoped next step

Bring the task, approved examples and existing tools. We can review feasibility, define the evaluation and estimate the first useful deliverable.

Start the conversationsales@steelabs.tech