Define the boundary before selecting a model

We map the data sources, users and tasks the system must support. The architecture identifies where documents are processed, where representations are stored and where prompts and outputs travel. We examine external dependencies such as hosted model calls, diagnostics or connected tools, rather than treating local inference as the whole privacy story. Access controls need to reflect the company’s permissions, including who can query which knowledge. The brief records required isolation and operational constraints. This gives your team a concrete boundary to review before committing to a model or buying infrastructure for an undefined workload.

Test the task against available resources

Model choice depends on the task, language, context needs, available hardware and relevant licence conditions. We evaluate candidates with representative requests instead of selecting only by model size or a public benchmark. Resource planning considers memory, response behaviour and concurrent use within the agreed workload. Document processing and retrieval may require components beyond the inference service. We identify where a smaller model is sufficient and where the task needs a different approach. If a fully private setup cannot meet the requirements, the trade-offs are discussed explicitly; external processing is not quietly introduced as a substitute.

Make the deployment operable

Implementation covers the agreed model service, knowledge connections, application access and deployment configuration. We define what diagnostic information is retained and how administrators can update or disable components. Evaluation is repeated when a model or important configuration changes. Handover includes ownership of code and accounts, dependency notes and procedures for common operations. Self-hosting means your environment also needs a plan for updates, backups and incidents; it does not make maintenance disappear. We agree those responsibilities before wider use so the private architecture remains understandable when the original project team is no longer handling every request.

We document the distinction between local inference and local knowledge preparation. A private chat endpoint is insufficient if documents are still sent elsewhere for parsing or representation. Reviewing every processing stage gives the client a meaningful basis for approving the setup and for checking future additions against the same boundary.