Exposing a local RAG API on the office network feels harmless because the weights never leave the building. The documents inside the index are still sensitive. Secure defaults matter as much as model quality.
Defaults worth enforcing on day one
Require authentication at the gateway. Separate collections by department instead of relying on prompt instructions to hide finance files. Return citations with path prefixes so audits are possible. Disable anonymous admin routes. Cap request rates so one script cannot knock over the GPU. Prefer allowlists for tool calls if the API can fetch files.
Implementation sketch
Put a reverse proxy in front of the API. Store service credentials in a secret manager, not in a shared spreadsheet. Log user id, collection, and latency; keep raw prompts out of default logs unless legal holds require a vaulted trail. Align architecture notes with solutions and inquiry paths at solutions/inquiry. Corpus prep still follows boring hygiene from tutorials—garbage filenames in, garbage retrieval out.
Capacity and product surfaces
RAG and chat fighting for the same VRAM create pressure to "just open permissions so demos work." Resist that shortcut; size hardware with hardware and match. Desktop products that emphasize private deployment expectations are described at MicroAI One. Methodology for reviewable changes lives at methodology.
Extra
Run a quarterly access review: who can hit which collection. Delete stale API keys the same week people leave. Lab indexes must not silently include production contracts (lab). Contact contact if you need a delivery checklist beyond DIY.
Local RAG security is mostly gatekeeping and deletion discipline, not a special cipher.
Appendix: collection naming
Name collections after departments and sensitivity, not after project nicknames that rot. Document who approves membership changes. When demos need a public-ish corpus, clone a sanitized collection instead of widening production ACLs. Pair naming with deletion SLAs so departed projects do not linger in embeddings forever.
Appendix B: logging red lines
Never log full prompts containing customer identifiers to a shared plaintext file. Prefer hashed user ids. If legal requires retention, encrypt at rest and name a destruction date. Teach support that needing the raw prompt is an exceptional request with a ticket, not a Slack habit. These rules keep local RAG from becoming a shadow CRM dump. Review access lists quarterly and delete stale API keys the same week people leave.
Published by MicroAI One (本地AI 1号网), a Shenzhen team working on local LLM hardware, software and deployment.