When several people use one local model server, the network setup becomes part of your security and reliability. You do not need a complex design, but you do need a deliberate one.
Decide who needs access
List users and applications. Not everyone in the building needs the model API. Limit access to the groups that do.
Network segments
If your equipment supports it, place the server in a separate segment or VLAN reachable from the staff network on specific ports only. At minimum, use host firewall rules to restrict who can connect.
Ports and services
- Expose only what is needed, usually one front end or one API port.
- Keep management interfaces (SSH, remote desktop) on a restricted path.
- Put a reverse proxy in front if you need TLS or authentication.
Authentication
Do not rely on "it is only on the internal network". Use accounts or API keys per person or application so that you can revoke access individually.
Logging
Record who connected and when, and keep the logs for a period appropriate to your policies. Do not store full prompts unless you have a clear need and a protection plan.
Capacity
Know how many simultaneous users the machine handles comfortably. Set limits so a runaway script does not block everyone.
Test and document
Check from a normal laptop that you can reach the service, and from a device that should not have access that you cannot. Keep a diagram of the setup for whoever maintains it next.
See our small-team shared LLM guide and the hardware overview for context. If something does not connect, start with the troubleshooting tutorials.
A short maintenance routine
Once a month, review the firewall rules and the list of keys in use, remove accounts of people who left, check that the logs are rotating and not filling the disk, and confirm the diagram still matches reality. Fifteen minutes of regular attention keeps a shared server predictable, and it makes any later security review far easier to pass.
Published by MicroAI One (本地AI 1号网), a Shenzhen team working on local LLM hardware, software and deployment.