A local model server that binds to all interfaces without a firewall plan is an accident waiting for a curious scanner. On a small office LAN you can stay practical without enterprise theatre.
Inventory first
List the ports your stack actually uses: the model server, the web UI, maybe an embedding service. Note which machines should call them. Everything else should not.
Rule pattern
- Allow inbound to the model port only from the office subnet or specific host IPs.
- Keep admin UIs tighter—sometimes only localhost plus VPN.
- Block inbound from guest Wi-Fi ranges if your router separates them.
- Document the rules in the same repo as compose or service files.
Verification
From an allowed PC, hit the health endpoint. From a disallowed PC, confirm failure. After a Windows update, re-check; updates sometimes reset expectations if someone clicked the wrong prompt.
Operations
Name firewall rules clearly. When an employee laptop is replaced, update IP allowlists the same day. Pair this with basic auth on the UI so a missed rule is not game over.
For broader deployment context see solutions, install orientation in the install guide, hardware placement ideas under hardware, and matching notes at device matching. Firewall policy will not improve token quality, but it keeps your internal assistant from becoming a hallway curiosity.
A short maintenance routine
Once a month, export the current rule list, compare it to your written diagram, remove entries for people who left, and confirm logs still show only expected clients. Fifteen quiet minutes here prevents noisy security reviews later and keeps tribal knowledge from living in one persons head.
Extended scenario
Imagine a contractor laptop on guest Wi-Fi. Without segmentation, it might reach your model port. With guest isolation and firewall allowlists, that path dies quietly. Add an annual tabletop exercise: pick a random IP, show whether it should connect, then verify. Keep screenshots of successful and blocked curls in the ops folder. Security for local AI is mostly classical LAN hygiene applied to new ports, and teams that already run file servers already know most of the moves—they just need to reuse them on the inference host.
Published by MicroAI One (本地AI 1号网), a Shenzhen team working on local LLM hardware, software and deployment.