Teams that run local LLMs usually stall on habits, not on model names: messy folders, unclear versions, and no acceptance checks. This note turns 「Chinese punctuation quirks after heavy quantization」 into a short procedure a small team can follow.
Why write it down
Verbal rules disappear after a rotation. A short page with links to verified sections on the site beats searching chat history. Start with basics; for hardware or build context open vllm sglang intro.
Steps
- Define done: what evidence counts and who signs off.
- Change one variable at a time—weights, quantization, or concurrency.
- Keep a minimal record: date, host name, command or UI path, and a log snippet.
- Reproduce failures before patching; avoid stacking fixes on unreproducible states.
Common drifts
- Copying demo configs into production without checking permissions and networks;
- Chasing larger models while ignoring disk, PSU and cooling headroom;
- Skipping a public “what we will not do” note, which invites over-trust.
Close
Sync the conclusion to your internal doc and validate with two real examples in the next standup. Steady local AI comes from repeatable small steps. Browse the site sections when you need deeper tutorials or build notes.
Published by MicroAI One (本地AI 1号网), a Shenzhen team working on local LLM hardware, software and deployment.
Extra operational notes for local teams
When you run models on your own hardware, write down the host name, GPU model, driver version and the exact file names you loaded. Keep a short smoke-test prompt that you run after every change so you know whether the stack still answers. Prefer one change at a time: new weights, new quantization, or new concurrency settings, not all three together. Store logs with timestamps and keep at least a week of history on a separate disk if the machine is shared. If a colleague reports a wrong answer, ask for the prompt, the model id and the retrieved chunks before you blame the model. Document what the system must refuse, especially anything that needs live external write access. Review disk usage weekly because model files and vector indexes grow quietly. For Chinese business text, keep a small internal glossary and feed it as context instead of hoping the base model remembers your product names. These habits matter more than chasing a slightly larger parameter count on a machine that is already thermally limited.