Context for "WSL2 Ollama disk growth warnings"
Move models to a dedicated VHDX; monitor free space weekly. Software is the part of a local AI stack that changes fastest. A setup that worked last month can break after a driver update, a runtime upgrade, or a change in how a model is packaged. A short, written procedure keeps those changes boring.
A minimal procedure
- Note the operating system, GPU driver, runtime version and model file before changing anything.
- Make one change: a new runtime, a new quantisation, a new context length, or a new model.
- Run the same ten to twenty prompts you always use, in the languages your users actually write in.
- Compare answers and resource use against the previous run.
- Keep or roll back, and record the decision with a date.
Installation choices
Native installs are simplest on a single machine and avoid GPU passthrough issues. Containers help when several services share a host or when you need reproducible deployments; mount model volumes rather than baking multi-gigabyte weights into images. On Windows, WSL2 works well but its virtual disk can grow quietly, so watch free space. On macOS, decide whether you will use llama.cpp builds or MLX and keep the directory layout tidy.
Settings worth documenting
- Default temperature and system prompt for the team, with a reason for any exception.
- Context length and number of parallel slots per model.
- Which features are enabled: tool calls, speculative decoding, vision input.
- Where logs go and how long they are kept.
When something goes wrong
Collect the exact error text, the command or UI action that triggered it, and resource use at that moment. Many reports turn out to be out-of-memory conditions, a wrong prompt template, or a partially downloaded file. Search a shared failure wiki before posting; add the fix once you find it.
A small troubleshooting story
A team reported that their local model suddenly "forgot Chinese" after an update. The cause was not the model at all: the new runtime used a different default prompt template, and the quantised file they had pulled was a different variant from the one they had tested. Rolling back the runtime confirmed the diagnosis. They then pinned both the runtime version and the model checksum, added the template to their configuration file, and re-ran their evaluation prompts before upgrading again. The fix took an afternoon; the lesson was to treat runtimes and model files like any other dependency that needs versioning.
For related guides, see Dify + Ollama on Windows; you may also find MicroAI One software useful.
Bottom line
Local AI rewards patience and record keeping more than raw spending. Decide what the system must do, measure it on your own hardware and documents, write down every change with a date and an owner, and review the setup after each significant upgrade. Small, documented steps are easier to debug, easier to hand over and easier to defend when someone asks why a particular choice was made. When in doubt, test on a smaller scale first and expand only when the evidence supports it.
Quick reminders
- Cap concurrent sessions before lowering quantisation.
- Pin driver and runtime versions together.
- Keep a printed fallback for offline demos.
- Read the model licence before commercial use.
Published by MicroAI One (本地AI 1号网), a Shenzhen team working on local LLM hardware, software and deployment.