Notes on "Role-based model size matrix"
Publish it; update quarterly. Local AI is still a hands-on field. Many of the useful answers come from people who share their exact configuration, what failed and how they fixed it, rather than from polished announcements.
What makes a useful post
- Full specs: CPU, GPU, RAM, storage, operating system, driver and runtime versions.
- The model file and quantisation, with where it came from.
- What you tried, in order, including the failures.
- Measured observations with the method described, never numbers copied from elsewhere.
- Noise, heat and power observations if the box lives in a shared room.
Offline and mobile use
Running small models on phones, tablets or mini PCs is practical for note-taking, translation drafts and offline FAQs. Pick model size by available RAM and leave room for the operating system. Expect heat during long sessions; short prompts and pauses help. For kiosks and offline demos, keep content signed and updates controlled.
Helping newcomers
A new user benefits from a single path: one runtime, one small model, one evaluation set and a checklist. Avoid sending beginners through five tools at once. Encourage them to write down what they did; their notes often become the next person's tutorial.
Keeping a failure wiki
Organise entries by symptom, configuration, fix, owner and date. Titles should match what people search for, such as the exact error message. Review old entries when runtimes change, and mark the ones that no longer apply.
Ways to contribute
- Reproduce someone else's setup and report whether it worked for you.
- Turn a long troubleshooting thread into a short summary with the final fix.
- Translate a useful guide and credit the original author.
- Add measurement methods to posts that only show results.
- Flag outdated tutorials when a runtime or model changes.
Small contributions add up. A community that documents its failures honestly saves everyone time and keeps beginners from repeating the same mistakes.
Further reading from MicroAI One: local AI community, and MicroAI One.
Bottom line
Local AI rewards patience and record keeping more than raw spending. Decide what the system must do, measure it on your own hardware and documents, write down every change with a date and an owner, and review the setup after each significant upgrade. Small, documented steps are easier to debug, easier to hand over and easier to defend when someone asks why a particular choice was made. When in doubt, test on a smaller scale first and expand only when the evidence supports it.
Quick reminders
- Write the date and owner on every change record.
- Test a restore, not just a backup.
- Publish what the assistant will refuse to do.
- Read the model licence before commercial use.
Published by MicroAI One (本地AI 1号网), a Shenzhen team working on local LLM hardware, software and deployment.