Local LLM ops checklist lp213

This English operating checklist helps local LLM teams keep hardware tables, runtimes, and refuse policies aligned before scaling seats. Define a named owner and a backup who can approve exceptions. Capture a fourteen-day baseline for failed pulls, OOM count, and mean latency on a fixed prompt. Set a kill switch that can pause the API without a meeting. Pilot on a constrained model size or a single team first so failures stay contained. Align every public claim with the hardware matching table. Keep refuse macros ready for medical, legal, and payroll questions. Review reliability after power cost, admin time, and eval cost before declaring a win. Archive the runbook with a date stamp so the next hire does not reinvent tribal knowledge.

Detailed steps

1. Name the owner and success metric before any model pull.
2. Document the baseline from the last two weeks with screenshots.
3. Ship a pilot with a kill switch; do not scale on vanity demos alone.
4. Capture screenshots and ticket IDs for every exception path.
5. Review eval fitness and data-class compliance with stakeholders.
6. Confirm VRAM headroom matches advertised context lengths.
7. Update refuse macros the same day capabilities change.
8. Revisit after the next model bump and delete steps that never changed outcomes.

Extra field notes

Shared GPUs amplify small mismatches. A demo that needs 24GB while the office box has 12GB free will create tickets faster than prompt optimization can fix. Hardware, software, and ops should sign the same one-pager covering scope, metric, kill switch, and review date. If any team cannot name the owner in thirty seconds, the play is not ready. Keep evidence folders by change ID for configs, eval outputs, and power logs. Prefer honest VRAM tables over optimistic copy.

仅密钥、防爆破、禁密码 root。

为什么 2026 还要写这张清单

本地大模型团队最常见的损耗，是硬件表、运行时与政策页互相打架。这篇给你一份下午就能跑完的检查表，不编造跑分。请把它当成操作文件：每一步都要留下下周别人能审计的产物。

共享显卡会放大小错位。演示假设 24GB，办公室机器只剩 12GB 空闲，工单速度会快过任何提示词优化。扩席位之前，先把硬件匹配表、安装说明与拒答策略对齐。

前置条件

• 玩法责任人与可批异常的备份各一名。
• 能打开推理机、模型仓库与评测集。
• 近十四天基线：拉取失败、OOM 次数、固定提示的平均延迟。
• 止损开关：谁能不经会议暂停 API。

步骤

1. 先写清责任人与成功指标，再谈采购或拉模型。
2. 记录近两周基线，方便诚实对比。
3. 小范围试跑并设止损，不凭演示截图放量。
4. 每个异常路径留截图与工单号。
5. 看稳定性与数据分级合规，再宣布胜利。

试跑设计

先在受限模型规模或单一团队试跑。给并发设顶，让失败可控。改设置前后各留截图。若评测适配（不是截图）变好，再分批放量，并每次复核显存余量。

预先写好的失败模式

• 模型卡按你扣除系统开销后并不存在的显存来写。
• 英文博客看起来很好的量化，在中文评测集上翻车。
• 拒答宏未发布，同事现场编造能力边界。
• 财务看到显卡发票却没有墙功耗日志。
• 市场部把 NPU 小模型演示说成可本地跑 70B。

现场备注

• 手册只设一个责任人，交接禁止发明新规则。
• 每次对外承诺都要对齐硬件匹配表与安装路径。
• 医疗、法律、薪酬问题提前备拒答宏。
• 复盘本地推理时计入电费、管理工时与评测成本。
• 手册版本注明日期归档，避免下个新人再靠口口相传。

交接清单

硬件、软件、运维签同一页：范围、指标、止损、复盘日。任何团队三十秒说不出责任人，就还没准备好。下个模型升级后复盘，删掉从未改变结果的步骤。

参考

本地 AI 实用笔记：DIY 实验室、本地AI 1号网、显存速查表。

作者：本地AI 1号网，深圳团队，专注本地大模型软硬件、装机与私有化部署。