TerslerThe little junk rig that could

Source files / docs · raw · rendered

changelog.md

# Changelog

Record every change to the pinned setup (fork commit, driver, CUDA, flags,
model) together with the benchmark result that justified it.

| Date | Change | Bench (tg tok/s puzzle / document) | Notes |
|---|---|---|---|
| 2026-11-03 | Initial pins: Kmic-68/llama.cpp `p100-optimizations` @ `d90ec36a3`, driver 580-server, CUDA 12.9, 150 W | — (hardware not assembled) | reference video: 66.7 / 49.2 |
| 2026-11-03 | First `check-updates` run: fork head `83a7f0839` is 803 commits past the pin (29 on the fork's own line, incl. MTP stop-rule options `LLAMA_SPEC_STOP_ISO`/`_THINK`, q5_K/q6_K matvec changes, upstream merge of 2026-10-24) | — | plan: reproduce the video on the pin first, then bench the head |
| 2026-11-06 | First bring-up on the pin `d90ec36a3`: driver 580.178.04, CUDA 12.9, 150 W, P2P on, model unsloth `Qwen3.8-27B-UD-Q6_K.gguf` (MTP head), flags as in `tersler.conf` | **54.5 / 57.2** (code 56.9; pp document ~5k tok: 412) | `tersler-bench` default (thinking on, 512 tok, 3 runs); draft accept 53–58 %. Thinking off, 4096-tok answers: 52–69, avg 62 (20-min load test). Earlier rows of 2026-11-06 21:5x in results.csv have bogus pp (prompt cache) |
| 2026-11-06 | Pin `d90ec36a3` → `790a3eaa1` (next commit: fixes `GGML_ASSERT(!ggml_cuda_gemm_fold_glu_pending())` on 9–16-token prompts under `-sm tensor`, which crashed the server on short follow-up chat turns) | 54.1 / 55.8 (code 51.7; pp document 415) | same speed as the old pin; 2-turn chat test passes |
| 2026-11-06 | Candidate: fork head `83a7f0839` (803 commits incl. upstream merge 2026-10-24), not pinned | 54.0 / 53.9 (code 54.4; pp document 434, short prompts 166–211 vs 118–132) | decode with thinking off, 1024 tok, 2 runs each, head vs pin: min-heap 77.1 vs 77.2, refrigerator 56.2 vs 57.7, CSV 66.4 vs 67.5 tok/s; same draft acceptance. Prefill +5 % (long) to +40–60 % (short). Fork's perplexity gate not run yet |
| 2026-11-06 | Context `-c 131072` → `262144` (the model's maximum; q4_0 KV, ~18 KiB/token: 16 of 64 layers have attention) | — | VRAM 12.8 → 13.7 GB per card, 2.6 GB free each |
| 2026-11-07 | Test only (not adopted): power limit 150 → 125 W, fan curve holding the hotter card at 75 °C | 150 W: 58.7 / 59.5 (code 57.3; pp doc 406). 125 W: 51.7 / 53.2 (code 48.1; pp doc 361) | load test, 4096-tok answers, thinking off: 61.5 → 57.5 tok/s (−7 %); power 140 → 119 W per card; fan 2490 → 2140 rpm (≈ −3 dB) at the same 74–75 °C |
| 2026-11-07 | Test only (reverted): `LLAMA_SPEC_SAMPLE_TEMP` 1.0 → 0, per the channel's recipe 2.0 | 1.0: 58.7 / 59.5 (code 57.3), draft accept 0.56-0.63. 0: 50.2 / 51.0 (code 45.9), accept 0.43-0.50 | in Kmic's build, T > 0 samples the draft and verifies with the speculative-sampling rule, so T = the target's --temp 1.0 is right; the video's gain was on junkgpulab's own fork |
| 2026-11-07 | Test only (not adopted): `-np 4 --kv-unified`, with and without MTP | production stays `-np 1` with MTP | 4 slots + MTP: OOM at 262k, 6.5 tok/s each at 131k; 4 slots no MTP: 17.4 each / 51 total; same-prefix subagent in another slot re-reads its whole prompt (see research.md) |
| 2026-11-08 | `--cache-ram 3072 -ctxcp 8` (were defaults 8192 MiB / 32) | — | llama-server OOM-killed at 01:29: 14.4 GB anon RSS on 16 GB RAM, no swap; one prompt-cache entry for a 212k-token chat was 6.2 GB. Raise after the RAM upgrade |
| 2026-11-08 | Test only: pin + `server-prompt-cache-keep.patch` (shared prefixes across slots) | — | 3 parallel subagents with the root's 29k system prompt: 196 s → 8.5 s to first token; not in production (needs RAM + a multi-slot config) |
| 2026-11-10 | `--mmproj` (unsloth `mmproj-F16.gguf`) via `MMPROJ_URL` in the local conf | image input works | VRAM after load 14.9 / 13.7 GB, after one image 15.1 / 14.0 GB (of 16.3) |
| 2026-11-10 | RAM 16 → 64 GB (4× 16 GB in A1, A2, C1, C2) | — | B1/C1/D1 hangs POST at 0xB7; 2 channels × 2 DIMMs at 1866 MT/s, triad 20 GB/s; EDAC 0 errors |
| 2026-11-10 | RAM layout A1/A2/C1/C2 → A1/B1/C1/C2 | — | channel D dead (0x55 + 4 long beeps with any stick in D); 3 channels at 1866 MT/s give the same 28.4 GB/s read / 20.2 GB/s triad as 2 channels |
| 2026-11-10 | `--cache-ram 3072 -ctxcp 8` → `16384 -ctxcp 32` (64 GB RAM) | — | context checkpoints are ~150 MiB each; prompt-cache entries can overshoot the limit |
| 2026-11-10 | Serving engine: llama.cpp 27B Q6_K → **Strata** (qr0n/p100-strata `8b1b0f5` + dp4a fix), Flash-Next GSQ-RCO IQ3_S, service `tersler-strata` on port 8080 (`SERVE_ENGINE=strata`) | speed test: decode 56-65 short, 42-50 at 8k, 46-47 at 32k; prefill 330-360 | lean_bench 0.983 vs 0.878 for the 27B (32k cap). No vision; 131k context. The 27B stays installed: `sudo systemctl start tersler-llama` |
| 2026-11-10 | Strata: image input (`strata-vision` on the CPU, ISTA `mmproj-Qwen3.8-Flash-Next-BF16.gguf`) and `--max-context 262144` (32k KV cells in VRAM, the rest in pinned RAM) | image test passes | load 3.5 min; 4,704 experts on GPU 0 (was 4,822); RAM 55 of 62 GB used |
| 2026-11-10 | Renamed junk-rig → **Tersler**: hostname `tersler` (mDNS `tersler.local`), `/opt/tersler` (symlink from `/opt/junk-rig`), `/etc/tersler`, `tersler-*` commands and services (`install/rename-from-junk-rig.sh`) | — | Strata back up in about 4 min, same config |