TerslerThe little junk rig that could

Source files / docs · raw · rendered

recreate.md

# Rebuilding Tersler from scratch

These are all the steps from parts to a running server, in order. Each step links to the doc with the
details. Everything needed is in this repository. The only things not included are the model weights
(downloaded by the installer) and two vendor PDFs (see [Reference documents](#reference-documents)).

Tersler's working title was *junk-rig*.

## 1. Parts

- Buy the parts in the [bill of materials](bom.md): about €620 in total: €250 for the
  pair of Tesla P100 16 GB cards, ~€350 for the board, CPU, cooler, 64 GB RAM, SSD, PSU and fan, and a used case.
- [research.md](research.md) explains why each part was chosen, and what to check if you substitute one.
  Most importantly, the board must support Above 4G Decoding and two x16 Gen3 slots on one CPU.
- RAM: start with one 16 GB RDIMM in A1. The full setup has 4× 16 GB (this rig: A1, B1, C1, C2, because its channel D doesn't work; normally one per channel)
  ([hardware notes](hardware-notes.md#ram-placement)).

## 2. Assembly

Follow [hardware-notes.md](hardware-notes.md):

1. **Without the GPUs first:** CPU, cooler, RAM in A1, PSU and onboard VGA. Check that it POSTs.
2. **BIOS:** Above 4G Decoding on, VT-d on ([BIOS settings](hardware-notes.md#bios-settings-set-before-installing-the-gpus)).
3. **Cards** in PCIE6 and PCIE4 (both x16 Gen3). Keep PCIE5 and PCIE3 empty.
4. **Power:** one card on the PSU's native EPS cable, the other on the 2× PCIe 6+2 → EPS Y-adapter
   that came with the cards, with **both** legs connected ([power](hardware-notes.md#power)). The P100
   takes a CPU-style EPS 8-pin plug, not PCIe. If your cards come without adapters, you can make one
   ([wiring](hardware-notes.md#homemade-2-pcie-62--eps-8-pin-adapter)); check the pinout with a meter
   before you plug it in.
5. **Cooling:** passive cards need forced air. Use one 120 × 38 mm fan and a printed duct
   (`shroud/p100_front_duct.scad` for a tower case, `shroud/p100_shroud.scad` for an open frame).
   The board's fan header is then driven by GPU temperature ([cooling](hardware-notes.md#cooling)).

## 3. Operating system

Build a ready-to-boot Ubuntu 24.04 image on any Linux PC with KVM, then write it to the SSD with `dd`.
See [image/README.md](../image/README.md).

```bash
image/build-image.sh
zstdcat image/out/tersler.img.zst | sudo dd of=/dev/sdX bs=4M conv=fsync status=progress
```

The rig boots into SSH (key only). The repository is already in `~/tersler`.

## 4. Install

On the rig:

```bash
cd ~/tersler
cp install/tersler.local.conf.example install/tersler.local.conf   # set MODEL_URL etc.
sudo ./install/install.sh          # all stages; reboot when told, then run it again
sudo ./install/install.sh verify   # driver 580, BAR1 16 GB each, x16 links, P2P, iommu=pt
sudo systemctl start tersler-llama
```

The installer runs these stages: `preflight config base driver cuda kernel build gpu model services verify`.
Each stage can be run on its own (`sudo ./install/install.sh build`). `--list` shows their state.

| Stage | What it does |
|---|---|
| driver | `nvidia-headless-580-server`, held with `apt-mark`. R580 is the last branch with Pascal support. |
| cuda | CUDA toolkit 12.9 from NVIDIA's repo (toolkit only). CUDA 13 dropped Pascal. |
| kernel | `iommu=pt` (needed for GPU peer-to-peer) |
| build | Kmic-68/llama.cpp `p100-optimizations` at the pinned commit, `sm_60` only, into `/opt/tersler/builds/<commit>` |
| gpu | persistence mode and a 150 W power limit at every boot |
| model | downloads the GGUF (and the vision mmproj, if set) and checks the sha256 |
| services | `tersler-llama`, `tersler-gpu` and `tersler-fan` systemd units |

All versions and server flags are pinned in [install/tersler.conf](../install/tersler.conf). The rig's
own overrides (model, vision encoder, fan mapping, LAN access) are in
[install/tersler.local.conf](../install/tersler.local.conf).

## 5. Tune and check

- `tersler-fan-control --discover` finds the board's PWM header for the GPU fan. Put it into the local conf.
- `tersler-loadtest` runs 20 minutes of sustained load. Both cards should stay below ~75 °C.
  The P100 slows down at 82 °C and shuts down at 85 °C.
- `tersler-bench` should give ~55-59 tok/s decode on the reference prompts
  ([changelog](changelog.md), [benchmarks](../bench/README.md)).

## 6. Keep it current

`tersler-check-updates` reports new fork commits, driver/CUDA updates and model updates. Build a candidate
next to the running one, benchmark it, and only then move the pin ([keeping-up.md](keeping-up.md)). Record
every change with its benchmark in the [changelog](changelog.md).

## Reference documents

[hardware-notes.md](hardware-notes.md) cites section numbers from two vendor PDFs. They are not
redistributed here. Get them from the vendors:

- ASUS Z10PA-U8 user guide (E13693): ASUS support site, Z10PA-U8, Manual & Document
- Dell Precision T5810 spec sheet: Dell support site, Precision T5810, Documentation
  (the reference build's platform, for comparison only)