prefill
layer 0 of 64
Gated DeltaNet · ffn_gate · pn3t
decode, one P100
30.03 tok/s
11.22 GiB of weights read per token
Custom engine and quantization for the Tesla P100
tensorless runs Qwen3.8-27B on an NVIDIA Tesla P100 at 30 tokens a second, in one command. The model is tuned for the card, and pinned to the engine it was tuned with.
curl -fsSL https://tensorless.dev/install.sh | shtensorless run iamMrDuncan/Qwen3.8-27B-PN3-GGUFWhat you're looking at · every dot is real
The PN format
A model is billions of numbers, its weights. Stored the usual way, at 16 bits each, Qwen3.8-27B needs 50.1 GiB, three times what the P100 holds. PN stores most of them in 3 bits, in a layout made for this card, and the model the engine loads comes to 11.73 GiB.
1 · Groups of 80, one ruler each
These are the first 80 weights of layer 0's ffn_gate, read straight from the file. All 80 share one scale, so each weight only has to say which of 8 marks it sits on: 3 bits. The scale and the 80 codes fill exactly 34 bytes.
2 · Turn the input first
A ruler wide enough for an outlier rounds everything else to zero. So before the P100 reads PN weights, the engine flips some signs (the file lists which) and mixes pairs of values, round after round, until the outlier is shared by all of them. The weights were stored already turned, so the answer comes out the same. Shown with 16 values, the file's first 16 signs and the real transform; the engine's has 256 values and 8 rounds, 4 times in every layer.
d, fp16, signed:
bits 0–15 · even codes
bits 16–31 · odd codes
all 80 weights of this block, decoded
Where the 16 GB go
Only 16 of the 64 layers keep a KV cache, the memory of earlier tokens that grows with the context. The other 48 are Gated DeltaNet layers: each keeps a small fixed-size state instead, the same at the first token and the 65,536th. That is how the whole context fits next to the weights.
Weights
11.73 GiB
The main model's tensors, as stored in the file: pn3t 6.07 GiBpn4t 4.45 GiBQ4_K 0.67 GiBQ3_K 0.51 GiBQ8_0 0.02 GiBF32 0.01 GiB
KV cache
1.13 GiB
16 attention layers × 65,536 positions × K and V, 4 heads of 256, at q4_0, reserved up front.
DeltaNet state
0.15 GiB
48 layers, each 48 heads × a 128 × 128 matrix in f32, plus a short convolution state: the same size at any context length.
Everything else
1.24 GiB
The CUDA context and compute buffers: what the measured total leaves after the three above.
Weights are summed from the file's header. The KV cache and the state are computed from the model's shape and the recipe's settings. The totals, 13.67 GiB idle and 14.24 GiB generating, were measured on the P100 that ran the release's receipt. The MTP layer in the file is not loaded by this recipe.
Measured
Before a recipe is marked released, its exact image runs on a real P100 and tensorless bench measures it. This is the receipt of v0.1.0's main target. The digest it measured is the digest the recipe pins, promoted without a rebuild.
The engine itself came out of an automated hill climb: 131 judged attempts, 28 confirmed improvements, taking this model from 7.74 to 30.06 tokens a second on the same card, with every step checked against the model's starting numerics.
TENSORLESS BENCH
2026-11-02 · ONE TESLA P100-PCIE-16GB
IMAGE0.1.0-sm60
DIGEST35409ff97c2f…
MODELQWEN3.8-27B-PN3
FILE SHA25615ce2d446967…
PROMPT429 TOKENS
GENERATE256 TOKENS, GREEDY
RUN 130.03 TOK/S
RUN 229.83 TOK/S
RUN 330.13 TOK/S
MEDIAN30.03 TOK/S
RESEARCH RECEIPT30.06 TOK/S
DIFFERENCE−0.1%
FIRST TOKEN4.46 S
GPU MEMORY14.24 GIB
LOAD TO LISTENING3.6 S
Get started
1Install it
curl -fsSL https://tensorless.dev/install.sh | shOne static binary in ~/.local/bin, checked by sha256 against the release.
2Check the machine
tensorless doctorThe card, the driver, podman or docker, NVIDIA container support and disk space, each with the command that fixes it.
3Run the model
tensorless run iamMrDuncan/Qwen3.8-27B-PN3-GGUFPulls the image by digest and the model by revision, checks both, and serves on 127.0.0.1:8080.
4Use it
curl http://127.0.0.1:8080/v1/chat/completions ...An OpenAI-compatible API. The model's name is its Hugging Face repository.
tensorless runs on your own machine. There is no account, and the launcher sends no telemetry: it talks only to Hugging Face for the model and to the registry for the image.
Launch targets
Each target is a recipe: one model on one card, with the exact image, file and settings that were measured. Name it by its Hugging Face repository, or by its short name.
Qwen3.8-27B in PN, our P100-native weight format: pn3t on most weight matrices, pn4t where they need it, and 19 more raised to pn4t (the mix P0k+19).
tensorless run iamMrDuncan/Qwen3.8-27B-PN3-GGUFUnsloth's UD-IQ3_XXS of Qwen3.8-27B with its 91 sub-3-bit tensors raised to IQ3_XXS: a 3-bit floor, in standard llama.cpp types. The model the P100 kernels were first tuned on.
tensorless run iamMrDuncan/Qwen3.8-27B-UD-IQ3_XXS-Q3Floor-GGUFDocumentation
Four kinds of pages, one job each. Every command, flag and number in them is checked against the code and the receipts.
Tutorials
From a bare P100 host to a model answering your first request, then your card measured against the receipt.
How-to guides
Prepare a host, serve in the background, connect a client, keep models on another disk, fix a failed start.
Reference
Every command and flag, the recipe file, the engine image and its 21 switches, the PN format and the receipts.
Explanation
How the pieces fit, why the P100, where the engine came from, and why you can trust the numbers.
For agents: /llms.txt lists every page, and /llms-full.txt holds them all. Questions, or results from your own P100: join the Discord.