Skip to content
tensorlessv0.1.0

Credits

This page is CREDITS.md from the repository, which is the source: a change goes there first.

tensorless stands on a great deal of other people’s work: an inference engine several forks deep, open models and quantizations, P100 owners who published their patches and numbers, and the papers behind the format, the kernels and the way we measured them. This page thanks them.

License obligations for the code tensorless ships are in NOTICE, THIRD_PARTY_NOTICES.md and LICENSES/. Model licenses are in each model’s card. If we missed you, or got something wrong, open an issue or tell us on Discord.

The code tensorless ships

Section titled “The code tensorless ships”
  • llama.cpp / ggml, Georgi Gerganov and contributors. The foundation of the engine, plus its quantization formats (K-quants and i-quants), llama-quantize, the importance matrix, llama-server and the test tools. MIT.
  • llama-cpp-turboquant, TheTom: TurboQuant KV-cache types and native MTP, the base of llamAmpere. MIT.
  • llamAmpere, JakeATX: the fork our engine starts from (2cb16936), with its EXL3 GGUF path and Qwen3.8 work. MIT.
  • exllamav3, turboderp: the EXL3 format, whose code the engine uses for its EXL3 weight types. MIT.
  • Vendored libraries: cpp-httplib, nlohmann/json, miniaudio, stb_image, subprocess.h, xxHash, SHA-1, SHA-256 and rotate-bits. See THIRD_PARTY_NOTICES.md.
  • BurntSushi/toml, used by the launcher. MIT.

Models and quantizations

Section titled “Models and quantizations”
  • The Qwen team (Alibaba): Qwen3.8-27B. The base model of every launch target. Apache-2.0.
  • Unsloth: unsloth/Qwen3.8-27B-GGUF. Qwen3.8-27B-UD-IQ3_XXS-Q3Floor is mostly their work:
    • about 73% of its bytes are their UD-IQ3_XXS, unchanged;
    • its token_embd is from their UD-IQ4_XS;
    • its re-quantized tensors come from their BF16 GGUF, quantized with their importance matrix. Their quantization ladder was also our main comparison, and their Dynamic 3.0 notes gave us the trajectory-divergence check we used to judge quantization quality.
  • turboderp/Qwen3.8-27B-exl3: the EXL3 quantizations we measured on the P100.
  • bartowski and AtomicChat: Qwen3.8-27B GGUF ladders we compared against.
  • Drafters we measured for speculative decoding on the P100: z-lab/Qwen3.8-27B-DFlash2 (with DFlash), incoai’s DFlash2 builds, and the RadixArk and Anbeeld DSpark drafters.

P100 and Pascal know-how from the community

Section titled “P100 and Pascal know-how from the community”

Engines and projects we learned from

Section titled “Engines and projects we learned from”
  • TensorFold, ashhart:
    • Its exact speculative decoding, in-place regrouping and chain kernels shaped our kernel and speculation work.
    • Its recipe harness was the evaluation baseline for the PN quantization.
    • Its pull / serve / info / capabilities command line shaped the tensorless launcher.
  • Lucebox, Luce-Org: DFlash with DDTree on the RTX 3090, and a hybrid DeltaNet megakernel.
  • 0xSero/qwen38-b70 (Qwen3.8-27B on Intel Arc Pro B70) and 0xSero/trellis-serve (EXL3 trellis kernels on the RTX 3090).
  • MiaAI-Lab’s one-click Qwen3.8-27B installer: an EXL3 configuration for 16 GB cards.
  • exllamav2, for reconstructing to fp16 for wide batches. SGLang and vLLM, for their Qwen3.8 and hybrid-model implementations; vLLM also ran our structured-output evaluation.
  • llama.cpp pull requests and discussions we built on:
    • the fused gated-delta-net kernel (#19504);
    • Hadamard rotation for the quantized KV cache (#21038);
    • deterministic mode (#16016);
    • DFlash2 support (#27342);
    • the CUDA graphs work (#6766);
    • the KV-cache quantization KLD tables (discussion #23470).

Papers

Section titled “Papers”

Quantization: the PN format and its quantizer

Section titled “Quantization: the PN format and its quantizer”

Measuring quality and exactness

Section titled “Measuring quality and exactness”

Speculative decoding

Section titled “Speculative decoding”

Kernels and systems

Section titled “Kernels and systems”
  • Flash-Decoding (Dao, Haziza, Massa, Sizov): the split-KV idea behind our P100 decode-attention kernel. FlashAttention (Dao et al.), 2205.14135.
  • DeepSeek-V3, 2412.19437: promoting partial sums to fp32 at fixed intervals.
  • GPU synchronization microbenchmarks (Zhang et al.), 2004.05371: the cost of a grid barrier on the P100.
  • Ideas we evaluated:

Data

Section titled “Data”

The PN quantization was calibrated and evaluated on these datasets; tensorless does not redistribute any of them.

Hardware documentation

Section titled “Hardware documentation”

NVIDIA:

  • the Pascal Tuning Guide;
  • the Tesla P100 whitepaper and datasheet;
  • the CUDA C Programming Guide, Math API and cuBLAS documentation;
  • the CUDA 13 release notes, which set our CUDA 12.x floor for Pascal;
  • the Container Toolkit’s CDI guide.

Tools, harnesses and the agents that did the work

Section titled “Tools, harnesses and the agents that did the work”
  • Thermite: the automated research harness that ran the PN campaigns: a judge the agents can’t see, held-out checks, A/A noise calibration, an auditor, and the decode hill climb (131 judged attempts, 28 confirmed wins).
  • pi-autoresearch (davebcn87) and the Pi coding agent: the experiment loop of the earlier P100 kernel campaigns. pod-pi: their rootless podman workspace.
  • NVIDIA’s nvprof, compute-sanitizer, cuobjdump and nvdisasm, which still support sm_60.
  • PyTorch, transformers, vLLM, llm-compressor, compressed-tensors and flash-linear-attention, for the quantization pipeline. LiteLLM, for model routing. Hugging Face, for hosting the models.
  • The models that worked as researchers:
    • GLM-5.3 Flash, DeepSeek V4.1 Flash, Ornith-1.5-122B and Claude Haiku 5.5 as workers;
    • GPT 6.1 Sol and Claude Opus 5.5 as teachers and auditors;
    • Claude Sonnet 5.5 as planner;
    • Claude Code and Codex for supervision, research and this repository.