---
license: apache-2.0
library_name: llama.cpp
base_model:
- migtissera/Tess-4-27B
base_model_relation: quantized
pipeline_tag: text-generation
quantized_by: ROCmFPX
language:
- en
tags:
- gguf
- rocm
- amd
- strix-halo
- gfx1151
- rocmfpx
---

# Tess-4-27B-ROCmFPX-GGUF

> ## ⚠️ These files do NOT load on standard llama.cpp
> They use AMD-native `*_ROCMFPX` tensor types from the experimental
> [ciru-ai/ROCmFPX](https://github.com/ciru-ai/ROCmFPX) llama.cpp fork (build from source).

Derivative of [Tess-4-27B](https://huggingface.co/migtissera/Tess-4-27B), quantized to AMD-native [ROCmFPX](https://github.com/ciru-ai/ROCmFPX) formats (fork-only) tuned for Strix Halo (gfx1151).

## Base Model

This is a derivative of [Tess-4-27B](https://huggingface.co/migtissera/Tess-4-27B).
All credit for the base model architecture and weights goes to the original authors.
The base model's license applies to this derivative.

## ROCmFPX (AMD-native, fork-only)

These GGUFs use AMD-native quantization schemes from the experimental
**[ciru-ai/ROCmFPX](https://github.com/ciru-ai/ROCmFPX)** llama.cpp fork,
tuned for and benchmarked on AMD Strix Halo (Radeon 8060S iGPU, gfx1151, unified memory):

- `ROCmFP3/4/6/8` tensor types with straight and "agent" presets (agent presets keep
  tool-calling / JSON-structured output reliable at low bit-widths)
- Files load **only** on the fork -- build it from source. Known-good commit these files were built and validated with:

```bash
git clone https://github.com/ciru-ai/ROCmFPX && cd ROCmFPX
git checkout 221402af8574faf652b101b6afe225a3f329561f
```

## GGUF Files

| File | Size | Quant |
|------|------|-------|
| [Tess-4-27B-Q3_0_ROCMFPX.gguf](./Tess-4-27B-Q3_0_ROCMFPX.gguf) | 15.1 GB | ROCmFP3 (fork-only) |
| [Tess-4-27B-Q3_0_ROCMFPX_AGENT.gguf](./Tess-4-27B-Q3_0_ROCMFPX_AGENT.gguf) | 19.5 GB | ROCmFP3 (fork-only), agent preset |
| [Tess-4-27B-Q4_0_ROCMFP4.gguf](./Tess-4-27B-Q4_0_ROCMFP4.gguf) | 17.7 GB | ROCmFP4 (fork-only) |
| [Tess-4-27B-Q4_0_ROCMFP4_COHERENT.gguf](./Tess-4-27B-Q4_0_ROCMFP4_COHERENT.gguf) | 15.7 GB | ROCmFP4 (fork-only), coherent preset |
| [Tess-4-27B-Q6_0_ROCMFPX.gguf](./Tess-4-27B-Q6_0_ROCMFPX.gguf) | 22.5 GB | ROCmFP6 (fork-only) |
| [Tess-4-27B-Q6_0_ROCMFPX_AGENT.gguf](./Tess-4-27B-Q6_0_ROCMFPX_AGENT.gguf) | 25.3 GB | ROCmFP6 (fork-only), agent preset |
| [Tess-4-27B-Q8_0_ROCMFPX.gguf](./Tess-4-27B-Q8_0_ROCMFPX.gguf) | 28.2 GB | ROCmFP8 (fork-only) |
| [Tess-4-27B-Q8_0_ROCMFPX_AGENT.gguf](./Tess-4-27B-Q8_0_ROCMFPX_AGENT.gguf) | 28.7 GB | ROCmFP8 (fork-only), agent preset |

## Usage

Requires a from-source build of the [ROCmFPX fork](https://github.com/ciru-ai/ROCmFPX)
(stock llama.cpp, LM Studio, and Ollama cannot load these files):

```bash
# Interactive chat
llama-cli -m Tess-4-27B-Q3_0_ROCMFPX.gguf -c 8192 -cnv

# Server mode
llama-server -m Tess-4-27B-Q3_0_ROCMFPX.gguf -c 8192 --port 8080 -ngl 99 -fa on
```

## Serving: MTP Speculative Decoding

This model includes **MTP ("nextn") draft tensors**, enabling self-speculative
decoding -- measured **~1.6-1.9x faster generation** with a ~95% first-token
accept rate (no separate draft model needed; it drafts from itself):

```bash
llama-server -m Tess-4-27B-Q3_0_ROCMFPX.gguf -c 8192 --port 8080 --host 127.0.0.1 -ngl 99 -md Tess-4-27B-Q3_0_ROCMFPX.gguf --spec-type draft-mtp -ctk q8_0 -ctv q8_0 -fa on
```

**Memory cost:** MTP needs its own draft context alongside the main context,
so serving with it uses roughly **2x the model's memory** compared to serving
without ``-md``/``--spec-type draft-mtp``.

## Caveats

- The base model's license (apache-2.0) applies to all derivative files
- **Fork-only files**: stock llama.cpp, LM Studio, and Ollama cannot load these -- build [ciru-ai/ROCmFPX](https://github.com/ciru-ai/ROCmFPX) from source
- Quantization reduces precision -- verify outputs for your specific use case

## Limitations

- Quantized models may exhibit subtle differences from the full-precision fine-tune
- This model inherits any limitations and biases present in the base model

---
*Generated with Foundry*