fp128-emu

fp128-emu is a quad-precision (double-double, ~107-bit significand) matrix multiply on INT8 tensor cores, loadable through kernels. GPUs have no quad-precision hardware and the usual route is CPU software double-double (MPLAPACK-class, single-digit GFLOP/s); this kernel reaches 15.5 quad-TFLOP/s at n = 16384 on RTX PRO 6000 Blackwell, at the double-double rounding floor on every distribution tested, and bit-exact on inputs representable at their row scale.

Double precision runs out at 53 bits of significand, and problems with condition numbers past ~10^16 return answers with no correct digits. This kernel carries ~107 bits by representing every value as a pair of doubles and computing the product exactly in integer residue arithmetic on the INT8 tensor cores, so ill-conditioned products that are meaningless in FP64 come back correct, at tensor-core speed rather than CPU software speed.

H @ H^-1 for the order-20 Hilbert matrix, condition ~6x10^28. Native FP64 (left) produces off-diagonal error reaching 10^11 times the answer scale; fp128-emu (right) returns L*I exactly, torch.equal against the integer target, computed live along with the measured 3.5 quad-TFLOP/s at n = 4096.

Usage

import torch
from kernels import get_kernel

emu = get_kernel("phanerozoic/fp128-emu", version=1, trust_remote_code=True)

# quad-accuracy product of fp64 matrices (B given as [N, K])
A = torch.randn(4096, 4096, dtype=torch.float64, device="cuda")
B = torch.randn(4096, 4096, dtype=torch.float64, device="cuda")
C_hi, C_lo = emu.mm(A, B)               # C = A @ B^T, ~107 correct bits

# full double-double operands
Chi, Clo = emu.mm_dd(A_hi, A_lo, B_hi, B_lo)

version selects the release branch; trust_remote_code is required by kernels for publishers without the trusted-publisher mark. The prime count is chosen automatically from K per the range rule; torch._int_mm shape constraints (M > 16; K, N multiples of 8) are handled by exact zero-padding.

API

Method

Values are double-double pairs v = hi + lo. Each row of A and row of B ([N, K] layout) is scaled by a power of two so its largest element lands in [2^105, 2^106); every element then splits exactly into T = T1*2^53 + T0 (both int64) plus a quantization residual of at most one unit:

u  = hi * 2^s                exact (power-of-two scale)
T1 = nearbyint(u * 2^-53)    |T1| <= 2^53
w  = u - T1*2^53             exact (both multiples of ulp(u) <= 2^53)
z  = w + lo * 2^s            one rounding, error <= 1/2
T0 = nearbyint(z);  r = z - T0

For each prime p <= 251 the balanced residue bal(bal(T0) + bal(T1)*(2^53 mod p)) fits int8, and one torch._int_mm tensor-core GEMM computes C mod p. The prime count is linear in the precision (Ozaki-II / CRT). Two additional INT8 GEMMs over 7-bit quantizations of T and r recover the first-order quantization error to ~2^-6, added at scale 2^94; the correction vanishes identically on exactly representable inputs. Balanced-Garner reconstruction produces mixed-radix digits, a Horner evaluation in 256-bit two's-complement integer arithmetic recovers the exact sum, and canonical double-double rounding produces the result. No FP64 arithmetic appears in the accumulation path.

Range rule: the reconstructed dot product must stay inside the signed CRT range, worst case 214 + ceil(log2 K) bits. 32 primes (~2^233.5) cover K = 2^17, which is also the int32 accumulator bound (K * 125^2 < 2^31).

Measured

Throughput (square n, random normal fp64 inputs, quad-TFLOP/s = 2n^3/time):

A vectorized CPU double-double GEMM (numpy, Dekker two-prod) measures 0.11 GFLOP/s on the same host; reference CPU double-double BLAS (MPLAPACK) reports single-digit GFLOP/s on many-core systems.

Accuracy against a 250-bit mpmath oracle (worst case over the output, max-norm; identical to the last digit on all three GPUs):

The double-double format carries ~106-107 bits: the kernel sits at the output rounding floor.

Correctness

On inputs exactly representable at their row scale the result is bit-identical to the true product, verified by torch.equal against bigint oracles: random integers, 90-bit integers as dd pairs (products to 2^185), 100-bit dyadic values with per-row exponents, and the Hilbert certificate. The order-20 Hilbert matrix scaled by L = lcm(1..39) is an integer matrix (53-bit entries) whose exact inverse has integer entries to 92 bits; fp128-emu computes H @ H^-1 = L*I exactly, through cancellation of ~2^155-magnitude terms, on every gated build. At n = 24 the inverse entries reach 111 bits, past what the double-double format can carry; that marks the edge of the format rather than the kernel. The pipeline is deterministic.

Requirements and limits

• NVIDIA GPU with compute capability 8.0+ and torch with _int_mm.
• float64 tensors (dd = pairs); K <= 2^17.
• Working memory ~34 primes x (MK + NK) int8 plus P x M*N int32: about 50 GB at n = 16384 (fits 80 GB+ cards; L4-class cards up to n ~ 4096).
• Accuracy semantics: a shared per-row 106-bit window anchored at the row maximum, the double-double analogue of row-max scaling. Within-row magnitude spread beyond trailing-zero slack is quantized at ~2^-106 of the row maximum and first-order corrected, matching classical double-double GEMM accumulation.

References

Ozaki, Ogita, Oishi, Rump 2012 (Numerical Algorithms 59); Ozaki, Uchino, Imamura 2025 (Ozaki Scheme II, arXiv:2504.08009; GEMMul8, RIKEN-RCCS); Dekker 1971 (double-double arithmetic); Nakata, MPLAPACK (CPU multiprecision BLAS reference).

License

Apache-2.0.