arxiv:2607.02770

Gemma 4 Technical Report

Published on Jul 2

· Submitted by

Omar Sanseviero on Jul 8

Google

Upvote

Authors:

Abstract

Gemma 4 introduces efficient, multimodal language models with diverse architectures, enhanced reasoning capabilities, and improved performance across various tasks.

Generated by Qwen/Qwen2.5-Coder-32B-Instruct

We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemma 4 model suite features dense and Mixture-of-Experts architectures, ranging from 2.3B to 31B parameters. Alongside improved vision and audio encoders for all model sizes, we propose a unified, encoder-free architecture for our 12B model, which ingests raw audio and image patches. Furthermore, we integrate a thinking mode, enabling Gemma models to generate reasoning traces prior to responding. We improve inference speed, memory, and compute efficiency, as well as long-context abilities through critical design choices. Gemma 4 establishes a leap in performance across STEM, multimodal, and long-context benchmarks, and rivals larger, frontier open models in human-rated tasks.

View arXiv page View PDF Add to collection

Community

shadowlilac

3 days ago

Interesting report. Unfortunately it looks like there has been almost no disclosure of any detail, unlike Gemma 3's technical report, and instead standard well-known procedures were simply explained, with almost no input, ablation, negative findings, positive findings, scaling laws, etc...