Instructions to use mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MLX LM

How to use mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit

Run Hermes

hermes

Tool Calling Failure: Qwen3.6-35B-A3B-OptiQ-4bit on oMLX

by thunderhit - opened Apr 27

Discussion

thunderhit

Apr 27

I downloaded mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit today. It worked fine for regular chat conversations, but failed to function with tool calling. I had to switch back to mlx-community/Qwen3.6-35B-A3B-4bit in the end.
My device is a MacBook Pro with M2 Max and 96GB of memory, and I use the oMLX client.The tools I tested include OpenCode and Claude Code.

codelion

MLX Community org May 23

This should be fixed now, try again and let me know if you still have issues.

Optionailed

May 24

Hi.
I get into endless loops on this model with Hermes. Any specific settings I should change in LM Studio?

codelion

MLX Community org May 26

Hi @Optionailed — the endless-loop issue you're seeing is almost certainly an old quant + LM Studio sampler defaults. Since the quants on this repo were pushed, generation_config.json now ships with the Qwen non-thinking instruct recipe (temp=0.7, top_p=0.8, top_k=20, min_p=0.0) — but LM Studio doesn't always read that file. You can try using https://mlx-optiq.com/ it works there

pip install mlx-optiq
optiq serve --model mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit --mtp

Optionailed

May 27

Hi.
Any way to achieve this with LMStudio (prefer to stay, if possible).
I am using Hermes agent, and I sometimes use models you don't have listed on your page.
Thanks!

codelion

MLX Community org May 27

temp=0.7, top_p=0.8, top_k=20, min_p=0.0

Try with these settings does it still give the issue?

codelion

MLX Community org May 27

Btw which other models did you use that don’t have optiq quants in mlx-community?

opnsrcntrbtrian

27 days ago

•

edited 27 days ago

Get below error when using on my MacBook Pro M4 and 48GB of unified memory with oMLX and Claude Code:

API Error: 409 Model 'Qwen3.6-35B-A3B-OptiQ-4bit' is unavailable after a previous load failure: VLM load failed: Missing 1281 parameters:
language_model.lm_head.biases,
language_model.lm_head.scales,
language_model.lm_head.weight,
language_model.model.layers.10.input_layernorm.weight,
language_model.model.layers.10.linear_attn.A_log,
language_model.model.layers.10.linear_attn.conv1d.weight,
language_model.model.layers.10.linear_attn.dt_bias,
language_model.model.layers.10.linear_attn.in_proj_a.biases,
language_model.model.layers.10.linear_attn.in_proj_a.scales,
language_model.model.layers.10.linear_attn.in_proj_a.weight,
language_model.model.layers.10.linear_attn.in_proj_b.biases,
language_model.model.layers.10.linear_attn.in_proj_b.scales,
language_model.model.layers.10.linear_attn.in_proj_b.weight,
language_model.model.layers.10.linear_attn.in_proj_qkv.biases,
language_model.model.layers.10.linear_attn.in_proj_qkv.scales,
language_model.model.layers.10.linear_attn.in_proj_qkv.weight…

Also loading same model in LM Studio gives below error:

Failed to load last used model.

Failed to load model.

Error when loading model: Exception: Encountered fatal exception in the backend generation thread: Traceback (most recent call last):
  File "/Users/<username>/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@32/lib/python3.11/site-packages/mlx_engine/model_kit/batched_vision/model_kit.py", line 360, in _generate_with_exception_handling
    self._generate()
  File "/Users/<username>/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@32/lib/python3.11/site-packages/mlx_engine/model_kit/batched_vision/model_kit.py", line 607, in _generate
    self._load_model()
  File "/Users/<username>/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@32/lib/python3.11/site-packages/mlx_engine/model_kit/batched_vision/model_kit.py", line 224, in _load_model
    self.model = mlx_vlm.utils.load_model(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/<username>/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@32/lib/python3.11/site-packages/mlx_vlm/utils.py", line 663, in load_model
    model.load_weights(list(weights.items()), strict=strict)
  File "/Users/<username>/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@32/lib/python3.11/site-packages/mlx/nn/layers/base.py", line 185, in load_weights
    raise ValueError(
ValueError: Received 37 parameters not in model: 
mtp.fc.weight,
mtp.layers.0.input_layernorm.weight,
mtp.layers.0.mlp.experts.down_proj,
mtp.layers.0.mlp.experts.gate_up_proj,
mtp.layers.0.mlp.gate.biases,
mtp.layers.0.mlp.gate.scales,
mtp.layers.0.mlp.gate.weight,
mtp.layers.0.mlp.shared_expert.down_proj.biases,
mtp.layers.0.mlp.shared_expert.down_proj.scales,
mtp.layers.0.mlp.shared_expert.down_proj.weight,
mtp.layers.0.mlp.shared_expert.gate_proj.biases,
mtp.layers.0.mlp.shared_expert.gate_proj.scales,
mtp.layers.0.mlp.shared_expert.gate_proj.weight,
mtp.layers.0.mlp.shared_expert.up_proj.biases,
mtp.layers.0.mlp.shared_expert.up_proj.scales,
mtp.layers.0.mlp.shared_expert.up_proj.weight,
mtp.layers.0.mlp.shared_expert_gate.biases,
mtp.layers.0.mlp.shared_expert_gate.scales,
mtp.layers.0.mlp.shared_expert_gate.weight,
mtp.layers.0.post_attention_layernorm.weight,
mtp.layers.0.self_attn.k_norm.weight,
mtp.layers.0.self_attn.k_proj.biases,
mtp.layers.0.self_attn.k_proj.scales,
mtp.layers.0.self_attn.k_proj.weight,
mtp.layers.0.self_attn.o_proj.biases,
mtp.layers.0.self_attn.o_proj.scales,
mtp.layers.0.self_attn.o_proj.weight,
mtp.layers.0.self_attn.q_norm.weight,
mtp.layers.0.self_attn.q_proj.biases,
mtp.layers.0.self_attn.q_proj.scales,
mtp.layers.0.self_attn.q_proj.weight,
mtp.layers.0.self_attn.v_proj.biases,
mtp.layers.0.self_attn.v_proj.scales,
mtp.layers.0.self_attn.v_proj.weight,
mtp.norm.weight,
mtp.pre_fc_norm_embedding.weight,
mtp.pre_fc_norm_hidden.weight.

codelion

MLX Community org 27 days ago

•

edited 26 days ago

There were two issues:

LM Studio Received 37 parameters not in model (the mtp.* tensors): those are optiq mtp.safetensors speculative-decoding head, which used to sit at the repo root where a *.safetensors glob picked it up. They were moved it into optiq/ on 11 July. You can delete the local copy and re-download.

oMLX Missing N parameters ... vision_tower.*: this is due to mlx-vlm only loading the shards named in model.safetensors.index.json, and the optiq bf16 vision tower sat in a sidecar the index never listed, so the loader built a vision_tower with no weights. This is now fixed in the quant. After re-downloading, mlx-vlm / oMLX / LM Studio can load the vision tower, mlx-lm text is unchanged, and mtp.safetensors stays hidden.

Note vision may still not generate correctly in oMLX: mlx-vlm 0.6.4 garbles a stock (non-OptiQ) Qwen3.5 quant too, which matches oMLX #1464 and #72. The change removes the load error. For text+images you can use, optiq serve --model <repo> which gives an OpenAI-compatible endpoint that accepts images.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment