All models

Qwen / LANGUAGE MODEL

Qwen3.5 27B VRAM requirements

Explore memory tradeoffs with sourced model data and transparent estimates.

Q4_K_M estimate

19.1 GiB

at 8,192 tokens

FP16 estimate

59.4 GiB

at 8,192 tokens

Suggested GPU

1 × RTX 4000 Ada Generation 20 GiB

Model architecture

Model computation diagram
  1. Token embedding · 5120
  2. 64 layers · GQA · hybrid
  3. Output head → token scores
Parameters27,781,427,952 (27.78B)
Layers64
Attention heads24
KV heads4
Head dimension256
Hidden size5120
Maximum context262,144 tokens
KV cache layers16 of 64 layers retain KV cache; hybrid attention.
NotesHybrid attention: only 16 of 64 layers are full-attention layers with a KV cache (the rest are Gated DeltaNet linear-attention layers). Parameter count includes the vision encoder and MTP head shipped in the checkpoint.

Minimum hardware and family comparison

Estimates use batch 1, an FP16 KV cache and 12% headroom. Multi-GPU capacity assumes even sharding and requires a compatible runtime.

DevicesRequired totalSmallest fitting device exampleUsable per device
119.13 GiBRTX 4000 Ada Generation 20 GiB20.0 GiB
219.46 GiBNVIDIA RTX 3080 (10 GiB)10.0 GiB
420.14 GiBGTX 1060 6 GiB6.0 GiB
821.48 GiBGTX 1060 3 GiB3.0 GiB

Quantization comparison

Batch 1 · FP16 KV cache · 1 GPU · totals include safety margin. Contexts beyond the model maximum are hypothetical and marked *.

PrecisionWeightsTotal @ 4kTotal @ 8kTotal @ 32k
FP16 / BF1616.0005 bpw · FP16 GGUF reference51.75 GiB59.08 GiB59.36 GiB61.04 GiB
FP88 bpw · Nominal bit width25.87 GiB30.10 GiB30.38 GiB32.06 GiB
INT88 bpw · Nominal bit width25.87 GiB30.10 GiB30.38 GiB32.06 GiB
NVFP66 bpw · Nominal bit width19.41 GiB22.85 GiB23.13 GiB24.81 GiB

Run with Ollama or llama.cpp

Q4_K_M GGUF commands. Install a compatible backend first; access conditions may apply.

ollama run hf.co/unsloth/Qwen3.5-27B-GGUF:Q4_K_M
llama-server -hf unsloth/Qwen3.5-27B-GGUF:Q4_K_M -c 8192 -ngl 99

GPU architecture and runtime requirements

NVIDIA architectureAttention backendPublished environment or release
Ampere / AdaFlashAttention-2CUDA ≥ 12.0 · PyTorch ≥ 2.2 · FP16 / BF16
Hopper (H100 / H800)FlashAttention-3CUDA ≥ 12.3
Blackwell (B200)FlashAttention-4 · CuTeDSLPyTorch 2.7 introduced Blackwell support in its CUDA 12.8 builds

Transformers version is checkpoint-specific. Model card · Kernel compatibility · PyTorch build notes

Recommended GPUs by memory fit

Q4_K_M @ 8k context, batch 1, FP16 KV cache. Smallest nominal memory first; green results only. Check backend and precision support separately.

Sources & verification

Try your configuration

Your workload

Workload

Precision of the weights held in memory.

Longer contexts make cache precision more important.

Runtime settingsRoofline estimate

Your hardware

Input settings

Sequences per batch. Serving concurrency and throughput require separate measurements.

Advanced runtime settings
Advanced · Model architecture
Layers
64
Attention heads
24
KV heads
4
Head dimension
256
Hidden size
5,120
Maximum context (tokens)
262,144
View model configuration

Performance and memory results

79.7%Memory use

Memory estimate

19.1 GiB needed

Fits4.9 GiB to spare24.0 GiB available

This estimate counts KV for the 16 full-attention layers. It omits the recurrent state of the other 48 layers. Actual memory use can exceed the estimate.

The estimated workload fits with the safety margin included.

Estimated GPU rental · $0.44/h · Per-GPU quote × devices

Generation speed
~38 token/s
Time per token
~26 ms
Time to first token
~0.499 s
Qwen3.5 27B

RTX 4090 24 GiB

Q4_K_M · FP16 KV

Model specifications
Memory breakdownGiB

Vertical line = usable hardware memory

  • Weights (Q4_K_M)15.83 GiB
  • KV cache (FP16)0.50 GiB
  • Runtime overhead0.75 GiB
  • CUDA Graph capture0.00 GiB · 0.0%
  • Safety margin2.05 GiB
Framework overhead breakdown

Planning allocations. Actual driver, library and allocator use varies by runtime.

Device runtime and driver
0.50 GiB
Operator workspaces
0.20 GiB
CUDA Graph
0.00 GiB
Allocator base
0.05 GiB
Other runtime buffers and communication
0.00 GiB
Compare context lengths
Compare GPU capacity

Inference simulation

Qwen3.5 27B · Q4_K_M · RTX 4090 24 GiB

Prompt processing
0.5 s
Token generation
3.3 s
Estimated duration
3.8 s
Ready
Timing playback · sample text0 / 128 · 0 s

Press play to preview how the sample streams.

Measured latency percentiles and request throughput

Frequently asked questions

How much VRAM does Qwen3.5 27B need?

Our default Q4_K_M estimate at 8,192 context tokens, batch 1 and FP16 KV cache is 19.1 GiB including the safety margin. These are estimates, not measured guarantees.

Why does context length change memory use?

KV cache grows linearly with context length and batch size. We use KV heads for grouped-query attention and the recorded KV-cache layer count.

Does a green result guarantee compatibility?

No. Green means the estimated memory including safety headroom fits. Backend support, precision support, kernels and runtime allocation can differ. Confirm compatibility and measure your actual workflow.

Where do these estimates come from?

Architecture and file sizes are traced to model sources. The precision table identifies measured GGUF rates, block storage and nominal bit widths. The source data for Qwen3.5 27B was last verified on 2026-11-18. Memory estimates use GiB.