Skip to content
AI360Xpert
Paper Breakdowns
Paper breakdown

DeepSeek-V2

The 2024 paper that disrupted the AI pricing landscape by introducing Multi-Head Latent Attention (MLA), drastically reducing the memory required for the KV Cache.

Paper: DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

Authors: DeepSeek-AI · 2024

Read the paper
DeepSeek-V2 compresses the massive Key-Value (KV) cache into a tiny latent vector using Multi-Head Latent Attention (MLA), saving massive amounts of VRAM.
DeepSeek-V2 compresses the massive Key-Value (KV) cache into a tiny latent vector using Multi-Head Latent Attention (MLA), saving massive amounts of VRAM.

The Problem

As LLMs became standard in production, companies realized the main bottleneck to serving them wasn't compute (FLOPs), but memory (VRAM). Specifically, to generate text quickly, models store the representations of all previous tokens in a "KV Cache" (Key-Value Cache). For long context windows and thousands of concurrent users, the KV cache grows to hundreds of gigabytes, severely limiting how many users a single GPU can handle. Techniques like Grouped-Query Attention (GQA) helped, but didn't solve the core memory bottleneck.

The Idea

DeepSeek-V2 introduced Multi-Head Latent Attention (MLA). Instead of storing the full Key and Value vectors for every token in every attention head, MLA compresses them into a single, low-dimensional "latent" vector. When the model needs to calculate attention, it uncompresses (projects) this latent vector back into Keys and Values on the fly. This trades a tiny amount of extra compute for a massive reduction in VRAM usage.

How It Works

In standard Multi-Head Attention (MHA) or GQA, the KK and VV matrices are stored in memory for the duration of the generation.

In MLA:

  1. The model projects the hidden state of a token into a small latent vector ctc_t.
  2. Only ctc_t is stored in the KV Cache. Because ctc_t is very small, the memory footprint drops by 90%+.
  3. During generation, ctc_t is multiplied by "up-projection" matrices to reconstruct the KK and VV vectors needed for the attention calculation.
  4. To handle positional information (RoPE), which doesn't compress well, a separate, small RoPE token is kept alongside the latent vector.

Combined with a highly optimized Sparse Mixture-of-Experts (MoE) architecture, DeepSeek-V2 achieved incredible efficiency.

Why It Mattered

DeepSeek-V2 matched the performance of top-tier models (like Llama 3 70B) but cost significantly less to serve. DeepSeek passed these savings on to developers, launching an API that was nearly 10x to 100x cheaper than OpenAI's GPT-4, triggering an industry-wide price war for API access.

What Came After

The success of DeepSeek-V2 and its successor V3 proved that extreme architectural efficiency (MLA + MoE) could allow smaller, well-funded labs to compete directly with trillion-dollar tech giants. It also established Chinese AI labs as absolute frontier players in open-weights model development.

What to Read Next