Attention Mechanism Demo

Phase 2

Visualize how LLMs focus on different parts of text to understand context

The

cat

sat

the

mat

because

was

comfortable.

Animation Speed1x

Attention is a key mechanism in transformer models that allows them to focus on different parts of the input when generating each word of output.

For each word, the model calculates attention scores for all other words in the context.
Higher scores mean the model pays more attention to that word when processing the current word.
This helps resolve references (like pronouns) and understand relationships.

Self‑attention lets the model…

Multi‑head attention helps by…