This post will provide an intuitive explanation of Query-Key (QK) and Output-Value (OV) circuits in transformer heads, breaking down these complex concepts into digestible pieces. We’ll explore how these fundamental building blocks work together to process and transform information in large language models.

Coming soon in April 2024