Multi-Head Attention — Formally Explained and Defined | by Jean Meunier-Pion | Jun, 2024

A comprehensive and detailed formalisation of multi-head attention

Multi-head attention plays a crucial role in transformers, which have revolutionized Natural Language Processing (NLP). Understanding this mechanism is a necessary step to getting a clearer picture of current state-of-the-art language models.