Self-Attention

Self-attention lets a model weigh the importance of different parts of the input sequence when processing it. Think of it like reading a sentence and focusing more on the words that are most relevant to understanding the whole meaning. It helps the model understand context and relationships within the data.