Scientific Computing in Python / 2027 · Lecture 6 · Proposed 2027; calendar dates pending

Embeddings and scaled dot-product attention

Reading guide

  1. Embedding and projection matricesp. 1
  2. Rowwise softmaxp. 1
  3. Weighted valuesp. 2
  4. A two-key calculationp. 3
  5. Causal maskingp. 3
  6. Scaling and learned representationsp. 4

Computation and practice