Sequence Modeling with RNNs and Attention

Implement a basic Recurrent Neural Network (RNN) cell and apply backpropagation through time (BPTT)

Implement Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) networks to overcome vanishing gradients

Build bidirectional RNNs to process sequence information from both directions

Implement a sequence-to-sequence (seq2seq) model for tasks like machine translation

Implement Bahdanau and Luong attention mechanisms to enhance seq2seq models

Derive and implement scaled dot-product self-attention

Build a multi-head attention layer, the core component of the Transformer