Skip to main content
Back to course
Log in
Get started
Someone wanted to learn this too, so Grasp built them a personal learning path.
Create your own
Audio AI Researcher & Developer
ยท
Module 4
Sequence Modeling with Transformers
1
1D CNNs for Audio Feature Extraction
Explain how 1D Convolutional Neural Networks (CNNs) can act as feature extractors for raw audio waveforms.
2
RNN and LSTM Architectures for Temporal Sequences
Describe the architecture of Recurrent Neural Networks (RNNs) and LSTMs for modeling temporal sequences.
3
Self-Attention: Beyond Recurrence for Long-Range Dependencies
Explain the self-attention mechanism and its advantages over recurrent models for capturing long-range dependencies.
4
Deconstructing the Transformer Architecture
Describe the complete Transformer architecture, including positional encoding, multi-head attention, and feed-forward layers.
5
Building Causal Multi-Head Attention in PyTorch
Implement a multi-head self-attention layer in PyTorch, including support for causal masking.
6
Building a Transformer Encoder Block
Assemble a Transformer encoder block combining multi-head attention and a feed-forward network with residual connections.
7
Building a Transformer Decoder Block
Assemble a Transformer decoder block, including masked self-attention, cross-attention, and a feed-forward network.
8
Building a Transformer from Scratch in PyTorch
Construct a full encoder-decoder Transformer model in PyTorch for sequence-to-sequence tasks.
Previous module
Audio Data Augmentation and Pipelines
Next module
Foundations of Generative Modeling