Deep Neural Network Fundamentals

Build a multi-layer perceptron (MLP) with forward propagation

Derive and implement the backpropagation algorithm from scratch

Compare properties and use cases of common activation functions (Sigmoid, Tanh, ReLU, Leaky ReLU)

Implement Xavier and He weight initialization strategies to improve convergence

Diagnose and mitigate vanishing and exploding gradients using techniques like gradient clipping

Implement batch normalization and explain its effect on training stability

Apply dropout as a regularization technique to prevent overfitting

Implement early stopping and learning rate scheduling strategies