Computer Architecture
- DRAM Modeling
- Understanding and Implementing DRAM Timings
- Matrix Multiplication: Inner Product, Outer Product & Systolic Array
Compression
Deep Learning
- What is Attention, Self Attention, Multi-Head Attention?
- Back Propagation in Batch Normalization Layer
- Back Propagation in Convolution Layer
- Back Propagation in Dilated Convolution Layer
- Back Propagation in Fully Connected Layer
- Back Propagation in Recurrent Neural Networks
- Back Propagation in Softmax Layer
- Back Propagation in Strided Convolution Layer
- Loss Functions: Cross Entropy, Log Likelihood and Mean Squared
- Exploding and Vanishing Gradients in Recurrent Neural Networks
- How does PyTorch (row major) interact with cuBLAS (column major)?
Deep Neural Networks
- Bidirectional Encoder Representations from Transformers (BERT)
- Deep Convolutional Generative Adversarial Network (DCGAN)
- Deep Learning Recommendation Model (DLRM)
- Neural Collaborative Filtering (NCF)
- ResNet50
- Tacotron 2