Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Introduction to Large Language Models (LLMs)
- Overview of LLMs
- Definition and significance
- Applications in AI today
Transformer Architecture
- What is a transformer and how does it work?
- Main components and features
- Embedding and positional encoding
- Multi-head attention
- Feed-forward neural network
- Normalization and residual connections
Transformer Models
- Self-attention mechanism
- Encoder-decoder architecture
- Positional embeddings
- BERT (Bidirectional Encoder Representations from Transformers)
- GPT (Generative Pretrained Transformer)
Performance Optimization and Pitfalls
- Context length
- Mamba and state-space models
- Flash attention
- Sparse transformers
- Vision transformers
- Importance of quantization
Improving Transformers
- Retrieval augmented text generation
- Mixture of models
- Tree of thoughts
Fine-Tuning
- Theory of low-rank adaptation
- Fine-Tuning with QLora
Scaling Laws and Optimization in LLMs
- Importance of scaling laws for LLMs
- Data and model size scaling
- Computational scaling
- Parameter efficiency scaling
Optimization
- Relationship between model size, data size, compute budget, and inference requirements
- Optimizing performance and efficiency of LLMs
- Best practices and tools for training and fine-tuning LLMs
Training and Fine-Tuning LLMs
- Steps and challenges of training LLMs from scratch
- Data acquisition and maintenance
- Large-scale data, CPU, and memory requirements
- Optimization challenges
- Landscape of open-source LLMs
Fundamentals of Reinforcement Learning (RL)
- Introduction to Reinforcement Learning
- Learning through positive reinforcement
- Definition and core concepts
- Markov Decision Process (MDP)
- Dynamic programming
- Monte Carlo methods
- Temporal Difference Learning
Deep Reinforcement Learning
- Deep Q-Networks (DQN)
- Proximal Policy Optimization (PPO)
- Elements of Reinforcement Learning
Integration of LLMs and Reinforcement Learning
- Combining LLMs with Reinforcement Learning
- How RL is used in LLMs
- Reinforcement Learning with Human Feedback (RLHF)
- Alternatives to RLHF
Case Studies and Applications
- Real-world applications
- Success stories and challenges
Advanced Topics
- Advanced techniques
- Advanced optimization methods
- Cutting-edge research and developments
Summary and Next Steps
Requirements
- Basic understanding of Machine Learning
Audience
- Data scientists
- Software engineers
21 Hours
