Great-Deep-Learning-Tutorials

A Great Collection of Deep Learning Tutorials and Repositories

General Deep Learning Tutorials:

• Browse state-of-the-art Deep Learning based Papers with their associated codes [Extremely Fantastic]
• Deep-Learning-Roadmap
• DeepLizard [Good Tutorials for Deep Learning]
• Sebastian Ruder - Blog [Great NLP & Deep Learning Posts]
• Jeremy Jordan - Blog
• Excellent Blog
• Torchvision Release Notes [Important]
• The 6 most useful Machine Learning projects of the past year (2018)
• ResNet Review
• Receptive Field Estimation [Great]
• An overview of gradient descent optimization algorithms [Useful]
• How to decide on learning rate
• Overview of State-of-the-art Machine Learning Algorithms per Discipline per Task
• Practical Machine Learning
• Awesome Machine Learning and AI Courses
• UVA Deep Learning II Course
• PyTorch Book
• Fast.ai Course: Practical Deep Learning for Coders [Great]
• Neuromatch Deep Learning Course [Great]
• labmlai: 59 Implementations/tutorials of deep learning papers with side-by-side notes [Great]
• labml.ai
• FightingCV-Paper-Reading: understand the most advanced research work in an easier way
• Learn PyTorch for Deep Learning: Zero to Mastery Course [Excellent]
• ML Papers Explained [Excellent]
• Alpha Signal: Latest Research in Machine Learning
• Harvard CS197: AI Research Experiences - The Course Book [Excellent]
• Deep learning jupyter notebook book
• A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT
• interconnects.ai: Great AI Blog Posts & Podcasts

Deep Learning Useful Resources for Computer Vision:

• Great Deep Learning Resources for Computer Vision Tasks [Excellent]

Deep Learning Useful Resources for Natural Language Processing (NLP):

• Great Deep Learning Resources for NLP Tasks [Excellent]

Deep Learning Useful Resources for Spoken Language Processing (Speech Processing):

• Great Deep Learning Resources for Speech Processing Tasks [Excellent]

Deep Learning & Machine Learning Useful Resources for General Data Science Tasks:

• Great Deep Learning Resources for Data Science Tasks [Excellent]

General Notes about Generative AI:

• Generative AI in action: real-world applications and examples

Quantization & Distillation of Deep Learning Models:

• Quantization
• Neural Network Distiller
• Introduction to Quantization on PyTorch [Excellent]
• Dynamic Quantization in PyTorch
• Static Quantization in PyTorch
• Intel(R) Math Kernel Library - Intel MKL-DNN
• Intel MKL-Dnn
• ONNX Float32 to Float16
• Neural Network Quantization Introduction [Tutorial]
• Quantization in Deep Learning [Tutorial]
• Speeding up Deep Learning with Quantization [Tutorial]
• Knowledge Distillation in Deep Learning
• Model Distillation Techniques for Deep Learning
• MMRazor: model compression toolkit [Great]
• FP8 Quantization: The Power of the Exponent
• Quanto: a pytorch quantization toolkit [Great]

AutoML:

• Auto Gluon AI
• AWS Auto Gluon

Diffusion Models:

• Diffusion Models via lilianweng
• Diffusion Models Papers Survey Taxonomy
• Phenaki: a text-to-video model

Multimodal Deep Learning:

• Multimodal Deep Learning Book

Deep Reasoning:

• What’s Next For AI? Enter: Deep Reasoning
• Deep Learning approaches to understand Human Reasoning

Deep Reinforcement Learning (Great Courses & Tutorials):

• A Free course in Deep Reinforcement Learning from beginner to expert [Great]
• Deep Reinforcement Learning Algorithms with PyTorch
• Deep Reinforcement Learning - CS 285 Berkeley Course
• solutions to UC Berkeley CS 285
• Reinforcement Learning: An Introduction - main book in this field
• CS234: Reinforcement Learning Course
• Introduction to Reinforcement Learning Course - by DeepMind

Graph Neural Networks:

• An Introduction to Graph Neural Networks
• How to Train Graph Convolutional Network Models in a Graph Database
• A comprehensive survey on graph neural networks
• Graph Neural Networks: A Review of Methods and Applications

Graph Neural Networks Frameworks:

• Spektral
• Deep Graph Library - DGL
• PyTorch Geometric - PyG
• ptgnn: A PyTorch GNN Library
• Graph Data Augmentation Papers
• Neo4j: Graph Data Platform

Best Practices for Training Deep Models:

General Notes for Training Deep Models:

• Deep Learning Tuning Playbook

PyTorch Lightening Notes & Accumulate Gradients:

• PyTorch Lightening: Effective Training Techniques
• Gradient Accumulation in PyTorch

Loss Functions:

• Loss Functions Explained

Imbalanced Dataset Handling:

• deal with an imbalanced dataset using weightedrandomsampler
• imbalanced-dataset-sampler [Great]
• demystifying pytorchs weightedrandomsampler
• weighted random sampler oversample or undersample

Weight Initialization:

• Deep Learning Best Practices (1) - Weight Initialization

Batch Normalization:

• Batch Normalization in Neural Networks
• Batch Normalization and Dropout in Neural Networks
• Difference between Local Response Normalization and Batch Normalization

Learning Rate Scheduling & Initialization:

• Automated Learning Rate Suggester
• Learning Rate Finder - fastai
• Cyclical Learning Rates for Training Neural Networks
• ignite - Example of FastaiLRFinder
• Find Learning Rate - a gist code
• Learning rate finder - PyTorch Lightning
• RAdam - On the Variance of the Adaptive Learning Rate and Beyond

Early Stopping:

• Early Stopping in PyTorch - Bjarten
• Catalyst - Early Stopping
• ignite - Early Stopping
• PyTorch High-Level Training Sample
• PyTorch Discussion about Early Stopping

Tuning Guide Recipes:

• PyTorch Tuning Guide Tutorial
• PyTorch memory leak with dynamic size tensor input
• Karpathy: A Recipe for Training Neural Networks

Training Optimizer:

• What is gradient accumulation in deep learning

PyTorch running & training on TPU (colab):

• PyTorch XLA
• PyTorch XLA Colab

Evaluation Metrics:

• Performance Metrics for Classification Problems in ML

Validating ML Models:

• Deepchecks: Validating ML Models & Data

Optimizing models when run on GPU:

• Tips for reducing vram of gpu memories

Conferences News:

• Latest Computer Vision Trends from CVPR 2019
• Interesting 2019 CVPR papers
• Summaries of CVPR papers on ShortScience.org
• Summaries of ICCV papers on ShortScience.org
• Summaries of ECCV papers on ShortScience.org

Deep Learning Frameworks and Infrustructures:

• set-up a Paperspace GPU Server
• Distributed ML with OpenMPI
• Tensorflow 2.0 vs Mxnet
• TensorFlow is dead, long live TensorFlow!

Great Libraries:

• The Unified Machine Learning Framework
• Skorch - A scikit-learn compatible neural network library that wraps PyTorch
• Hummingbird - traditional ML models into tensor computations via PyTorch
• BoTorch - Bayesian Optimization in PyTorch
• torchvision 0.3: segmentation, detection models, new datasets and more
• TorchAudio: an audio library for PyTorch
• AudTorch
• TorchAudio-Contrib
• fastText - Facebook AI Research (FAIR)
• Fairseq - Facebook AI Research (FAIR)
• ParlAI - dialogue models - Facebook AI Research (FAIR)
• DALI - highly optimized engine for data pre-processing
• Netron - GitHub [Visualizer for deep learning Models (Excellent)]
• Netron - Web Site
• JupyterLab GPU Dashboards [Good]
• PyTorch Hub
• Neural Structured Learning (NSL) in TensorFlow
• Pywick - High-Level Training framework for Pytorch
• torchbearer: A model fitting library for PyTorch
• torchlayers - Shape inference for PyTorch (like in Keras)
• torchtext - GitHub
• torchtext - Doc
• Optuna - hyperparameter optimization framework
• PyTorchLightning
• Nvidia - runx - An experiment management tool
• MLogger: a Machine Learning logger
• ClearML - ML/DL development and production suite
• Lime: Explaining the predictions of any ML classifier
• Microsoft UniLM AI [Great]
• mlnotify: No need to keep checking your training
• NVIDIA NeMo - toolkit for creating Conversational AI (ASR, TTS, and NLP)
• Microsoft DeepSpeed
• Mojo: a new programming language for AI developers
• MLX: An array framework for Apple silicon

Great Models:

• ResNext WSL [Great Pretrained Model]
• Semi-Weakly Supervised (SWSL) ImageNet Models [Great Pretrained Model]
• Deep High-Resolution Representation Learning (HRNet)

Deep Model Conversion:

• Convert Full ImageNet Pre-trained Model from MXNet to PyTorch [Great]
• ONNX Runtime

Great Deep Learning Repositories (for learning DL-based programming):

• deeplearning-models - PyTorch & TensorFlow Learning [Very Excellent Repository]
• PyTorch Image Models [Great]
• 5 Advanced PyTorch Tools to Level up Your Workflow [Interesting]

PyTorch High-Level Libraries:

• Catalyst - PyTorch framework for Deep Learning research and development [Great]
• PyTorch Lightning - GitHub [Great]
• PyTorch Lightning - Web Page
• Ignite - GitHub [Great]
• Ignite - Web Page
• TorchMetrics
• Ludwig AI: Data-centric declarative deep learning framework [Great]
• PyTorch Kineto: CPU+GPU Profiling library
• PyTorch Profiler
• PyTorch Benchmarks

Annotation Tools:

• label-studio
• label-studio with RTL Support (for Persian)

Other:

• Clova AI Research - NAVER & LINE
• Exploring Weight Agnostic Neural Networks
• Weight Agnostic Neural Networks
• Weight Agnostic Neural Networks - GitHub
• SAM: Sharpness-Aware Minimization for Efficiently Improving Generalization
• Qualcomm Discusses Secret Dataset Generation Data
• State of AI Report 2021
• Project Blink: AI-powered video editing on the web
• PyTorch Incremental Learning
• Google Research, 2022 & Beyond: Language, Vision and Generative Models
• Elicit: Ask a research question [Interesting]
• Google People + AI Research (PAIR) [Interesting business based AI topics]