AI Notes

Everything I know about Artificial Intelligence/Data Science/Machine Learning/Statistical Modeling/Pattern Recognition/whatever you wanna call the contents of this note. The line between all of these is pretty blurry, but they all try to answer the same question: "How can we learn from data?"

Machine Learning is basically just fancy function approximation.

1. Required Background
   1. Statistics
   2. Probability
   3. Linear Algebra
   4. Calculus
   5. Computer Science
   6. Programming
      • Python (preferably) or Matlab
   7. Linear Systems and Signals
      • Convolution
2. History (or how we got to now)
   1. Rosenblatt's Perceptrons
3. Pre-processing
   1. PCA
   2. t-SNE
   3. Dimensionality reduction
      • Manifold Learning
   4. Cross Validation
      • K-fold CV
      • Leave-One-Out CV
   5. Fisher Vectors
   6. Data Encoding
   7. Data Compression
   8. Linear Discriminant Analysis (LDA)
   9. Singular Value Decomposition (SVD)
   10. One-Hot Encoding
4. Feature Analysis
   1. Feature Extraction
   2. VLAD
   3. Bag of Words
   4. tf-idf
5. Distance Measures
   1. Euclidean Measures
   2. Nearest Neighbour
   3. Fisher Condition
6. Information Theory & Topology
   1. KL Divergence
   2. Entropy
   3. Tangent: Huffman Coding
   4. Manifold Hypothesis
7. Regression
   1. Least Squares Regression
   2. Polynomial Regression
   3. Cubic Splines
   4. Logistic Regression
8. Classification
   1. MICD Classifier
   2. Support Vector Machines (SVMs)
   3. Kernels
9. Clustering
   1. K-Means
   2. Self-Organizing Maps (SOMs)
   3. Hierarchical Clustering
   4. Density-Based Clustering
10. Model Estimation
    1. Parametric Estimation
    2. Non-Parametric Estimation
       • Parzen Window
    3. K-NN Estimation
11. Deep Learning & Neural Nets
    1. Motivation
    2. Linear Discriminants
    3. Model of a Neuron
    4. MLPs/ANNs
    5. Gradient Descent and Backpropogation
    6. Activation Functions
    7. CNNs
    8. RNNs
       • LSTM
    9. Generative Adversarial Networks (GANs)
    10. Restricted Boltzmann Machines (RBMs)
    11. Auto-encoders
12. Reinforcement Learning
    1. Motivation
    2. Temporal Differencing
    3. Q-Learning
    4. Deep Q-Learning
13. Decision Trees and Random Forests
14. Fuzzy Systems and Fuzzy Logic
    1. Set Theory
    2. Fuzzy C-Means
    3. Fuzzy Inference Systems
15. Bio-Inspired Algorithms
    1. Evolutionary Algorithms
    2. Genetic Algorithms
    3. Differential Evolution
    4. Ant Colonies
    5. Particle Swarm
16. Probabilistic Methods
    1. Markov Chains and Models
    2. Maximum Likelihood and Maximum A Priori
    3. Bayes and Naive Bayes Classifier
    4. Bayesian Belief Networks
    5. Hidden Markov Models
17. Optimization
    1. Constrained Optimization
    2. Unconstrained Optimization
    3. Linear and Quadratic Programming
    4. Convex Optimization
    5. Gradient Descent
    6. Stochastic Gradient Descent
    7. Adam
18. Natural Language Processing
19. Ensemble Methods
    1. Bagging
    2. Boosting
       • Adaboost
    3. Stacking
20. Transfer Learning
21. Overfitting
    1. Bias-Variance Tradeoff
    2. Occam's Razor
    3. Regularization
22. Practical Machine Learning
    1. Keras
    2. TensorFlow
    3. Scikit-Learn
23. Philosophy
    1. Turing Test
    2. Chinese Room
    3. Mary's Room
    4. Consciousness
24. Ethics
    1. Utilitarianism
       • Harm Principle
25. Neuroscience
    • How the brain works:
      • https://www.khanacademy.org/science/biology/human-biology#neuron-nervous-system
      • http://waitbutwhy.com/2017/04/neuralink.html#part2

Directly related courses I've taken:

• SYDE 522: Machine Intelligence
• SYDE 372: Pattern Recognition
• Stanford Machine Learning: Andrew Ng

Background Courses:

• Calculus 1,2,3
• Probability & Statistics
• Linear Algebra 1 & 2
• Linear Systems and Signals
• Data Structures and Algorithms
• Numerical Methods
• Optimization

If you wanna chit-chat about any of this stuff, holla at me @ m.me/krishnr.