This repository contains an optimization generalized linear regression(GLM) problem solved from scratch using only the Numpy library. Throughout the development of this work, matrix operations with Numpy have been utilized whenever possible to ensure optimal performance. The code also includes certain classes and global functions that have been created from scratch to facilitate the implementation of the methodologies. Explanations of how these functions and classes work are explicitly mentioned and are provided when they are invoked in the code. Furthermore, additional comments on methodology and mathematical proofs are included in certain places, where they deemed necessary, to provide a deeper understanding of the solutions.
The solutions involve implementing with Numpy and analyzing the prediction bias-variance of the following methods:
- The Least Squares method
- The Ridge Regression method
- Full Bayesian Inference
- The Expectation-Maximization method
- Different noise samples
- Different polynomial model complexities
- Prior parameter knowledge
The goal of this project is to compare and contrast the different models & methods mentioned and discuss the importance of balance between bias and variance concerning model predictions.