In this project, we explore how we can entropy and information in language models and how we can optimize it for generative tasks.

Novel technique to fit a target distribution with a class of distributions using SVI (via NumPyro). Unlike standard SVI, our "data" is a distribution rather than a finite collection of samples.

Using Monte-Carlo simulated datasets, a completely transparent Boltzmann Machine trained on 1-D Ising chain data is implemented to predict model couplers in the absence of past coupler values. Methods from machine learning applied to theoretical physics are on display in this work.

A PyTorch Implementation of Generating Sentences from a Continuous Space by Bowman et al. 2015.

Implementation of the Non-negative Multiple Matrix Factorization (NMMF) algorithm proposed in Takeuchi et al, 2013 with some modifications. There is a python native version NMMFlexPy and a R wrapper NMMFlexR

Kullback-Leibler divergence in Python

This repository contains the lab work for Coursera course on "Generative AI with Large Language Models".

My MSc project on applying, tuning and modifying the PPO and A2C algorithms to Pettingzoo MARL library two player poker game

Change point detection using KL divergence

The Unstable Population Indicator

A pytorch package for non-negative matrix factorization.

Implementation of a Denoising Diffusion Probabilistic Model with some mathematical background.

Some code to Get the Optimal relative Transport started. This will be slowly updated if needed.

average-KL-divergence-calculator.py is a Python script that calculates the average KL divergence for each FASTA file in a directory and produces separate output files and a combined output file with the results.

The Dirichlet Mechanism for Differentially Private KL Divergence Minimization

Forward Sampling-Conversion of BN

IJCAI 2021, "Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation"

Scheduling TRPO's KL Divergence Constraint

PyTorch implementations of the beta divergence loss.

This repository includes some detailed proofs of "Bias Variance Decomposition for KL Divergence".