Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Post Submission tasks #197

Open
1 of 9 tasks
sampan501 opened this issue Jan 7, 2024 · 2 comments
Open
1 of 9 tasks

Post Submission tasks #197

sampan501 opened this issue Jan 7, 2024 · 2 comments

Comments

@sampan501
Copy link
Member

sampan501 commented Jan 7, 2024

@adam2392
Copy link
Collaborator

adam2392 commented Jan 8, 2024

The general MVN approach maybe can be done as Jovo suggested (w/ some open questions):

X_i | Y ~ MVN, where for CoMIGHT, we generate two such instances that are either directly dependent or not.

Y = mixture of MVN Gaussians, so the MI terms is then: $I(X1, X2; Y) = H(X1, X2) - H(X1, X2 | Y) = H(X1 | X2) + H(X2) - H(X1 | X2, Y) + H(X2 | Y)$

where the non-trivial parts to currently compute are:

  • H(X1 | X2) is unsure how to compute analytically, unless we numerically integrate?...
  • H(X1 | X2, Y) is the same

Maybe we generate a huge MVN first where we know the $\Sigma_{X1, X2}$ for the subset of variables we denote X1, X2, which is still MVN, and therefore we know H(X1, X2). Then, we use Y as the mixture of Gaussians w/ varying mixture probability?

@adam2392
Copy link
Collaborator

adam2392 commented Jan 9, 2024

Structuring the covariance in blocks as such and then using $Y \in [1, 2]$ to select the corresponding multivariate normal should allow us to:

  1. arbitrarily apply feature-wise transformations for a specific class -> then there is a functional relationship between $X$ and $Y$.
  2. compute analytical CMI and MI cuz we would have analytical solution for $H(X)$ and $H(X | Y) = H(X | Y=1) + H(X | Y=2) = H(X^{(1)}) + H(X^{(2)})$

Now I'm not 100% sure how to fit this in w/ the Marron/Wald

IMG_2247

and

IMG_2248

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants