Bayesian ICA?
Jul. 21st, 2007 02:38 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
guslI just had some ideas about making the ICA method for learning linear causal DAGs more Bayesian:
* add prior probabilities about edges' existence: this might be straightforward: can we somehow adjust the alpha level when pruning? (I should look up how to make quantile tests Bayesian... and read up on the Wald statistic)
* add prior distributions on the coefficients that exist: this should be straightforward: just make the current test of the mean bootstrap coefficient Bayesian (but this may need to deal with an existing problem: is the bootstrap coefficient mean an unbiased estimator of the coefficient given by ICA on the whole sample? Certainly, with bootstrap samples, we will be less confident about independences than with the whole sample, because they are less diverse.)
* add prior probabilities about conditional independence relations: this is more complicated, as there should be a notion of coherence (in the Dutch book sense). Is it possible (and easy) to get an incoherent prior?
In other words, is there always a prior distribution over the set of graphs that agrees with the priors put on graph properties?
It's easy to settle this question in the negative if we choose priors that give 0 probability to some values, and construct a logical contradiction this way. But if none of my priors rule out any possibilities, can my set of priors still be incoherent?
Answer: yes! Imagine I have the following priors (first one about conditional independence, the second about the existence of an edge):
P (A _||_ B) = 0.9
P (A --> B) = 0.5
The second statement implies the negation of the former. Detecting incoherence sounds like an NP-complete problem. Do we care about being incoherent afterall?
* Finally, I should study GES search, a Bayesian alternative to PC search.
* add prior probabilities about edges' existence: this might be straightforward: can we somehow adjust the alpha level when pruning? (I should look up how to make quantile tests Bayesian... and read up on the Wald statistic)
* add prior distributions on the coefficients that exist: this should be straightforward: just make the current test of the mean bootstrap coefficient Bayesian (but this may need to deal with an existing problem: is the bootstrap coefficient mean an unbiased estimator of the coefficient given by ICA on the whole sample? Certainly, with bootstrap samples, we will be less confident about independences than with the whole sample, because they are less diverse.)
* add prior probabilities about conditional independence relations: this is more complicated, as there should be a notion of coherence (in the Dutch book sense). Is it possible (and easy) to get an incoherent prior?
In other words, is there always a prior distribution over the set of graphs that agrees with the priors put on graph properties?
It's easy to settle this question in the negative if we choose priors that give 0 probability to some values, and construct a logical contradiction this way. But if none of my priors rule out any possibilities, can my set of priors still be incoherent?
Answer: yes! Imagine I have the following priors (first one about conditional independence, the second about the existence of an edge):
P (A _||_ B) = 0.9
P (A --> B) = 0.5
The second statement implies the negation of the former. Detecting incoherence sounds like an NP-complete problem. Do we care about being incoherent afterall?
* Finally, I should study GES search, a Bayesian alternative to PC search.