(no subject)

[gusl]

Good stuff from Cosma Shalizi. I want to be like him: I want to work on philosophical questions (or simply highly conceptual questions) on a technical/mathematical level.

Machine Learning, Statistical Inference and Induction

Complexity, Entropy and the Physics of gzip, where he criticizes physicists using this approach. What I don't understand is when physicists would want to calculate the entropy of a source.

and his review of "Jorma Rissanen - Stochastic Complexity in Statistical Inquiry"

The Backwards Arrow of Time of the Coherently Bayesian Statistical Mechanic

---

As I recently found out, Herbert Simon is also someone I really look up to. Besides pioneering cognitive science, which aims for a coherent computational theory of cognition, and in particular high-level "chunking" and "rules of the mind", he studied human decision-making and wrote a book called "Administrative Behavior", which examines rationality and decision-making... the sort of question I want to give consult about.

Comments

[mathemajician.livejournal.com]

I really don't understand his criticism of the gzip approach to estimating the Kolmogorov complexity of a string. It makes no sense to me, but perhaps I'm biased as this might be a PhD topic for me.

He seems to be saying that the gzip approach is no good because he can make a simple case where it fails badly. Well that's no big deal, in fact you can prove that all estimates of the Kolmogorov complexity will fail horribly in some cases because Kolmogorov complexity is not just incomputable, it's really badly incomputable.

The point is that gzip is fast and easy and it often produces useful results. If you want better estimates use a better compressor like bzip2 or even better a PPM compressor. But what ever you do there will always be simple cases where you estimate is screwed and there's nothing you can do about it.

[gustavolacerda.livejournal.com]

Actually, after browsing it more carefully, it makes no sense to me either. I thought he would be talking about the use of KC in physics.

Have the people in your institute made a compressor out of a universal learning machine?

[mathemajician.livejournal.com]

No, our universal machines aren't really buildable. Though I think Jan might be trying to build something based on these ideas for some special case.

At the moment I've just tried to use compression for a few basic problems like text categorization. I normally use bzip2 as it's better than gzip, though I have tried PPM compressors as well. My results so far seem to be that the approach works reasonably well, but perhaps not well enough to replace existing methods.