PhD-application-related reflection

This is from an email conversation with a professor (who I will leave anonymous for now, by default):

<< I like the idea of making Machine Learning a tool usable by "the scientists", by doing model search with a prior that encodes the scientist's domain knowledge. This isn't always very straightforward, but I'd like to work on making it so... I gravitate around knowledge engineering when confronting this kind of question.

While I can get very excited by the idea of "automated science", I am in practice more fond of the engineering ideology. I'm really only satisfied when I have a program that works. I suppose at one level, it's as if I were a stupid person who is skeptical of abstract arguments: even as a math major, I leaned very experimental. (I think this is the same skepticism that led me to my interest in mathematics formalization systems, which turned out to be completely impractical, and very painful and unrewarding to work with. That's a different long-term dream.) >>
-Gustavo Lacerda

CheatSearch

Last month, I defined an interface called SemLearningMethod, which outputs given data. PcSearch, being a way to learn SEMs, implements SemLearningMethod. Likewise for Shimizu2006Search.

I needed a golden standard to compare these methods to. I made one and named it CheatSearch... it's a wonderful algorithm that performs as well as is theoretically possible.

I keep having to treat CheatSearch as an exception. In my latest experiments, I'm trying to tune parameters so as to minimize the mean loss. Guess who is the all-around champion? CheatSearch!