"surprising predictions" as a criterion for model selection

[gusl]

For about a year now, I've been meaning to write Henkjan Honing to argue against the position he expresses on Computational modeling of music cognition: a case study on model selection.

I started reading the paper from the beginning, and saw that the source of this view is Roberts & Pashler - How Persuasive Is a Good Fit? A Comment on Theory Testing. This is Seth Roberts, the Berkeley psychologist who has done a lot of (and written about) self-experimentation. Here is his blog. Here is the LJ syndication I created.

Comments

[kvschwartz.livejournal.com]

"Surprising predictions as a criterion of model section" ... is that in any way related to Popper's claim that the "risky theory" is "more scientific" than the "unrisky theory"?

[gustavolacerda.livejournal.com]

I'm not familiar with Popper's claim, but from glancing at Roberts's paper, I understand that this criterion is a way of countering conservative biases... so this doesn't apply if there is no such bias to begin with, e.g. if the candidate models are picked randomly, or if every possible model is tested exhaustively.

[kvschwartz.livejournal.com]

Popper's claim was a way of countering conservative biases, not only in TESTING, but, I think, in FORMULATING -- i.e., desires, both conscious and unconscious, not to have one's theory proven wrong.

If all candidate models are easily tested, then Popper would probably label them ALL "risky." His chief issue was "falsifiability," and so long as a theory (or model) could easily be tested (i.e., shown to be false), he would consider it "scientific").

Example of a highly scientific theory, in the Popperian sense: general relativity.

Example of a highly UNscienfific theory, in the Popperian sense: Freudianism.

Popper

[(Anonymous)]

The reference was to Popper (1963): "Confirmations [of a theory] should count only if they are the result of risky predictions; that is to say, if, unenlightened by the theory in question, we should have expected an event which was incompatible with the theory — an event which would have refuted the theory."

See http://www.musiccognition.nl/blog/2007/07/what-makes-theory-compelling_23.html

hh