![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
guslOn Sunday we had our yearly student conference. Almost everybody spoke about their research. Vince, however, gave a talk about teaching, and I am very glad that he did. His focus was on our introductory courses, which he renamed as follows:
1211: "Statistics with calculus"
1111: "Statistics with algebra"
1001: "Statistics with words"
His main point was that most students in the latter two courses don't think the way that we do, and that constructivism was a better teaching philosophy than the mere "information transfer" approach (which is very tempting, especially given the ubiquity of the illusion of transparency, but this mode of communication is unusual in the general population and it takes time to develop). There is a bit of a false dichotomy here, since information transfer does not exist without some degree of interpreting words in terms of pre-existing concepts, but I think the general idea is clear: the more you tie it to things they understand, the better they will understand it.
The hard reality is that many students don't understand if-statements, and somehow they still hang in there. When I see students who can do MLE derivations but who don't understand what a parameter is, I am disillusioned but I'm also impressed at their level of dedication... and I am more than happy to teach them. In particular, I like to illustrate the concept of a parametric family with the metaphor of an oscilloscope with a number of knobs: "first knob is location, second knob is scale. "Which settings best fit this data?". "The likelihood is a product of densities, and to maximize that you don't want any points to get a density close to 0."
---
Last semester I TA'd 1001, which was basically a course on critical thinking about statistics. Their first project consisted of taking a news article involving statistics and critiquing it from a methodological perspective, ultimately trying to answer "Does the study support the conclusions presented by the news article?". They had a rough outline of points to address (including: author biases, experiment vs observational study, randomization, control groups, sample representativeness / self-selection biases, causal confounding, etc).
I really enjoyed the experience of grading this project, because there is a large number of concepts that go into it, and it forced me to organize my thoughts, e.g. to explicitly see that the question "is there selection bias?" is one failure mode of another item: "is the sample representative?" which is equivalent to "do the conclusions generalize to the wider population?" (i.e. what population does the sample come from / can tell us about, without resorting to common-sense assumptions?).
Other basic but cool insights:
* proper randomization controls for unobserved (and unimagined!) confounders.
* common-sense assumptions are ubiquitous; after we hammered "correlation is not causation" into their skulls, I was amused when a student hinted that maybe such-and-such disease causes changes in DNA.
* students often said that experimental data pertaining to certain questions would be impossible to get for ethical reasons, because they only imagined intervening in one direction. But it was often the case that you could ethically intervene in the other direction, e.g. by asking people not to smoke.
* causal language can be slippery
---
In case anyone is interested, here is the grading scheme that I came up with:
The biggest downside of this scheme is that it can be gamed by taking a methodologically simple, unambitious study (not asking causal questions, not attempting to generalize further, no human subjects), but we could fix this by rewarding more ambitious projects.
1211: "Statistics with calculus"
1111: "Statistics with algebra"
1001: "Statistics with words"
His main point was that most students in the latter two courses don't think the way that we do, and that constructivism was a better teaching philosophy than the mere "information transfer" approach (which is very tempting, especially given the ubiquity of the illusion of transparency, but this mode of communication is unusual in the general population and it takes time to develop). There is a bit of a false dichotomy here, since information transfer does not exist without some degree of interpreting words in terms of pre-existing concepts, but I think the general idea is clear: the more you tie it to things they understand, the better they will understand it.
The hard reality is that many students don't understand if-statements, and somehow they still hang in there. When I see students who can do MLE derivations but who don't understand what a parameter is, I am disillusioned but I'm also impressed at their level of dedication... and I am more than happy to teach them. In particular, I like to illustrate the concept of a parametric family with the metaphor of an oscilloscope with a number of knobs: "first knob is location, second knob is scale. "Which settings best fit this data?". "The likelihood is a product of densities, and to maximize that you don't want any points to get a density close to 0."
---
Last semester I TA'd 1001, which was basically a course on critical thinking about statistics. Their first project consisted of taking a news article involving statistics and critiquing it from a methodological perspective, ultimately trying to answer "Does the study support the conclusions presented by the news article?". They had a rough outline of points to address (including: author biases, experiment vs observational study, randomization, control groups, sample representativeness / self-selection biases, causal confounding, etc).
I really enjoyed the experience of grading this project, because there is a large number of concepts that go into it, and it forced me to organize my thoughts, e.g. to explicitly see that the question "is there selection bias?" is one failure mode of another item: "is the sample representative?" which is equivalent to "do the conclusions generalize to the wider population?" (i.e. what population does the sample come from / can tell us about, without resorting to common-sense assumptions?).
Other basic but cool insights:
* proper randomization controls for unobserved (and unimagined!) confounders.
* common-sense assumptions are ubiquitous; after we hammered "correlation is not causation" into their skulls, I was amused when a student hinted that maybe such-and-such disease causes changes in DNA.
* students often said that experimental data pertaining to certain questions would be impossible to get for ethical reasons, because they only imagined intervening in one direction. But it was often the case that you could ethically intervene in the other direction, e.g. by asking people not to smoke.
* causal language can be slippery
---
In case anyone is interested, here is the grading scheme that I came up with:
<< * everyone starts with 100
* if they don't mention anything about who the authors are, the funding comes from, etc, take off 3 or 4 points. If they do, but don't show their judgement ("probably impartial", "probably biased", etc), I take off 2 points.
* if they don't make the correct judgement of whether it's a "randomized experiment" vs "observational study", take off 3 points.
* if they use the word "randomize" to mean "random sample", take off 3 points
* if they don't report "self-selected sample" or do but fail to criticize it as a bad thing, take off 2 points
* if they are overly harsh, e.g. saying that there is bias because the researchers only see reported cases of diabetes, without explaining why reporting might covary with the other variable of interest (e.g. pollution levels), maybe take off 1 point (or none)
* if they criticize the sample size as too small for no good reason, take off 1-2 points
* if they don't address issues of generalization / representativeness, take off 2-3 points
* in case of observational study, if they fail to come up with obvious potential confounders (e.g. age), take off 2 points
* in case of observational study, if they say there are no potential confounders, take off 5 points
* if they talk about "confounding" in the sense of "comparing-apples-with-oranges", I write a comment but don't penalize them
* if they misuse words like "statistically significant", I write a comment but don't penalize them
* if they present their own opinion strongly and irresponsibly (i.e. no justification), take off 5 points
* in case it's a randomized experiment, if they think that a flaw is that they didn't address potential confounders, take off 2 points
* if they criticize that the authors don't address causality when there is no causal question being asked (e.g. the research question is whether the prevalence of Celiac Disease has increased in the past 60 years), maybe take off 1 point
* if the writing is confused, seemingly contradictory, and difficult to understand, maybe take off another 3 points (most students don't lose any points here!)
Of course, there are exceptions, e.g. questionnaires don't have issues of causality.
More holistically, I also try to make a subjective assessment of how badly they need help, based on their project, and I try to make the grade encode this. So you can't compute the grade by just looking for the minus signs. >>
The biggest downside of this scheme is that it can be gamed by taking a methodologically simple, unambitious study (not asking causal questions, not attempting to generalize further, no human subjects), but we could fix this by rewarding more ambitious projects.