![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
guslPerhaps I should change the name, since I don't actually build models.
Before you leave a comment on this, I know that I need to give more background about autism and the box task.
This paper is about:
* cognitive modeling (a way of formalizing cognitive theories by implementing simulations of minds).
* cognitive theories of autism (in particular, why we believe the Executive Dysfunction theory (ED) )
* specifying possible mechanisms through which ED can account for the box task (the ultimate goal being a formalized theory of autism)
* why cognitive modeling should be the ultimate test of a theory.
My program is very ambitious, and I haven't been able to find anyone working towards the same goal.
If you don't have an LJ account, please remember to leave your name & contact info. (Unless you want to be an anonymous reviewer, that is.)
----------------------------------------------------------------------------------
Modeling Cognitive Theories of Autism
Gustavo Lacerda
1. Introduction:
Autism is a complex, pervasive abnormality of human development which manifests itself in multiple ways, including deficiencies in language, reasoning, planning and theory-of-mind, as well as super-abilities in some perceptual tasks.
There exist many theories for autism, offering different levels of explanation, for example: genetic, neurological, developmental and functional/information-processing explanations.
This paper is concerned with the latter kind: theories which explain autistic symptoms and behaviors (manifested in performance on various cognitive tasks) by postulating ways in which autists process high-level information differently from normals, and deriving the observed behaviors from these assumptions.
We intended to use a cognitive modeling system, ACT-R, in which this "derivation" occurs in the form of simulations: a cognitive model (i.e. a simulated human being) participating in a simulated experiment. We wished to see if it was possible to implement these theories into the ACT-R architecture, and if so, to implement models of them on novel experiments as a way of testing their generality and explanatory power. If such a level were achieved, it would be relatively straightforward for us to compare competing theories of autism against reality. (Psychology would be like physics). However, it seems unlikely that such a level will be reached.
In this paper, we are mostly concerned with the Executive Dysfunction Theory, as defended by Russell, versus the theory-of-mind theory.
This paper:
Ch.1 Introduction
Ch.2 introduces the logico-philosophical framework of cognitive modeling.
Ch.3 motivates, introduces and discusses competing theories of autism from a general perspective.
Ch.4 discusses possible models of the ED theory on the box task, the False Belief Task and the False Photograph Task.
2.Frameworks, Theories and Models
Frameworks
According to (3:1.2), "frameworks are composed of general claims about cognition. They are the sets of constructs that define the important aspects of cognition.", for example claims such as "there are two memory stores: a short-term store and a long-term one". Frameworks are not sufficiently specified to derive predictions. As such, they are inherently unfalsifiable.
Production rules are a particular framework for understanding human cognition: namely the assumption that "cognitive skills are realized by production rules". ACT-R is a theory which instantiates this framework: it theorizes a general cognitive architecture, encompassing perception, attention, memory and learning, among others.
Theories, Models and Simulated Experiments
Frameworks can be elaborated by adding more assumptions to make them into "theories", which can generate predictions about particular kinds of cognitive phenomena, being thus closer to being falsifiable. Many different theories can be built from the same framework, for example many theories have been built based on the distinction between short-term and long-term memory.
However, even reasonably precise theories are not in general enough to predict behavior in a specific experiment. For this, one must make additional assumptions to define how the theory applies to the specific situation. The theory and its interpretation for the given situation (extra assumptions) make up what is called a "model". A model will provide a simulation of the mind, and will thus contain structures representing memory, perceptions, etc., and most importantly, a representation of “how cognition is implemented”. In ACT-R, the latter are represented as production rules.
In the context of cognitive modeling, an experiment is a simulation, where the model interacts with the simulated world. Of course, a cognitive model is only valuable as far as the results of its simulated experiments are mirrored by real experiments (i.e. experiments with human subjects).
The Embodiment of Theories
It would seem that the pure concept of a theory, without experiment-specific assumptions, doesn’t have a concrete, formal form in ACT-R. However, we do have informally-specified ways to create models from the same theory, and could in principle also formalize this. The difficulty would be that even formal theories may require some common-sense in order to be interpreted for specific experiments, and in fact different theorists may offer different interpretations (i.e. different predictions) of the same theory (or at least, what they call the same theory).
Why Cognitive Modeling?
Cognitive modeling forces theorists to think of the implementation level, to account for the computational aspect of the mind. Psychological theories normally exist at a high, informal level, and are, for this reason, hard to compare against each other, and hard to interpret on specific instances. Models, on the other hand, are theories which have been interpreted and formalized for a specific situation. Modeling therefore allows one to make precise predictions about possible experiments, thereby making theories very precise. Cognitive modeling may allow us to make more objective judgements about a theory's value, and thus fight our all-too-human tendency towards theoretical fallacies and oversights, including those stemming from biases and ideologies.
Difficulties in Concrete Theorizing
There are two problems which make it difficult to do concrete theorizing. The identifiability and uniqueness problems: namely finding models that fit the data, and choosing among the many candidates that remain (3:1.4).
The identifiability problem becomes slightly easier in our case, since we are translating existing informal theories, instead of searching in the space of all possible theories. However there is still plenty of room for vagueness and multiple formal interpretations of informal theories, as there are many ways of making a model from a theory. We will run into problems trying to create models from the higher-level theories we try to translate, since specific mechanisms are hardly ever explicitly suggested.
For example, Russell does not propose a mechanism to explain at what point autistic subjects are impaired in performing the box task. It could be that they fail to copy the experimenter's behavior, or that they fail to create plans, or that they have difficulty executing them.
Yet we believe his Executive Disorder theory because there exist plausible mechanisms by which failure of executive function could cause autistic subjects to fail the box task.
3. Cognitive Theories of Autism, motivated:
The False Belief Task (FBT)
Normal children fail false belief tasks until they are about 3.5 years old. It is a known fact children with autism keep failing such tasks for longer.
Meta-representation Deficit Theory (MR)
Alternatively known as Theory of Mind (ToM), mentalizing deficit or mindblindness, this theory is the most straight-forward explanation for autistic performance on the False Belief Task. It states that autists' poor performance in the FBT is due to difficulty in processing meta-representations, such as representation of other people's beliefs about the world (strangely, this does not seem to include meta-representations about photographs. Am I confusing two different theories here?). This theory seems to postulate the existence of a module for representing other people's minds.
Executive Dysfunction Theory (ED)
Executive function is defined as “the appropriate initiation and inhibition of actions”. The ED theory of autism provides an alternative to the MR theory. It says that autistic behaviors can be explained, at least partly, by their poor executive function. According to Russell(4), the FBT requires the participant to “suppress one's own true belief whilst simultaneously holding in mind the requirement to answer a question about what the protagonist will do”.
The ED theory has the advantage that it also seems to explain autistic behavior in many different kinds of experiments, and that normally developing children begin passing the FBT at around ~3.5 years of age (5), which is about the same time that they start passing box task, a supposedly purely executive task. This age would supposedly mark a development of the executive functions.
An essential component of explanations involving the ED Theory is salience: salient representations create strong prepotent responses, i.e. responses which are hard for autistic subjects to suppress. For this reason, ED theorists sometimes call the FBT the “Conflicting Beliefs Task”.
The Box Task
The False Photograph Task
Leekam, however, argues that the ED theory cannot explain autistic performance on the FBT because it has the same executive structure as a task where autists perform normally: the Zaitchik’s (1990) false photograph task.
Russell offers a counter-argument by saying that it's not clear that a 2D representation of a photograph is less salient than a 3D representation of a mind, so it’s therefore not clear that the task encourages a strong prepotent response (4). He is arguing, therefore, that autists' good performance on the FPT could be due to the correct response being relatively more salient, compared to the FBT, where the correct response involves pointing to an empty container.
To test this hypothesis, Russell created the modified false photograph test (MFPT) (Experiment 2 from [4]), with the intent of being more taxing to the executive functions: in this task, the correct response is “nothing” or pointing to an empty screen. The results show a failure rate similar to the FBT and thus seem to support the ED explanation for autistic performance on the FBT.
Conflicting Belief vs Conflicting Desire
Another piece of evidence which corroborates the ED theory is the Conflicting Desires Task (Experiment 1 from [4]). It is argued that desires are representationally different from beliefs, while making similar executive demands. The results show that children with autism do as badly here as they do on the FBT (a.k.a. the Conflicting Beliefs Task).
Some Alternative Speculative Theories of Mine
The representation of a photograph might be more salient than that of a mind if the camera produced a flash or noise.
Another alternative hypothesis of mine is that children by default assume that others' minds are the same as theirs, because this is a computationally simple and yet accurate way to deal with many situations, since children do, in fact, share a lot with most human beings. Objects perceived as non-human, such as a camera, will not activate the child's “similar mind” response, making False-Belief-like tasks involving them easier to pass. This theory makes one wonder how autistic children would respond to an anthropomorphic robot with a camera, or to animals.
4 Towards Cognitive Models of the ED Theory on the Box Task
Implementing cognitive theories of a phenomenon such as autism in all generality would be quite a big project, for it would first require implementing good models of normal competence for each experiment; and then, again for each experiment, interpreting each theory into a specific model. But this is hard because the theories, being informal and underspecified, are not always clear in their predictions: in some cases, we would need an expert to tell us what a theory would say about autistic behavior in a particular experiment, and even then, we are skeptical that this will always work, since the theories are, afterall, informal, and even experts may disagree on their proper interpretation.
Let's look at a few possible interpretations of the ED theory on the box task. Most of them are still at the informal stage. Note that each interpretation is associated with a pair of models: a normal one and an autistic one.
Van Lambalgen and Smid’s interpretation (5)
Under this interpretation, the subject starts out by devising his plan to retrieve the ball. The subject may ignore the demonstration altogether or only use it when he runs into trouble.
NORMAL PERFORMANCE
The subject starts out with the plan
IF the goal is the solve the box task AND I reach for the ball AND nothing funny is going on
THEN the ball will be retrieved
With evidence of the plan’s failure, this leads to
“something funny is going on”.
After exploring the box or remembering or figuring out what the experimenter demonstrated, the plan gets revised to:
IF the goal is the solve the box task AND the light switch is turned off AND I reach for the ball AND nothing funny is going on
THEN the ball will be retrieved
This revised plan leads to the action of hitting the switch and reaching for the ball.
Success!
AUTISTIC PERFORMANCE
A precise mechanism is not suggested, but the discussion suggests that
“the autistic child does not code this temporal sequence into a *causal* sequence which can become entrenched”
and
“we hypothesize that autists have difficulty with the reasoning *pattern* outlines above, in particular with applying closed world reasoning to unknown preconditions of actions, here formulated as ‘there is nothing funny going on’ ”
Interpretation #2: Failure to copy the experimenter
Before creating a model for this interpretation, we would need a theory of how copying someone else’s performance works.
Normal subjects copy exactly what the experimenter does at the physical level, and do the same thing without “understanding” the plan.
Autistic subjects are impaired in this ability of copying other’s physical actions / movements.
Interpretation #3: Failure to understand the experimenter’s plan
Normal subjects can understand and thus copy the experimenter’s plan at a rule level.
Autistic subjects cannot interpret the experimenter’s actions as rules. This would be related to autists’ known failure to grasp concepts.
Interpretation #4: Failure to execute the plan by failing to disengage from the ball
Normal subjects form a plan and execute it.
Autistic subjects likewise form a plan but fail at its execution, because they cannot resist the prepotent response of reaching for the ball.
One thing to note is that many of these executive failures may be happening at the same time.
To create a model pair for van Lambalgen and Smid’s interpretation of the box task, one would need to implement their theory of how humans create and revise plans. But this would be a significant project in itself.
I originally tried to implement the disengagement theory in ACT-R, but I ran into the problem that the ACT-R architecture has a uniform mechanism for weight adjustment, and it would therefore seem extremely implausible that the autistic brain would only differ on the weight-adjustment of this one rule.
A more plausible implementation of the naïve disengagement theory would have been: “autistic brains have difficulty updating activation weights of production rules in general.”
Even though a model of the disengagement theory seems implausible in ACT-R, this does not necessarily give us a reason to worry: it’s far from clear that the ACT-R theory is correct.
Note that all of these interpretations assume the same theory and aim to explain the same same phenomenon: they are all possible mechanisms for the box task.
Our aim is larger. Once we had interpretations of autistic performance on enough tasks, we would want to generalize all these mechanisms into a formal “theory of autism” (see “theory embodiment”), from which one would be able to automatically derive predictions for novel experiments. Such an automatic prediction-system could be evaluated against human interpreters or against real world experiments. This is the ultimate goal of this research program.
Conclusion
Our hope was that cognitive modeling would shed light on theories of autism. While this may be possible someday, it remains to be done. I lack knowledge of explicit theories of planning and execution which would be necessary for such an undertaking. I can only hope that the more plausible and accepted theories, after passing the tests of experimental evidence and logical criticism, will be put to this test. I believe this would indeed be the “ultimate test” for any theory of human cognition.
Thanks to Maarten van Someren.
Appendix 1: Data I Wish I Had
* A variant of the FBT with smaller executive difficulties. For example, instead of moving one object to a different box (leaving an empty box), have two objects switching places, such that no box will be empty. Are there no suggestions of an FBT with no significant executive difficulties?
How Production Rules get Updated / How Normals Learn to use the switch:
* data about time / hesitation should be useful.
* if disengagement theory is true: after first failure, autists should continue to quickly reach for the ball, as normals should stop to think.
* if it's a bug on the behavior revision loop: autists should take more time (?)
ASK the autists what they believe. ASK them if they understand what happens if the switch is pulled.
Compare theories: while ED seems to be formalizable, the meta-representation theory seems unclear.
Appendix 2: On why the knob route is easier
Russell speculates that autists find the knob route easier because it has a direct causal relationship to the goal (whereas the switch route is more indirect). This seems to presuppose that normal performance on the box task depends on the execution of a plan, and that autists have difficulty either making or carrying out such a plan (or both). (note: perhaps they don't need to recreate the plan, since they are just copying what the experiment did in front of them; i.e. the deficit is in the execution)
But how do we translate these theories into concrete production rules? One possibility is to write production rules for creating and executing plans. But ACT-R may be unsuitable for this.
References:
(1)Hughes, Russell - Autistic Children's Difficulty with Mental Disengagement from an Object: Its Implications for Theories of Autism
(DO NOT QUOTE)
(2)Stenning, van Lambalgen - Human Reasoning and Cognitive Science, p.195
(3) Anderson - Rules of the Mind
(4)Russell, Saltmarsh, Hill – What Do Executive Factors Contribute to the Failure on False Belief Tasks by Children with Autism?
(5)van Lambalgen, Smid - Reasoning patterns in autism: rules and exceptions
(6)van Rijn, van Someren, van der Maas – Modeling developmental transitions on the balance scale task
Before you leave a comment on this, I know that I need to give more background about autism and the box task.
This paper is about:
* cognitive modeling (a way of formalizing cognitive theories by implementing simulations of minds).
* cognitive theories of autism (in particular, why we believe the Executive Dysfunction theory (ED) )
* specifying possible mechanisms through which ED can account for the box task (the ultimate goal being a formalized theory of autism)
* why cognitive modeling should be the ultimate test of a theory.
My program is very ambitious, and I haven't been able to find anyone working towards the same goal.
If you don't have an LJ account, please remember to leave your name & contact info. (Unless you want to be an anonymous reviewer, that is.)
----------------------------------------------------------------------------------
Modeling Cognitive Theories of Autism
Gustavo Lacerda
1. Introduction:
Autism is a complex, pervasive abnormality of human development which manifests itself in multiple ways, including deficiencies in language, reasoning, planning and theory-of-mind, as well as super-abilities in some perceptual tasks.
There exist many theories for autism, offering different levels of explanation, for example: genetic, neurological, developmental and functional/information-processing explanations.
This paper is concerned with the latter kind: theories which explain autistic symptoms and behaviors (manifested in performance on various cognitive tasks) by postulating ways in which autists process high-level information differently from normals, and deriving the observed behaviors from these assumptions.
We intended to use a cognitive modeling system, ACT-R, in which this "derivation" occurs in the form of simulations: a cognitive model (i.e. a simulated human being) participating in a simulated experiment. We wished to see if it was possible to implement these theories into the ACT-R architecture, and if so, to implement models of them on novel experiments as a way of testing their generality and explanatory power. If such a level were achieved, it would be relatively straightforward for us to compare competing theories of autism against reality. (Psychology would be like physics). However, it seems unlikely that such a level will be reached.
In this paper, we are mostly concerned with the Executive Dysfunction Theory, as defended by Russell, versus the theory-of-mind theory.
This paper:
Ch.1 Introduction
Ch.2 introduces the logico-philosophical framework of cognitive modeling.
Ch.3 motivates, introduces and discusses competing theories of autism from a general perspective.
Ch.4 discusses possible models of the ED theory on the box task, the False Belief Task and the False Photograph Task.
2.Frameworks, Theories and Models
Frameworks
According to (3:1.2), "frameworks are composed of general claims about cognition. They are the sets of constructs that define the important aspects of cognition.", for example claims such as "there are two memory stores: a short-term store and a long-term one". Frameworks are not sufficiently specified to derive predictions. As such, they are inherently unfalsifiable.
Production rules are a particular framework for understanding human cognition: namely the assumption that "cognitive skills are realized by production rules". ACT-R is a theory which instantiates this framework: it theorizes a general cognitive architecture, encompassing perception, attention, memory and learning, among others.
Theories, Models and Simulated Experiments
Frameworks can be elaborated by adding more assumptions to make them into "theories", which can generate predictions about particular kinds of cognitive phenomena, being thus closer to being falsifiable. Many different theories can be built from the same framework, for example many theories have been built based on the distinction between short-term and long-term memory.
However, even reasonably precise theories are not in general enough to predict behavior in a specific experiment. For this, one must make additional assumptions to define how the theory applies to the specific situation. The theory and its interpretation for the given situation (extra assumptions) make up what is called a "model". A model will provide a simulation of the mind, and will thus contain structures representing memory, perceptions, etc., and most importantly, a representation of “how cognition is implemented”. In ACT-R, the latter are represented as production rules.
In the context of cognitive modeling, an experiment is a simulation, where the model interacts with the simulated world. Of course, a cognitive model is only valuable as far as the results of its simulated experiments are mirrored by real experiments (i.e. experiments with human subjects).
The Embodiment of Theories
It would seem that the pure concept of a theory, without experiment-specific assumptions, doesn’t have a concrete, formal form in ACT-R. However, we do have informally-specified ways to create models from the same theory, and could in principle also formalize this. The difficulty would be that even formal theories may require some common-sense in order to be interpreted for specific experiments, and in fact different theorists may offer different interpretations (i.e. different predictions) of the same theory (or at least, what they call the same theory).
Why Cognitive Modeling?
Cognitive modeling forces theorists to think of the implementation level, to account for the computational aspect of the mind. Psychological theories normally exist at a high, informal level, and are, for this reason, hard to compare against each other, and hard to interpret on specific instances. Models, on the other hand, are theories which have been interpreted and formalized for a specific situation. Modeling therefore allows one to make precise predictions about possible experiments, thereby making theories very precise. Cognitive modeling may allow us to make more objective judgements about a theory's value, and thus fight our all-too-human tendency towards theoretical fallacies and oversights, including those stemming from biases and ideologies.
Difficulties in Concrete Theorizing
There are two problems which make it difficult to do concrete theorizing. The identifiability and uniqueness problems: namely finding models that fit the data, and choosing among the many candidates that remain (3:1.4).
The identifiability problem becomes slightly easier in our case, since we are translating existing informal theories, instead of searching in the space of all possible theories. However there is still plenty of room for vagueness and multiple formal interpretations of informal theories, as there are many ways of making a model from a theory. We will run into problems trying to create models from the higher-level theories we try to translate, since specific mechanisms are hardly ever explicitly suggested.
For example, Russell does not propose a mechanism to explain at what point autistic subjects are impaired in performing the box task. It could be that they fail to copy the experimenter's behavior, or that they fail to create plans, or that they have difficulty executing them.
Yet we believe his Executive Disorder theory because there exist plausible mechanisms by which failure of executive function could cause autistic subjects to fail the box task.
3. Cognitive Theories of Autism, motivated:
The False Belief Task (FBT)
Normal children fail false belief tasks until they are about 3.5 years old. It is a known fact children with autism keep failing such tasks for longer.
Meta-representation Deficit Theory (MR)
Alternatively known as Theory of Mind (ToM), mentalizing deficit or mindblindness, this theory is the most straight-forward explanation for autistic performance on the False Belief Task. It states that autists' poor performance in the FBT is due to difficulty in processing meta-representations, such as representation of other people's beliefs about the world (strangely, this does not seem to include meta-representations about photographs. Am I confusing two different theories here?). This theory seems to postulate the existence of a module for representing other people's minds.
Executive Dysfunction Theory (ED)
Executive function is defined as “the appropriate initiation and inhibition of actions”. The ED theory of autism provides an alternative to the MR theory. It says that autistic behaviors can be explained, at least partly, by their poor executive function. According to Russell(4), the FBT requires the participant to “suppress one's own true belief whilst simultaneously holding in mind the requirement to answer a question about what the protagonist will do”.
The ED theory has the advantage that it also seems to explain autistic behavior in many different kinds of experiments, and that normally developing children begin passing the FBT at around ~3.5 years of age (5), which is about the same time that they start passing box task, a supposedly purely executive task. This age would supposedly mark a development of the executive functions.
An essential component of explanations involving the ED Theory is salience: salient representations create strong prepotent responses, i.e. responses which are hard for autistic subjects to suppress. For this reason, ED theorists sometimes call the FBT the “Conflicting Beliefs Task”.
The Box Task
The False Photograph Task
Leekam, however, argues that the ED theory cannot explain autistic performance on the FBT because it has the same executive structure as a task where autists perform normally: the Zaitchik’s (1990) false photograph task.
Russell offers a counter-argument by saying that it's not clear that a 2D representation of a photograph is less salient than a 3D representation of a mind, so it’s therefore not clear that the task encourages a strong prepotent response (4). He is arguing, therefore, that autists' good performance on the FPT could be due to the correct response being relatively more salient, compared to the FBT, where the correct response involves pointing to an empty container.
To test this hypothesis, Russell created the modified false photograph test (MFPT) (Experiment 2 from [4]), with the intent of being more taxing to the executive functions: in this task, the correct response is “nothing” or pointing to an empty screen. The results show a failure rate similar to the FBT and thus seem to support the ED explanation for autistic performance on the FBT.
Conflicting Belief vs Conflicting Desire
Another piece of evidence which corroborates the ED theory is the Conflicting Desires Task (Experiment 1 from [4]). It is argued that desires are representationally different from beliefs, while making similar executive demands. The results show that children with autism do as badly here as they do on the FBT (a.k.a. the Conflicting Beliefs Task).
Some Alternative Speculative Theories of Mine
The representation of a photograph might be more salient than that of a mind if the camera produced a flash or noise.
Another alternative hypothesis of mine is that children by default assume that others' minds are the same as theirs, because this is a computationally simple and yet accurate way to deal with many situations, since children do, in fact, share a lot with most human beings. Objects perceived as non-human, such as a camera, will not activate the child's “similar mind” response, making False-Belief-like tasks involving them easier to pass. This theory makes one wonder how autistic children would respond to an anthropomorphic robot with a camera, or to animals.
4 Towards Cognitive Models of the ED Theory on the Box Task
Implementing cognitive theories of a phenomenon such as autism in all generality would be quite a big project, for it would first require implementing good models of normal competence for each experiment; and then, again for each experiment, interpreting each theory into a specific model. But this is hard because the theories, being informal and underspecified, are not always clear in their predictions: in some cases, we would need an expert to tell us what a theory would say about autistic behavior in a particular experiment, and even then, we are skeptical that this will always work, since the theories are, afterall, informal, and even experts may disagree on their proper interpretation.
Let's look at a few possible interpretations of the ED theory on the box task. Most of them are still at the informal stage. Note that each interpretation is associated with a pair of models: a normal one and an autistic one.
Van Lambalgen and Smid’s interpretation (5)
Under this interpretation, the subject starts out by devising his plan to retrieve the ball. The subject may ignore the demonstration altogether or only use it when he runs into trouble.
NORMAL PERFORMANCE
The subject starts out with the plan
IF the goal is the solve the box task AND I reach for the ball AND nothing funny is going on
THEN the ball will be retrieved
With evidence of the plan’s failure, this leads to
“something funny is going on”.
After exploring the box or remembering or figuring out what the experimenter demonstrated, the plan gets revised to:
IF the goal is the solve the box task AND the light switch is turned off AND I reach for the ball AND nothing funny is going on
THEN the ball will be retrieved
This revised plan leads to the action of hitting the switch and reaching for the ball.
Success!
AUTISTIC PERFORMANCE
A precise mechanism is not suggested, but the discussion suggests that
“the autistic child does not code this temporal sequence into a *causal* sequence which can become entrenched”
and
“we hypothesize that autists have difficulty with the reasoning *pattern* outlines above, in particular with applying closed world reasoning to unknown preconditions of actions, here formulated as ‘there is nothing funny going on’ ”
Interpretation #2: Failure to copy the experimenter
Before creating a model for this interpretation, we would need a theory of how copying someone else’s performance works.
Normal subjects copy exactly what the experimenter does at the physical level, and do the same thing without “understanding” the plan.
Autistic subjects are impaired in this ability of copying other’s physical actions / movements.
Interpretation #3: Failure to understand the experimenter’s plan
Normal subjects can understand and thus copy the experimenter’s plan at a rule level.
Autistic subjects cannot interpret the experimenter’s actions as rules. This would be related to autists’ known failure to grasp concepts.
Interpretation #4: Failure to execute the plan by failing to disengage from the ball
Normal subjects form a plan and execute it.
Autistic subjects likewise form a plan but fail at its execution, because they cannot resist the prepotent response of reaching for the ball.
One thing to note is that many of these executive failures may be happening at the same time.
To create a model pair for van Lambalgen and Smid’s interpretation of the box task, one would need to implement their theory of how humans create and revise plans. But this would be a significant project in itself.
I originally tried to implement the disengagement theory in ACT-R, but I ran into the problem that the ACT-R architecture has a uniform mechanism for weight adjustment, and it would therefore seem extremely implausible that the autistic brain would only differ on the weight-adjustment of this one rule.
A more plausible implementation of the naïve disengagement theory would have been: “autistic brains have difficulty updating activation weights of production rules in general.”
Even though a model of the disengagement theory seems implausible in ACT-R, this does not necessarily give us a reason to worry: it’s far from clear that the ACT-R theory is correct.
Note that all of these interpretations assume the same theory and aim to explain the same same phenomenon: they are all possible mechanisms for the box task.
Our aim is larger. Once we had interpretations of autistic performance on enough tasks, we would want to generalize all these mechanisms into a formal “theory of autism” (see “theory embodiment”), from which one would be able to automatically derive predictions for novel experiments. Such an automatic prediction-system could be evaluated against human interpreters or against real world experiments. This is the ultimate goal of this research program.
Conclusion
Our hope was that cognitive modeling would shed light on theories of autism. While this may be possible someday, it remains to be done. I lack knowledge of explicit theories of planning and execution which would be necessary for such an undertaking. I can only hope that the more plausible and accepted theories, after passing the tests of experimental evidence and logical criticism, will be put to this test. I believe this would indeed be the “ultimate test” for any theory of human cognition.
Thanks to Maarten van Someren.
Appendix 1: Data I Wish I Had
* A variant of the FBT with smaller executive difficulties. For example, instead of moving one object to a different box (leaving an empty box), have two objects switching places, such that no box will be empty. Are there no suggestions of an FBT with no significant executive difficulties?
How Production Rules get Updated / How Normals Learn to use the switch:
* data about time / hesitation should be useful.
* if disengagement theory is true: after first failure, autists should continue to quickly reach for the ball, as normals should stop to think.
* if it's a bug on the behavior revision loop: autists should take more time (?)
ASK the autists what they believe. ASK them if they understand what happens if the switch is pulled.
Compare theories: while ED seems to be formalizable, the meta-representation theory seems unclear.
Appendix 2: On why the knob route is easier
Russell speculates that autists find the knob route easier because it has a direct causal relationship to the goal (whereas the switch route is more indirect). This seems to presuppose that normal performance on the box task depends on the execution of a plan, and that autists have difficulty either making or carrying out such a plan (or both). (note: perhaps they don't need to recreate the plan, since they are just copying what the experiment did in front of them; i.e. the deficit is in the execution)
But how do we translate these theories into concrete production rules? One possibility is to write production rules for creating and executing plans. But ACT-R may be unsuitable for this.
References:
(1)Hughes, Russell - Autistic Children's Difficulty with Mental Disengagement from an Object: Its Implications for Theories of Autism
(DO NOT QUOTE)
(2)Stenning, van Lambalgen - Human Reasoning and Cognitive Science, p.195
(3) Anderson - Rules of the Mind
(4)Russell, Saltmarsh, Hill – What Do Executive Factors Contribute to the Failure on False Belief Tasks by Children with Autism?
(5)van Lambalgen, Smid - Reasoning patterns in autism: rules and exceptions
(6)van Rijn, van Someren, van der Maas – Modeling developmental transitions on the balance scale task