learning argumentative structures
Jan. 19th, 2007 02:59 pmI finally have a concrete plan for my "learning argumentative structures" project.
(1) Make a corpus from TruthMapping.com and from iLogos logs, from arguments that have short boxes and use a formalistic style.
(1.1) Ask my sources for XML files containing the raw arguments.
(1.2) Annotate each inference (triangle) as: WTF!, Modus Ponens, Modus Tollens, UI-MP, UI-MT, EI, and some commonly-occurring fallacies.
*: I'm not sure what to do when the inference step is not a triangle (i.e. #premises!=2). At first pass I intend to just ignore them.
(2) Train a classifier with this annotated corpus. Triangle-shaped subtrees are positive examples. As negative examples, I could use triangles randomly-generated from other boxes in the argument.
(3) Use this to judge when a triangle is valid (and, in particular, what kind of inference it is). Evaluate using cross-validation.
---
Building on this idea: we may have a corpus of an argument-diagramming task, showing the source text and the diagram next to each other.
(1) Annotate it by linking (this can probably be done automatically, by text similarity)
(2) Learn how to roughly perform this task automatically (the text in the boxes is likely to be clunky).
(2.1) Evaluate: do automatically-generated diagrams correspond closely to human-made diagrams?
(3) Using the classifier above, automatically check whether the inferences in the generated map are valid. This is a judgement of the validity of the argument presented in textual form.
(4) New task for the purposes of evaluating the combined algorithms: formalize a text, and judge the validity of the inferences. Does the computer's performance correspond closely to human performance?
(1) Make a corpus from TruthMapping.com and from iLogos logs, from arguments that have short boxes and use a formalistic style.
(1.1) Ask my sources for XML files containing the raw arguments.
(1.2) Annotate each inference (triangle) as: WTF!, Modus Ponens, Modus Tollens, UI-MP, UI-MT, EI, and some commonly-occurring fallacies.
*: I'm not sure what to do when the inference step is not a triangle (i.e. #premises!=2). At first pass I intend to just ignore them.
(2) Train a classifier with this annotated corpus. Triangle-shaped subtrees are positive examples. As negative examples, I could use triangles randomly-generated from other boxes in the argument.
(3) Use this to judge when a triangle is valid (and, in particular, what kind of inference it is). Evaluate using cross-validation.
---
Building on this idea: we may have a corpus of an argument-diagramming task, showing the source text and the diagram next to each other.
(1) Annotate it by linking (this can probably be done automatically, by text similarity)
(2) Learn how to roughly perform this task automatically (the text in the boxes is likely to be clunky).
(2.1) Evaluate: do automatically-generated diagrams correspond closely to human-made diagrams?
(3) Using the classifier above, automatically check whether the inferences in the generated map are valid. This is a judgement of the validity of the argument presented in textual form.
(4) New task for the purposes of evaluating the combined algorithms: formalize a text, and judge the validity of the inferences. Does the computer's performance correspond closely to human performance?