|
|
Invited Talk
Garbage In, Garbage Out: Thinking about what we're asking subjects to do
In this talk I want to focus on several behavioural tasks commonly used in
(psycho)linguistic research and encourage a (re)consideration of the
following issues:
1) Are we asking our subjects to do something reasonable, i.e. something we
believe that (sometimes) linguistically naive speakers understand and can
carry out?
2) Do we have sufficient reason to believe that they are doing what we think
they're doing in generating responses to our tasks?
3) Might there be better ways to get at the constructs we are interested in,
i.e. better tasks or wordings of tasks that would lead subjects to respond
in a way more directly reflective of the cognitive systems we intend to
study?
I suggest that the answers to these questions are, respectively: No in some
cases; not necessarily, in some cases; and I think so, and I invite the
participants to ask themselves and each other this question more often. Two
examples follow.
One area where it is notoriously difficult to collect consistent data
involves relative scope of two or more operators (quantifiers, modals,
negation, etc.) within a sentence. Grammaticality judgments of particular
scope readings seem difficult even for linguists, and often not worth trying
on naive subjects, even if posed in the form "Can this sentence mean that
there are at least two cars that Mary didn't wash?" etc. I submit that this
is the wrong task to use for this purpose, and that we should take a
methodological page from the book of child language acquisition.
Specifically, we should apply the Truth Value Judgment Task (Crain &
McDaniel 1985) to such problems: the linguist/experimenter describes a
real-world scenario (perhaps with a diagram, to reduce short-term memory
load) and asks the subject whether the target sentence is true in that
situation. This moves the task (almost?) out of the realm of the
metalinguistic and puts it in a maximally natural context; more importantly,
it reduces the task demands--the meaning of just one multi-quantifier
sentence needs to be computed. By contrast, the traditional scenario
described above requires processing two sentences: the target ("Mary didn't
wash two cars") and the (supposedly) unambiguous paraphrase, and comparing
the resulting meanings, which is complicated by the fact that the sentences
overlap almost completely in lexical content and are hence hard to keep
apart.
A second example arises in the domain of morphology: the Wug test (Gleason
1958), where subjects are induced to generated forms from an inflectional
paradigm for a nonce root. On all sides of the "Dual Mechanism" debates, the
results are taken to reflect the subject's current representation of
knowledge of the inflectional system (whether it be rules, associations, or
both), but this ignores another crucial module of the mind. What are we
really asking subjects to do when we ask them to imagine the existence of a
nonce verb like 'spling'? There are two importantly different tasks they
could implicitly undertake. One is what we can call the dictionary scenario:
We are saying "I bet you didn't know this, but if you look in the dictionary
you'll find English has an obscure verb "spling"; now, tell me what you
think its past tense is." The other is what we can call the neologism
scenario: We are saying "I've just decided to make up a new word to describe
this really cool way of bouncing a paper clip that I discovered. Now, how do
you think I should say it in the past tense?" (The latter is more or less
explicitly how the task is presented to children.)
Under the dictionary scenario, it would be reasonable for the subject to
make a judgment about the shape of the (static) lexicon of English: for a
speaker of 50 years ago who had 'spling' in their vocabulary along with all
the words the subject already knows, how would that verb most likely have
been inflected? Under the neologism scenario, it would be reasonable for the
subject to do a couple of things: 1) invoke knowledge of how novel or
"weird" words inflect (as has been argued for the alleged default German
noun plural -s); or 2) treat the word as if they were acquiring it in the
normal course of everyday events, i.e. run it through the Language
Acquisition Device (LAD), the aforementioned neglected player in this mental
game. Now, given what we know about the history of irregular inflection,
most likely our LAD does not simply dump new roots into the default class
until it encounters direct evidence for an irregularity; rather, it probably
makes guesses as to inflectional patterning based on the shape of the root
(in languages that have such patterns). This would be the same sort of
"guess" that the LAD of speakers 50 years ago would have made during
childhood, so it might lead to responses similar to the dictionary scenario.
Thus, two points should be clear: there is a disturbing amount of leeway in
how subjects could respond to a Wug task, and it is not immediately obvious
which kind of response we want them to make, indeed this may differ with
one's theoretical perspective. (E.g., the Albright & Hayes learner seeks to
model Wug test behavior with a system of rules that play no role in
producing forms of words that a speaker already knows.)
|
