Copy Link
Add to Bookmark
Report
AIList Digest Volume 8 Issue 068
AIList Digest Saturday, 27 Aug 1988 Volume 8 : Issue 68
Philosophy:
Connectionist model for past tense formation in English verbs
Two Points (ref AI Digests passim)
Can we human being think two different things in parallel?
Rates of change
----------------------------------------------------------------------
Date: 24 Aug 88 18:17:58 GMT
From: mind!harnad@princeton.edu (Stevan Harnad)
Subject: Connectionist model for past tense formation in English verbs
[Editor's note - Steve Pinker (MIT), and Alan Prince (Brandeis)
co-authored an article in the journal 'Cognition', critiquing Rumelhart
& McClelland's model for past tense formation in English verbs. This is
Stevan Harnad's critique of that critique. - nick]
On Pinker & Prince on Rules & Learning
Steve: Having read your Cognition paper [28(1-2) 1988] and twice seen
your talk (latest at cogsci-88), I thought I'd point out what look like
some problems with the argument (as I understand it). In reading my
comments, please bear in mind that I am NOT a connectionist; I am on
record as a sceptic about connectionism's current accomplishments (and
how they are being interpreted and extrapolated) and as an agnostic
about its future possibilities. (Because I think this issue is of
interest to the connectionist/AI community as a whole, I am branching a
copy of this challenge to connectionists and comp.ai.)
(1) An argument that pattern-associaters (henceforth "nets") cannot do
something in principle cannot be based on the fact that a particular net
(Rumelhart & McClelland [PDP Volume 2 1986 and MacWhinney 1987,
Erlbaum]) has not done it in practice.
(2) If the argument is that nets cannot learn past tense forms (from
ecologically valid samples) in principle, then it's the "in principle"
part that seems to be missing. For it certainly seems incorrect that past
tense formation is not learnable in principle. I know of no
poverty-of-the-stimulus argument for past tense formation. On the
contrary, the regularities you describe -- both in the irregulars and
the regulars -- are PRECISELY the kinds of invariances you would
expect a statistical pattern learner that was sensitive to higher
order correlations to be able to learn successfully. In particular, the
form-independent default option for the regulars should be readily
inducible from a representative sample. (This is without even
mentioning that surely no one imagines that past-tense formation is an
independent cognitive module; it is probably learned jointly with
other morphological regularities and irregularities, and there may
well be degrees-of-freedom-reducing cross-talk.)
(3) If the argument is only that nets cannot learn past tense forms without
rules, then the matter is somewhat vaguer and more equivocal, for
there are still ambiguities about what it is to be or represent a "rule."
At the least, there is the issue of "explicit" vs. "implicit"
representation of a rule, and the related Wittgensteinian distinction
between "knowing" a rule and merely being describable as behaving in
accordance with a rule. These are not crisp issues, and hence not a
solid basis for a principled critique. For example, it may well be
that what nets learn in order to form past tenses correctly is
describable as a rule, but not explicitly represented as one (as it
would be in a symbolic program); the rule may simple operate as a causal
I/O constraint. Ultimately, even conditional branching in a symbolic
program is implemented as a causal constraint; "if/then" is really
just an interpretation we can make of the software. The possibility of
making such systematic, decomposable semantic intrepretations is, of course,
precisely what distinguishes the symbolic approach from the
connectionistic one (as Fodor/Pylyshyn argue). But at the level of a few
individual "rules," it is not clear that the higher-order interpretation AS
a formal rule, and all of its connotations, is justified. In any case, the
important distinction is that the net's "rules" are LEARNED from statistical
regularities in the data, rather than BUILT IN (as they are,
coincidentally, in both symbolic AI and poverty-of-the-stimulus-governed
linguistics). [The intermediate case of formally INFERRED rules does
not seem to be at issue here.]
So here are some questions:
(a) Do you believe that English past tense formation is NOT learnable
(except as "parameter settings" on an innate structure, from
impoverished data)? If so, what are the supporting arguments for that?
(b) If past tense formation IS learnable in the usual sense (i.e.,
by trial-and-error induction of regularities from the data sample), then do
you believe that it is specifically unlearnable by nets? If so, what
are the supporting arguments for that?
(c) If past tense formation IS learnable by nets, but only if the
invariance that the net learns and that comes to causally constrain its
successful performance is describable as a "rule," what's wrong with that?
Looking forward to your commentary on Lightfoot (in Behavioral and Brain
Sciences), where poverty-of-the-stimulus IS the explicit issue, -- best
wishes, Stevan Harnad
--
Stevan Harnad ARPANET: harnad@mind.princeton.edu harnad@princeton.edu
harnad@confidence.princeton.edu srh@flash.bellcore.com harnad@mind.uucp
BITNET: harnad%mind.princeton.edu@pucc.bitnet UUCP: princeton!mind!harnad
CSNET: harnad%mind.princeton.edu@relay.cs.net
------------------------------
Date: Thu, 25 Aug 88 10:51:01 +0100
From: "Gordon Joly, Statistics, UCL"
<gordon%stats.ucl.ac.uk@ESS.Cs.Ucl.AC.UK>
Subject: Two Points (ref AI Digests passim).
[a] More people died in the fire bombing of Dresden than in Hiroshima;
the atom bomb is a more powerful image than naplam and hence we forget.
[b] With regard to what Einstein said, Heisenberg's uncertainty princinple
is also pertinent to "AI". The principle leads to the notion that the
observer influences that which is observed. So how does this affect the
observer who preforms a self analysis?
Gordon Joly.
------------------------------
Date: 25 Aug 88 14:39:01 GMT
From: hartung@nprdc.arpa (Jeff Hartung)
Reply-to: hartung@nprdc.arpa (Jeff Hartung)
Subject: Re: Can we human being think two different things in
parallel?
In a previous article, Ken Johnson writes:
>>Can we human being think two different things in parallel?
>
>I think most people have had the experience of suddenly gaining insight
>into the solution of a problem they last deliberately chewed over a few
>hours or days previously. I'd say this was evidence for the brain's
>ability to work at two or more (?) high-order tasks at the same time.
>But I look forward to reading what Real Psychologists say.
The above may demonstrate that the brain can "process" two jobs
simultaneously, but is this what we mean by "think"? If so, this still
doesn't demonstrate adequately that parallel processing is what is
going on. It may be equally true that serial processing on several
jobs is happening, only some processing is below the threshold of
awareness. Or, there may be parallel processing, but with a limited
number of processes at the level of awareness of the "thinker".
On the other hand, if we take "thinking" to mean an activity which the
"thinker" is aware of, at least in that it is going on, then there is
strong evidence that there is only limited capacity to attand to
multiple tasks simultaneously, but there is no final conclusion on this
ability as far as I know. Many studies in the ability to attand to
multiple tasks or perceptual stimuli simultaneously are still being
done.
--Jeff Hartung--
ARPA - hartung@nprdc.arpa hartung@sdics.ucsd.edu
UUCP - !ucsd!nprdc!hartung !ucsd!sdics!hartung
------------------------------
Date: Fri, 26 Aug 88 16:25:24 EDT
From: <mcharity@ATHENA.MIT.EDU>
Subject: Rates of change
In a previous article, John Nagle writes:
>... Look at Marc
>Raibert's papers. He's doing very significant work on legged locomotion.
>Progress is slow; ...
>Along the way
>are endless struggles with hydraulics, pneumatics, gyros, real-time control
>systems, and mechanical linkages. (I spent the summer of '87 overhauling
>an electrohydraulic robot, and I'm now designing a robot vehicle. I can
>sympathise.)
>... It's depressing to think that it might take
>a century to work up to a human-level AI from the bottom. Ants by 2000,
>mice by 2020 doesn't sound like an unrealistic schedule for the medium term,
>and it gives an idea of what might be a realistic rate of progress.
> I think it's going to be a long haul. But then, so was physics.
>So was chemistry. For that matter, so was electrical engineering. We
>can but push onward. Maybe someone will find the Philosopher's Stone.
>If not, we will get there the hard way. Eventually.
Continued use of a bottom-up experimental approach to AI need not
demand continued use of the current experimental MEDIUM which so
constrains the rate of change.
While today one may be better off working directly with mechanical
systems, rather than with computational simulations of mechanical
systems, it is unclear that this will be the case in 5 or 10 years.
If a summer's overhaul could be a week's hacking, you have an order of
magnitude acceleration. If your tools develop similarly, the _rate_
of change is sharply exponential.
Science, like engineering, is limited by the feedback lags of its
development cycles. Many (most?) of these lags are in information
handling. Considering our increasing competence, are current
challenges so much vaster than past as to require similar periods of
calendar time?
Mitchell Charity
------------------------------
End of AIList Digest
********************
