Copy Link
Add to Bookmark
Report
Neuron Digest Volume 06 Number 31
Neuron Digest Friday, 11 May 1990 Volume 6 : Issue 31
Today's Topics:
Response to David Lovell re Defense Related Research
Re: Neuron Digest V6 #30
Research Studentships in Cognitive Science
NIPS proceedings
Neural Networks and Fractal Geometry
ICSI Deputy Ad
TR available
Summary and tech report and thesis availability (long)
new technical report available
2 TRs available
preprint: Predicting the Future (Weigend, Huberman, Rumelhart)
PREPRINT: Contrastive Hebbian
Send submissions, questions, address maintenance and requests for old issues to
"neuron-request@hplabs.hp.com" or "{any backbone,uunet}!hplabs!neuron-request"
Use "ftp" to get old issues from hplpm.hpl.hp.com (15.255.176.205).
------------------------------------------------------------
Subject: Response to David Lovell re Defense Related Research
From: nelsonde%avlab.dnet@wrdc.af.mil
Date: Wed, 09 May 90 11:24:52 -0400
I know that many will think that my response is tainted because I work
for the Air Force but, I assure you it is not. I think because of my
job, that maybe I have a little more insight.
I get tired of people constantly hitting on the defense establishment as
wanting war and the characterization as baby killers. As Peter pointed
out, much of the research that is being accomplished in our universities
today, including fellowships to many of the graduate students, is
provided by the DoD. My group supports universities with close to $7M
each year. Also, all who are using this E-mail network should realize
that it too was funded by DoD. Many of the advances made for the
military are actively being used in every day life. For instance,
microwave ovens, transistors, VLSI chips, and computers to name just a
few. If it were not for the military, these things would probably not
have been developed, or not developed this soon.
Another thing we need to learn in the United States is to learn from
history. Do you think that the situation we see in the USSR today would
have happened without a strong defense posture? The reason there was a
WWII was that after WWI the countries dismantled their armies permitting
Germany to make great advances without much opposition. We in the US
prepare not to fight but so that we won't have to fight.
In this time of lessening world tension, let us use the DoD to support
research which has application to many areas, including defense. Let us
do this in anticipation of preventing war, not causing it. Remember, the
purpose of SDI, is to DEFEND not attack. Perhaps we can truly make DoD a
department of DEFENSE not offense. I feel that money spent in this way
will provide jobs and products within the US. It is far better to spend
money and get product for it than to just give it away.
Taking money from the defense budget and giving it to Central America or
the USSR buys NOTHING and removes it from the US economy! This money can
better be spent either to reduce the deficit, without raising taxes, or
to fund basic research within the US and the US universities.
Dale E. Nelson
------------------------------
Subject: Re: Neuron Digest V6 #30
From: J. P. Letellier <jp@radar.nrl.navy.mil>
Date: Wed, 09 May 90 09:24:50 -0400
Re: Defense and Neural Networks
In response to David Lovell's letter: The line in research between
what is a true defensive weapon and what is an offensive (pun included)
weapon is truly not clear. If a radar system had been able to better assist
the operator in determining the difference between an F-14 on an attack
mission and an airliner on a flaky schedule, a couple of hundred innocent
people might not have been killed. However, when you are in harm's way,
and afraid that you may die, the tendency is to take the conservative
approach - shoot dangerous enemy and maybe dangerous enemy. And, before the
obvious response that we should never put our forces in harm's way, consider
the history of peoples who have refused to defend themselves. The bullies
keep on coming until they are stopped. In a more perfect world, there
would be a just court which would settle all the disputes between peoples.
But even there, there is no justice sometimes. Believe it or not, there
are questions in which neither party is right or wrong, and in that case
the weak is forced to yield unless the strong is in some way moral about the
use of its available force.
On a more pragmatic note, even this network depends on the oil
that flows from the Middle East. We would not be in such a resource
bind if we had paid proper attention to natural resource management,
alternate power sources, and population control. But we are in that
fix, and seem to be doing nothing to get out of it. Maybe David could
work on a cheap neural net to keep solar collectors at maximum efficiency.
This is a serious suggestion, and not a put down. Everything which helps
us be self sufficient helps defense. And, :-) packaged as making autonomous
units (forts, ships, air bases) more self sustaining can make this into
a "Defense" proposal! (-;
jp
------------------------------
Subject: Research Studentships in Cognitive Science
From: tony@cogs1.essex.ac.uk (Lawson T)
Organization: University of Essex, Colchester, UK
Date: 08 May 90 09:43:08 +0000
University of Essex
COGNITIVE SCIENCE CENTRE
RESEARCH STUDENTSHIPS IN
COGNITIVE SCIENCE
Applications are invited for three research studentships, funded via the joint
ESRC/MRC/SERC Initiative in Cognitive Science and HCI. Potential areas of
research include:
Distributed AI and socio-cultural modelling
HCI for multiple agent systems
Formal models of communication
Connectionist knowledge representations
Cognitive phonetics and neural networks
Text content analysis by learning
The research studentships are tenable for three years from 1 October 1990, with
a closing date for applications of 15 June 1990. Further particulars and
application forms are available from:
Professor Simon Lavington
Director, Cognitive Science Centre
c/o Department of Computer Science
University of Essex
COLCHESTER CO4 3SQ
(0206 872677. E-mail: lavington@uk.ac.essex).
Ann
------------------------------
Subject: NIPS proceedings
From: Dave.Touretzky@DST.BOLTZ.CS.CMU.EDU
Date: Tue, 08 May 90 19:49:28 -0400
The proceedings of the 1989 NIPS conference have started arriving in
people's mailboxes. They were supposed to be out a few weeks ago, but
there was a problem with the quality of the binding, so Morgan Kaufmann
sent the whole batch back to the bindery to be redone. The second time
around they got it perfect.
If you are an author or co-author of a paper in the volume, OR if you
attended the conference, you should receive a copy of the proceedings.
If you don't get yours some time this week, call Morgan Kaufmann on
Monday to check on it. Their number is 415-578-9911; ask for Shirley
Jowell.
If you would like to order extra copies of the proceedings, they are
available from:
Morgan Kaufmann Publishers
2929 Campus Drive, Suite 260
San Mateo, CA 94403
tel. 415-965-4081 (order department)
fax: 415-578-0672.
Enclose a check for $35.95 per copy, plus shipping charge of $3.50 for
the first copy and $2.50 for each additional copy. California residents
must add sales tax. There are higher shipping charges for air mail or
international orders; contact the publisher for information. Note: the
catalog code for this volume is "100-7"; include that in you order.
An example of proper citation format for the volume is:
Cowan, J. D. (1990) Neural networks: the early days. In
D. S. Touretzky (ed.), Advances in Neural Information Processing
Systems 2, pp. 828-842. San Mateo, CA: Morgan Kaufmann.
-- Dave
------------------------------
Subject: Neural Networks and Fractal Geometry
From: bmcconne@vtssi.vt.edu (Brian McConnell)
Date: Tue, 08 May 90 21:21:43 -0700
Hello,
I am a layman when it comes to neural networks, so I suppose you
can place this under the half-baked ideas category.
I was curious to what degree the work being done in manufacturing
neural networks and in fractal geometry is related. I recently read
an article outlining the similarities between fractal geometry and
biological systems such as neurons. Since it appears there is a
relationship, however tenuous, I was wondering if the principles of
fractal geomtery (i.e. self-similiarity, etc.) have been widely
applied to the manufacture and simulation of neural networks.
I don't wish to clutter the net with a layman's discussion, so if
someone would be kind enough to mail me a response, I would
appreciate it. Thank you,
Brian McConnell, Undergraduate
Virginia Tech
------------------------------
Subject: ICSI Deputy Ad
From: Jerry Feldman <jfeldman%icsib2.Berkeley.EDU@jade.berkeley.edu>
Date: Wed, 09 May 90 09:18:52 -0700
We are starting a search for a full-time Deputy Director for the
International Computer Science Institute (ICSI). We would highly
appreciate any help you can give us in this search. The enclosed ad
describes the position. Please feel free to distribute it electronically
to anybody who might be interested. Thank you in advance, and best
regards.
Jerry Feldman Domenico Ferrari
PS: If you need more information about duties and perks, please
let us know.
===============================================================
DEPUTY DIRECTOR
International Computer Science Institute
Nominations and Applications are solicited for the position of
Deputy Director of the International Computer Science Institute. The
Institute is an independent basic research laboratory affiliated with
and physically near the University of California at Berkeley. Support
comes from U.S. sources and sponsor nations, currently Germany, Italy
and Switzerland.
The Deputy Director will have the primary responsibility for the
internal administration of the Institute and its post-doctoral and
exchange programs with sponsor nations. There are also many
opportunities for new initiatives. The position is like the chair of
a research oriented computer science department and the ideal
candidate would have such experience. ICSI is also expanding its
research staff and welcomes applications from outstanding scientists
at any post-doctoral level.
Please respond to:
Dr. Domenico Ferrari
Deputy Director
International Computer Science Institute
1947 Center Street, Suite 600
Berkeley, CA 94704-1105
------------------------------
Subject: TR available
From: Arun Jagota <jagota@cs.Buffalo.EDU>
Date: Tue, 13 Mar 90 16:09:48 -0500
The following technical report is available:
A Hopfield-style network for content-addressable memories
Arun Jagota
Department of Computer Science
State University Of New York At Buffalo
90-02
ABSTRACT
With the binary Hopfield network as a basis, new learning and energy
descent rules are developed. It is shown, using graph theoretic
techniques, that the stable states of the network are the maximal cliques
in the underlying graph and that the network can store an arbitrary
collection of memories without interference (memory loss, unstable fixed
points). In that sense (and that sense alone), the network has
exponential capacity (upto 2^(n/2) memories can be stored in an n-unit
network). Spurious memories can (and are likely) to develop. No
analytical results for these are derived, but important links are
established between the storage and recall properties of the network and
the properties of the memories that are stored. In particular it is
shown, partly by analysing the graph underlying the network, that the
network retrieval time and other desirable properties depend on the
'sparse-ness' of the memories and whether they have a 'combinatorial'
structure (as defined in the report). It is shown that the network
converges in <= n iterations and for sparse memories (and initial states)
with sparseness k, 0 < k < n, it converges in <= k iterations.
------------------------------------------------------------------------
The report is available in PostScript form by anonymous ftp as follows:
unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62)
Name: anonymous
Password: neuron
ftp> cd pub/neuroprose/Inbox
ftp> binary
ftp> get jagota.hsn.ps.Z
ftp> quit
unix> uncompress jagota.hsn.ps.Z
unix> lpr jagota.hsn.ps (use flag your printer needs for Postscript)
[sometime soon, the report may be moved to pub/neuroprose]
------------------------------------------------------------------------
It is recommended that hard-copy requests be made only if it is not
possible (or too inconvenient) to access the report via ftp.
I have developed a software simulator that I am willing to share with
individuals who might be interested (now or later). It has been carefully
'tuned' for this particular model, implementing the network algorithms in
a most efficient manner. It allows configurability, (any size 1-layer
net, other parameters etc) and provides a convenient 'symbolic'
interface.
For hard-copy (and/or simulator) requests send e-mail (or write to) the
following address. Please do not reply with 'r' or 'R' to this message.
Arun Jagota
e-mail: jagota@cs.buffalo.edu
Dept Of Computer Science
226 Bell Hall,
State University Of New York At Buffalo,
NY - 14260
------------------------------
Subject: Summary and tech report and thesis availability (long)
From: Tony Robinson <ajr%engineering.cambridge.ac.uk@NSFnet-Relay.AC.UK>
Date: Thu, 15 Mar 90 19:41:28 +0000
There are three topics in this (long) posting:
Summary of replies to my message "problems with large training sets".
Tech report availability announcement "Phoneme Recognition from the TIMIT
database using Recurrent Error Propagation Networks"
Thesis availability announcement "Dynamic Error Propagation Networks"
Mail me (ajr@eng.cam.ac.uk) if you would like a copy of the tech report and
thesis (I will be at ICASSP if anyone there would like to discuss (or save me
some postage)).
Tony Robinson
/*****************************************************************************/
Subject: Summary of replies to my message "problems with large training sets"
Thanks to: Ron Cole, Geoff Hinton, Yann Le Cun, Alexander Singer, Fu-Sheng
Tsung, Guy Smith and Rich Sutton for their replies, here is a brief summary:
Adaptive learning rates: The paper that was most recommended was:
Jacobs, R. A. (1988)
Increased rates of convergence through learning rate adaptation.
{Neural Networks}, {\em 1} pp 295-307.
The scheme described in this paper is nice in that it allows the step
size scaling factor (\eta) for each weight to vary independently and
variations of two orders of magnitude have been observed.
Use a faster machine: Something like a 2.7 GFlop Connection Machine could
shake some of these problems away! There are two issues here, one is
understanding the problem from which more efficient algorithms naturally
develop, the other is the need to get results. I don't know how the two
will balance in future, but my guess is that we will need more compute.
Combined subset training: Several people have used small subsets for initial
training, with later training combining these subsets. The reference I was
sent was:
Fu-Sheng Tsung and Garrison Cottrell (1989)
A Sequential Adder with Recurrent Networks
IJCNN 89, June, Washington D.C
For reasons of software homogeneity, I prefer to use an increasing momentum
term, initially it smooths over one "subset" but this increases until the
smoothing is over the whole training set. I've never done a comparison of
these techniques.
Use of higher order derivatives: A good step size can be estimated from the
second order derivatives. To me this looks very promising, but I haven't
had time to play with it yet. The reference is:
Le Cun, Y.: "Generalization and Network Design Strategies",
Tech Report CRG-TR-89-4, Dept. of computer science,
University of Toronto, 1989.
/*****************************************************************************/
Subject: Tech report availability announcement:
Phoneme Recognition from the TIMIT database using
Recurrent Error Propagation Networks
CUED/F-INFENG/TR.42
Tony Robinson and Frank Fallside
Cambridge University Engineering Department,
Trumpington Street, Cambridge, England.
Enquiries to: ajr@eng.cam.ac.uk
This report describes a speaker independent phoneme recognition system
based on the recurrent error propagation network recogniser described in
(RobinsonFallside89, FallsideLuckeMarslandOSheaOwenPragerRobinsonRussell90).
This recogniser employs a preprocessor which generates a range of types
of output including bark scaled spectrum, energy and estimates of formant
positions. The preprocessor feeds a fully recurrent error propagation
network whose outputs are estimates of the probability that the given
frame is part of a particular phonetic segment. The network is trained
with a new variation on the stochastic gradient descent procedure which
updates the weights by an adaptive step size in the direction given by
the sign of the gradient. Once trained, a dynamic programming match is
made to find the most probable symbol string of phonetic segments. The
recognition rate is improved considerably when duration and bigram
probabilities are used to constrain the symbol string.
A set of recognition results is presented for the trade off between
insertion and deletion errors. When these two errors balance the
recognition rate for all 61 TIMIT symbols is 68.6% correct (62.5%
including insertion errors) and on a reduced 39 symbol set the
recognition rate is 75.1% correct (68.9%). This compares favourably with
the results of other methods on the same database
(ZueGlassPhillipsSeneff89, DigalakisOstendorfRohlicek89, HataokaWaibel89,
LeeHon89, LevinsonLibermanLjoljeMiller89).
/*****************************************************************************/
Subject: Thesis availability announcement "Dynamic Error Propagation Networks"
Please forgive me for the title, a better one would have been "Recurrent
Error Propagation Networks". This is my PhD thesis, submitted in Feb
1989 and is a concatenation of the work I had done to that date.
Summary:
This thesis extends the error propagation network to deal with time
varying or dynamic patterns. Examples are given of supervised,
reinforcement driven and unsupervised learning.
Chapter 1 presents an overview of connectionist models.
Chapter 2 introduces the error propagation algorithm for general node types.
Chapter 3 discusses the issue of data representation in connectionist models.
Chapter 4 describes the use of several types of networks applied to the
problem of the recognition of steady state vowels from multiple speakers.
Chapter 5 extends the error propagation algorithm to deal with time
varying input. Three possible architectures are explored which deal with
learning sequences of known length and sequences of unknown and possibly
indefinite length. Several simple examples are given.
Chapter 6 describes the use of two dynamic nets to form a speech coder.
The popular method of Differential Pulse Code Modulation for speech
coding employs two linear filters to encoded and decode speech. By
generalising these to non-linear filters, implemented as dynamic nets, a
reduction in the noise imposed by a limited bandwidth channel is
achieved.
Chapter 7 describes the application of a dynamic net to the recognition
of a large subset of the phonemes of English from continuous speech. The
dynamic net is found to give a higher recognition rate both in comparison
with a fixed window net and with the established k nearest neighbour
technique.
Chapter 8 describes a further development of dynamic nets which allows
them to be trained by a reinforcement signal which expresses the
correctness of the output of the net. Two possible architectures are
given and an example of learning to play the game of noughts and crosses
is presented.
------------------------------
Subject: new technical report available
From: jacobs@gluttony.cs.umass.edu
Date: Wed, 28 Mar 90 09:42:52 -0500
The following technical report is now available:
Task Decomposition Through Competition In
a Modular Connectionist Architecture:
The What and Where Vision Tasks
Robert A. Jacobs (UMass)
Michael I. Jordan (MIT)
Andrew G. Barto (UMASS)
COINS Technical Report 90-27
Abstract
--------
A novel modular connectionist architecture is presented in which the
networks composing the architecture compete to learn the training
patterns. An outcome of the competition is that different networks learn
different training patterns and, thus, learn to compute different
functions. The architecture performs task decomposition in the sense
that it learns to partition a task into two or more functionally
independent tasks and allocates distinct networks to learn each task. In
addition, the architecture tends to allocate to each task the network
whose topology is most appropriate to that task. The architecture's
performance on ``what'' and ``where'' vision tasks is presented and
compared with the performance of two multi--layer networks. Finally, it
is noted that function decomposition is an underconstrained problem and,
thus, different modular architectures may decompose a function in
different ways. We argue that a desirable decomposition can be achieved
if the architecture is suitably restricted in the types of functions that
it can compute. Appropriate restrictions can be found through the
application of domain knowledge. A strength of the modular architecture
is that its structure is well--suited for incorporating domain knowledge.
If possible, please obtain a postscript version of this technical report
from the pub/neuroprose directory at cheops.cis.ohio-state.edu.
Here are the directions:
unix> ftp cheops.cis.ohio-state.edu # (or ftp 128.146.8.62)
Name (cheops.cis.ohio-state.edu:): anonymous
Password (cheops.cis.ohio-state.edu:anonymous): neuron
ftp> cd pub/neuroprose
ftp> type binary
ftp> get
(remote-file) jacobs.modular.ps.Z
(local-file) foo.ps.Z
ftp> quit
unix> uncompress foo.ps.Z
unix> lpr -P(your_local_postscript_printer) foo.ps
If you do not have access to a postscript printer, copies of this
technical report can be obtained by sending requests to Connie Smith at
smith@cs.umass.edu. Remember to ask for COINS Technical Report 90-27.
------------------------------
Subject: 2 TRs available
From: Andreas Stolcke <stolcke%icsib12.Berkeley.EDU@jade.berkeley.edu>
Date: Wed, 18 Apr 90 18:31:33 +0100
The following Technical Reports are available. Please refer to the end of
this message for information on how to obtain them.
----------------------------------------------------------------------------
MINIATURE LANGUAGE ACQUISITION:
A TOUCHSTONE FOR COGNITIVE SCIENCE
Jerome A. Feldman, George Lakoff, Andreas Stolcke and Susan Hollbach Weber
International Computer Science Institute
Technical Report TR-90-009
March 1990
ABSTRACT
Cognitive Science, whose genesis was interdisciplinary, shows signs of
reverting to a disjoint collection of fields. This paper presents a
compact, theory-free task that inherently requires an integrated
solution. The basic problem is learning a subset of an arbitrary natural
language from picture-sentence pairs. We describe a very specific
instance of this task and show how it presents fundamental (but not
impossible) challenges to several areas of cognitive science including
vision, language, inference and learning.
-----------------------------------------------------------------------------
LEARNING FEATURE-BASED SEMANTICS WITH SIMPLE RECURRENT NETWORKS
Andreas Stolcke
International Computer Science Institute
Technical Report TR-90-015
April 1990
ABSTRACT
The paper investigates the possibilities for using simple recurrent
networks as transducers which map sequential natural language input into
non-sequential feature-based semantics. The networks perform well on
sentences containing a single main predicate (encoded by transitive verbs
or prepositions) applied to multiple-feature objects (encoded as
noun-phrases with adjectival modifiers), and shows robustness against
ungrammatical inputs. A second set of experiments deals with sentences
containing embedded structures. Here the network is able to process
multiple levels of sentence-final embeddings but only one level of
center-embedding. This turns out to be a consequence of the network's
inability to retain information that is not reflected in the outputs over
intermediate phases of processing. Two extensions to Elman's original
recurrent network architecture are introduced.
-----------------------------------------------------------------------------
[Versions of these papers have been submitted to the 12th Annual
Conference of the Cognitive Science Society.]
The reports can be obtained as compressed PostScript files from host
cis.ohio-state.edu via anonymous ftp. The filenames are
feldman.tr90-9.ps.Z and
stolcke.tr90-15.ps.Z
in directory /pub/neuroprose.
Hardcopies may be requested via e-mail to
weber@icsi.berkeley.edu or
stolcke@icsi.berkeley.edu
or physical mail to one of the authors at the following address:
International Computer Science Institute
1947 Center Street, Suite 600
Berkeley, CA 94704
U.S.A.
-----------------------------------------------------------------------------
Andreas Stolcke
------------------------------
Subject: preprint: Predicting the Future (Weigend, Huberman, Rumelhart)
From: Andreas Weigend <andreas@psych.Stanford.EDU>
Date: Tue, 24 Apr 90 17:11:08 -0700
______________________________
PREDICTING THE FUTURE -
A CONNECTIONIST APPROACH
______________________________
Andreas S. Weigend [1]
Bernardo A. Huberman [2]
David E. Rumelhart [3]
______________________________
We investigate the effectiveness of connectionist networks
for predicting the behavior of non-linear dynamical systems. We use
feed-forward networks of the type used by Lapedes and Farber to ob-
tain forecasts in the context of noisy real world data from sunspots
and computational ecosystems. The networks generate accurate future
predictions from knowledge of the past and consistently outperform
traditional statistical non-linear approaches to these problems.
The problem of having too many weights compared to the number of data
points (overfitting) is addressed by adding a term to the cost function
that penalizes large weights. We show that this weight-elimination
procedure successfully shrinks the net down. We compare different
classes of activation functions and explain why the convergence of
sigmoids is significantly better than the convergence of of radial
basis functions for higher dimensional input. We suggest the use
of the concept of mutual information to interpret the weights. We
introduce two measures of non-linearity and compare the sunspot and
ecosystem data to a series generated by a linear autoregressive model.
The solution for the sunspot data is found to be moderately non-linear,
the solution for the prediction of the ecosystem highly non-linear.
Submitted to "International Journal of Neural Systems"
If you would really like a copy of the preprint, send
your physical address to: hershey@psych.stanford.edu
(preprint number: Stanford-PDP-90-01, PARC-SSL-90-20)
[1] Physics Department, Stanford University, Stanford, CA 94305
[2] Dynamics of Computation Group, Xerox PARC, Palo Alto, CA 94304
[3] Psychology Department, Stanford University, Stanford, CA 94305
------------------------------
Subject: PREPRINT: Contrastive Hebbian
From: Javier Movellan <jm2z+@ANDREW.CMU.EDU>
Date: Thu, 26 Apr 90 19:05:26 -0400
This preprint has been placed in the account kindly provided by Ohio State.
CONTRASTIVE HEBBIAN LEARNING IN INTERACTIVE NETWORKS
Javier R. Movellan
Department of Psychology
Carnegie Mellon University
Pittsburgh, Pa 15213
email: jm2z+@andrew.cmu.edu
Submitted to Neural Computation
Interactive networks, as defined by Hopfield (1984), Grossberg (1978),
and McClelland & Rumelhart(1981) may have an advantage over feed-forward
architectures because of their completion properties, and flexibility in
the treatment of units as inputs or outputs. Ackley, Hinton and Sejnowski
(1985) derived a learning rule to train Boltzmann machines, which are
discrete, interactive networks. Unfortunately, because of the discrete
stochasticity of its units, Boltzmann learning is intolerably slow.
Peterson and Anderson (1987) showed that Boltzmann machines with large
number of units can be approximated with deterministic networks whose
logistic activations represent the average activation of discrete
Boltzmann units (Mean Field Approximation). Under these conditions a
learning rule that I call Contrastive Hebbian Learning (CHL) was shown to
be a good approximation to the Boltzmann weight update rule and to
achieve learning speeds comparable to backpropagation. Hinton (1989)
showed that for Mean Field networks, CHL is at least a first order
approximation to gradient descent on an error function. The purpose in
this paper is to show that CHL works with any interactive network with
bounded, continuous activation functions and symmetric weights. The
approach taken does not presume the existence of Boltzmann machines whose
behavior is approximated with mean field networks. It is also shown that
CHL performs gradient descent on a contrastive function of the same form
investigated by Hinton (1989) The paper is divided in two sections and
one appendix. In Section 1 I study the dynamics of the activations in
interactive networks. Section 2 shows how to modify the weights for the
stable states of the network to reproduce desired patterns of
activations. The appendix contains mathematical details, and some
comments on how to implement Contrastive Hebbian Learning in Interactive
Networks.
The format is Latex. Here are the instructions to get the file:
unix> ftp cheops.cis.ohio-state.edu
Name:anonymous
Password:neuron
ftp> cd pub/neuroprose
ftp> get Movellan.CHL.LateX
------------------------------
End of Neuron Digest [Volume 6 Issue 31]
****************************************
