Copy Link
Add to Bookmark
Report
Neuron Digest Volume 09 Number 05
Neuron Digest Tuesday, 11 Feb 1992 Volume 9 : Issue 5
Today's Topics:
paper on 2nd order methods in Neuroprose
preprints and thesis TR -- Connectionist Neuropsychology
New book - Neural Network Parallel Computing
TR - Training algorithms for limited precision nets
PREPRINT - Currency exchange prediction via neural nets
TR Top-down teaching enables non-trivial clustering
International Comparison of Learning Algorithms: MONK
Send submissions, questions, address maintenance, and requests for old
issues to "neuron-request@cattell.psych.upenn.edu". The ftp archives are
available from cattell.psych.upenn.edu (128.91.2.173). Back issues
requested by mail will eventually be sent, but may take a while.
----------------------------------------------------------------------
Subject: paper on 2nd order methods in Neuroprose
From: BATTITI@ITNVAX.CINECA.IT
Date: Thu, 14 Nov 91 10:01:00 +0100
A new paper is available from the Neuroprose directory.
FILE: battiti.second.ps.Z (ftp binary, uncompress, lpr (PostScript))
TITLE: "First and Second-Order Methods for Learning:
between Steepest Descent and Newton's Method"
AUTHOR: Roberto Battiti
ABSTRACT: On-line first order backpropagation is sufficiently fast
and effective for many large-scale classification problems but for
very high precision mappings, batch processing may be the method of
choice.This paper reviews first- and second-order optimization methods
for learning in feed-forward neural networks. The viewpoint is that
of optimization: many methods can be cast in the language of optimiza-
tion techniques, allowing the transfer to neural nets of detailed
results about computational complexity and safety procedures to ensure
convergence and to avoid numerical problems.
The review is not intended to deliver detailed prescriptions for the
most appropriate methods in specific applications, but to illustrate
the main characteristics of the different methods and their mutual
relations.
PS: the paper will be published in Neural Computation.
PPSS: comments and/or new results welcome.
======================================================================
| | |
| Roberto Battiti | e-mail: battiti@itnvax.cineca.it |
| Dipartimento di Matematica | tel: (+39) - 461 - 88 - 1639 |
| 38050 Povo (Trento) - ITALY | fax: (+39) - 461 - 88 - 1624 |
| | |
======================================================================
------------------------------
Subject: preprints and thesis TR -- Connectionist Neuropsychology
From: David_Plaut@K.GP.CS.CMU.EDU
Date: Thu, 14 Nov 91 09:58:02 -0500
I've placed two papers in the neuroprose archive. Instructions for
retrieving them are at the end of the message. (Thanks again to Jordan
Pollack for maintaining the archive.)
The first (plaut.thesis-summary.ps.Z) is a 15 page summary of my thesis,
entitled "Connectionist Neuropsychology: The Breakdown and Recovery of
Behavior in Lesioned Attractor Networks" (abstract below).
For people who want more detail, the second paper (plaut.dyslexia.ps.Z)
is a 119 page TR, co-authored with Tim Shallice, that presents a
systematic analysis of work by Hinton & Shallice on modeling deep
dyslexia, extending the approach to a more comprehensive account of the
syndrome. FTP'ers should be forewarned that the file is about 0.5 Mbytes
compressed, 1.8 Mbytes uncompressed.
For true die-hards, the full thesis (325 pages) is available as CMU-CS-91-185
from
Computer Science Documentation
School of Computer Science
Carnegie Mellon University
Pittsburgh, PA 15213-3890
reports@cs.cmu.edu
To defray printing/mailing costs, requests for the thesis TR must be
accompanied by a check or money order for US$10 (domestic) or US$15
(overseas) payable to "Carnegie Mellon University."
Enjoy,
Dave
Connectionist Neuropsychology:
The Breakdown and Recovery of Behavior
in Lesioned Attractor Networks
David C. Plaut
An often-cited advantage of connectionist networks is that they degrade
gracefully under damage. Most demonstrations of the effects of damage
and subsequent relearning in these networks have only looked at very
general measures of performance. More recent studies suggest that damage
in connectionist networks can reproduce the specific patterns of behavior
of patients with neurological damage, supporting the claim that these
networks provide insight into the neural implementation of cognitive
processes. However, the existing demonstrations are not very general,
and there is little understanding of what underlying principles are
responsible for the results. This thesis investigates the effects of
damage in connectionist networks in order to analyze their behavior more
thoroughly and assess their effectiveness and generality in reproducing
neuropsychological phenomena.
We focus on connectionist networks that make familiar patterns of
activity into stable ``attractors.'' Unit interactions cause similar but
unfamiliar patterns to move towards the nearest familiar pattern,
providing a type of ``clean-up.'' In unstructured tasks, in which inputs
and outputs are arbitrarily related, the boundaries between attractors
can help ``pull apart'' very similar inputs into very different final
patterns. Errors arise when damage causes the network to settle into a
neighboring but incorrect attractor. In this way, the pattern of errors
produced by the damaged network reflects the layout of the attractors
that develop through learning.
In a series of simulations in the domain of reading via meaning,
networks are trained to pronounce written words via a simplified
representation of their semantics. This task is unstructured in the
sense that there is no intrinsic relationship between a word and its
meaning. Under damage, the networks produce errors that show a
distribution of visual and semantic influences quite similar to that of
brain-injured patients with ``deep dyslexia.'' Further simulations
replicate other characteristics of these patients, including additional
error types, better performance on concrete vs.\ abstract words,
preserved lexical decision, and greater confidence in visual vs.\
semantic errors. A range of network architectures and learning
procedures produce qualitatively similar results, demonstrating that the
layout of attractors depends more on the nature of the task than on the
architectural details of the network that enable the attractors to
develop.
Additional simulations address issues in relearning after damage: the
speed of recovery, degree of generalization, and strategies for
optimizing recovery. Relative differences in the degree of relearning
and generalization for different network lesion locations can be
understood in terms of the amount of structure in the subtasks performed
by parts of the network.
Finally, in the related domain of object recognition, a similar
network is trained to generate semantic representations of objects from
high-level visual representations. In addition to the standard weights,
the network has correlational weights useful for implementing short-term
associative memory. Under damage, the network exhibits the complex
semantic and perseverative effects of patients with a visual naming
disorder known as ``optic aphasia,'' in which previously presented
objects influence the response to the current object. Like optic
aphasics, the network produces predominantly semantic rather than visual
errors because, in contrast to reading, there is some structure in the
mapping from visual to semantic representations for objects.
Taken together, the results of the thesis demonstrate that the
breakdown and recovery of behavior in lesioned attractor networks
reproduces specific neuropsychological phenomena by virtue of the way the
structure of a task shapes the layout of attractors.
unix> ftp 128.146.8.52
Name: anonymous
Password: neuron
ftp> cd pub/neuroprose
ftp> binary
ftp> get plaut.thesis-summary.ps.Z (or plaut.dyslexia.ps.Z)
ftp> quit
unix> zcat plaut.thesis-summary.ps.Z | lpr
David Plaut dcp+@cs.cmu.edu
Department of Psychology 412/268-5145
Carnegie Mellon University
Pittsburgh, PA 15213-3890
------------------------------
From: takefuji@axon.EEAP.CWRU.Edu (Yoshiyasu Takefuji)
Date: Sat, 23 Nov 91 17:39:23 -0500
Subject: New book - Neural Network Parallel Computing
[[ Editor's Note: The opening blurb has typical marketing description.
Judge the book, not the advert. -PM ]]
NEW FROM KLUWER ACADEMIC PUBLISHERS
Neural Network Parallel Computing
By Yoshiyasu Takefuji, Case Western Reserve University, USA
1992 ISBN 0-7923-9190-X Cloth 240 pages $65.00
Neural Network Parallel Computing is the first book and the only single
book available for the professional on neural network computing for
optimization problems. This introductory book is for experts in a variety
of areas including parallel computing, neural network computing, computer
science, communications, graph theory, computer aided design for VLSI
circuits, molecular biology, management science, and operations research,
as well as for the novice in these areas.
Neural Network Parallel Computing provides real applications and
real-world examples. The computational power of neural network computing
is demonstrated by solving numerous problems such as N-queen, crossbar
switch scheduling, four-coloring and k-colorability, graph planarization
and channel routing, RNA secondary structure prediction, knight's tour,
spare allocation, sorting and searching, and tiling.
Neural Network Parallel Computing presents a major breakthrough in
science as well as a variety of engineering fields. This book is an
excellent reference for researchers in all areas covered in this book.
This text may also be used as a senior or graduate level course on the
topic.
Contents
Preface
Acknowledgements
Chapter 1 Neural network models and N-queen problems
1.1 Introduction
1.2 Mathematical neural network models
1.3 N-queen neural network
1.4 General optimization programs
1.5 N-queen simulation programs
1.6 References
1.7 Exercises
Chapter 2 Crossbar switch scheduling problems
2.1 Introduction
2.2 Crossbar scheduling problems and N-queen problems
2.3 References
2.4 Exercises
Chapter 3 Four-coloring map problems and k-colorability problems
3.1 Introduction
3.2 Four-coloring neural network
3.3 K-colorability neural network
3.4 References
3.5 Exercises
Chapter 4 Graph planarization problems
4.1 Introduction
4.2 Neural representation and motion equations
4.3 References
4.4 Exercises
Chapter 5 Channel routing problems
5.1 Introduction
5.2 Graph planarization and channel routing
5.3 References
5.4 Exercises
Chapter 6 RNA secondary structure prediction
6.1 Introduction
6.2 Maximum independent set problems
6.3 Predicting the secondary structure in ribonucleic acids
6.4 Graph planarization and RNA secondary structure prediction
6.5 References
6.6 Exercises
Chapter 7 Knight's tour problems
7.1 Introduction
7.2 Neural representation and motion equations
7.3 References
7.4 Exercises
Chapter 8 Spare Allocation problems
8.1 Introduction
8.2 Neural representation and motion equations
8.3 References
8.4 Exercises
Chapter 9 Sorting and Searching
9.1 Introduction
9.2 Sorting
9.3 Searching
9.4 References
Chapter 10 Tiling problems
10.1 Introduction
10.2 Neural representation and motion equations
10.3 References
10.4 Exercises
Chapter 11 Silicon neural networks
11.1 Introduction
11.2 Analog implementations
11.3 Digital implementations
11.4 References
11.5 Exercises
Chapter 12 Mathematical background of the artificial neural network
12.1 Introduction and four neuron models
12.2 Why is the decay term harmful?
12.3 Basic analog convergence theorem and proof
12.4 Discrete sigmoid neural network convergence theorem and proof
12.5 McCulloch-Pitts neural network convergence theorem and proof
12.6 Hysteresis McCulloch-Pitts neural network convergence theorem
and proof
12.7 Maximum neural network convergence theorem and proof
12.8 Other neuron models
12.9 References
Chapter 13 Forthcoming applications
13.1 Introduction
13.2 Time slot assignment in TDM hiearchical switching system
13.3 Broadcast scheduling in packet radio networks
13.4 Module orientation problems
13.5 Maximum clique problems
Chapter 14 Conjunctoids and artificial learning
14.1 Introduction
14.2 Multinomial conjunctoid concepts
14.3 Multnomial conjunctoid circuitry
14.4 References
Subject index
To order the book by mail
Kluwer Academic Publishers
Order Department
P.O. Box 358 Accord Station
Hingham, MA 02018-0358
Credit Card Customers call: (617)871-6600
Fast Convenient Service: Fax (617)871-6528 or
Email:kluwer@world.std.com
The book will be available from January 1992.
------------------------------
Subject: TR - Training algorithms for limited precision nets
From: Xie Yun <xie@ee.su.OZ.AU>
Date: Tue, 03 Dec 91 11:07:21 +1100
The following technical report is placed in the neuroprose archive:
\title{Training Algorithms for Limited Precision Feedforward Neural Networks }
\author{Yun Xie \thanks{Permanent address:
Department of Electronic Engineering,
Tsinghua University,
Beijing 100084, P.R.China}
\hskip 30pt Marwan A. Jabri\\
\\
Department of Electrical Engineering\\
The University of Sydney\\
N.S.W. 2006, Australia}
\date{}
\maketitle
\begin{abstract}
A statistical quantization model is used to analyze the effects of
quantization on the performance and the training dynamics of a
feedforward multi-layer neural network implemented in digital hardware.
The analysis shows that special techniques have to be employed to train
such networks in which each variable is represented by limited number of
bits in fixed point format. Based on the analysis, we propose a training
algorithm that we call the Combined Search Algorithm (CS). It consists of
two search techniques and can be easily implemented in hardware.
Computer simulations were conducted using IntraCardiac ElectroGrams
(ICEGs) and sonar reflection patterns and the results show that: using
CS, the training performance of feedforward multi--layer neural networks
implemented in digital hardware with 8 to 10 bit resolution can be as
good as that of networks implemented with unlimited precision; CS is
insensitive to training parameter variation; and importantly, the
simulations confirm that the numbers of quantization bits can be reduced
in the upper layers without affecting the performance of the network.
\end{abstract}
You can get the report by FTP:
unix>ftp 128.146.8.52
name:anonymous
Passord:neuron
ftp>binary
ftp>cd pbu/neuroprose
ftp>get yun.cs.ps.Z
ftp>bye
unix>uncompress yun.cs.ps.Z
unix>lpr yun.cs.ps
Yun
------------------------------
Subject: PREPRINT - Currency exchange prediction via neural nets
From: P.Refenes@cs.ucl.ac.uk
Date: Tue, 10 Dec 91 17:52:59 +0000
CURRENCY EXCHANGE RATE PREDICTION & NEURAL NETWORK DESIGN STRATEGIES
A. N. REFENES, M. AZEMA-BARAC, L. CHEN, & S. A. KAROUSSOS
Department of Computer Science,
University College London,
Gower Street, WC1, 6BT,
London, UK.
ABSTRACT
This paper describes a non trivial application in
forecasting currency exchange rates, and its implementation
using a multi-layer perceptron network. We show that with
careful network design, the backpropagation learning
procedure is an effective way of training neural networks
for time series prediction. The choice of squashing
function is an important design issue in achieving fast
convergence and good generalisation performance. We
evaluate the use of symmetric and asymmetric squashing
functions in the learning procedure, and show that
symmetric functions yield faster convergence and better
generalisation performance. We derive analytic results to
show the conditions under which symmetric squashing
functions yield faster convergence, and to quantify the
upper bounds on the convergence improvement. The network
is evaluated both for long term forecasting without feed-
back (i.e. only the forecast prices are used for the
remaining trading days) and for short term forecasting with
hourly feed-back. The network learns the training set near
perfect and shows accurate prediction, making at least 22%
profit on the last 60 trading days of 1989.
------------------------------
Subject: TR Top-down teaching enables non-trivial clustering
From: desa@cs.rochester.edu
Date: Wed, 11 Dec 91 17:25:03 -0500
The following technical report has been placed in the neuroprose archive:
Top-down teaching enables non-trivial clustering
via competitive learning
Virginia de Sa Dana Ballard
desa@cs.rochester.edu dana@cs.rochester.edu
Dept. of Computer Science
University of Rochester
Rochester, NY 14627-0226
Abstract:
Unsupervised competitive learning classifies patterns based on
similarity of their input representations. As it is not given external
guidance, it has no means of incorporating task-specific information
useful for classifying based on semantic similarity. This report
describes a method of augmenting the basic competitive learning algorithm
with a top-down teaching signal. This teaching signal removes the
restriction inherent in unsupervised learning and allows high level
structuring of the representation while maintaining the speed and
biological plausibility of a local Hebbian style learning algorithm.
Examples, using this algorithm in small problems, are presented and the
function of the teaching input is illustrated geometrically. This work
supports the hypothesis that cortical back-projections are important for
the organization of sensory traces during learning.
To retrieve by anonymous ftp:
unix> ftp cheops.cis.ohio-state.edu
Name (cheops.cis.ohio-state.edu:): anonymous
Password (cheops.cis.ohio-state.edu:anonymous): <ret>
ftp> cd pub/neuroprose
ftp> binary
ftp> get desa.top_down.ps.Z
ftp> quit
unix> uncompress desa.top_down.ps
unix> lpr -P(your_local_postscript_printer) desa.top_down.ps
Hard copy requests can be sent to
tr@cs.rochester.edu
or
Technical Reports
Dept. of Computer Science
University of Rochester
Rochester, NY 14627-0226
(There is a nominal $2 charge for hard copy TR's)
------------------------------
Subject: International Comparison of Learning Algorithms: MONK
From: Sebastian.Thrun@B.GP.CS.CMU.EDU
Date: Mon, 16 Dec 91 13:27:00 -0500
Dear Connectionists:
The technical report "The MONK's Problems - A Performance Comparison
of Different Learning Algorithms" is now available via anonymous ftp.
Copies of the report as well as the MONK's database can be obtained in
the following way:
unix> ftp archive.cis.ohio-state.edu
Name: anonymous
Password: <your user id>
ftp> cd pub/neuroprose
ftp> binary
ftp> get thrun.comparison.ps.Z (=report)
ftp> get thrun.comparison.dat.Z (=data)
ftp> quit
unix> uncompress thrun.comparison.ps.Z
unix> uncompress thrun.comparison.dat.Z
unix> lpr thrun.comparison.ps
unix> lpr thrun.comparison.dat
If this does not work, send e-mail to reports@cs.cmu.edu
asking for the Technical Report CMU-CS-91-197.
Sebastian Thrun
thrun@cs.cmu.edu
SCS, CMU, Pittsburgh PA 15213
Some things changed - here is the abstract and the table of contents again:
The MONK's Problems
A Performance Comparison of Different Learning Algorithms
S. Thrun, J. Bala, E. Bloedorn, I. Bratko, B. Cestnik, J. Cheng, K. De Jong,
S. Dzeroski, S.E. Fahlman, D. Fisher, R. Hamann, K. Kaufman, S. Keller,
I. Kononenko, J. Kreuziger, R.S. Michalski, T. Mitchell, P. Pachowicz,
Y. Reich, H. Vafaie, W. Van de Welde, W. Wenzel, J. Wnek, and J. Zhang
CMU-CS-91-197
This report summarizes a comparison of different learning techniques
which was performed at the 2nd European Summer School on Machine
Learning, held in Belgium during summer 1991. A variety of symbolic and
non-symbolic learning techniques - namely AQ17-DCI, AQ17-HCI, AQ17-FCLS,
AQ14-NT, AQ15-GA, Assistant Professional, mFOIL, ID5R, IDL, ID5R-hat,
TDIDT, ID3, AQR, CN2, CLASSWEB, PRISM, Backpropagation, and Cascade
Correlation - are compared on three classification problems, the MONK's
problems.
The MONK's problems are derived from a domain in which each training
example is represented by six discrete-valued attributes. Each problem
involves learning a binary function defined over this domain, from a
sample of training examples of this function. Experiments were performed
with and without noise in the training examples.
One significant characteristic of this comparison is that it was
performed by a collection of researchers, each of whom was an advocate of
the technique they tested (often they were the creators of the various
methods). In this sense, the results are less biased than in comparisons
performed by a single person advocating a specific learning method, and
more accurately reflect the generalization behavior of the learning
techniques as applied by knowledgeable users.
=----------------------------------------------------------------------
================================
RESULTS - A SHORT OVERVIEW
================================
MONK-1 MONK-2 MONK-3(noisy)
AQ17-DCI 100.0% 100.0% 94.2%
AQ17-HCI 100.0% 93.1% 100.0%
AQ17-FCLS 92.6% 97.2%
AQ14-NT 100.0%
AQ15-GA 100.0% 86.8% 100.0%
(by J. Bala, E. Bloedorn, K. De Jong, K. Kaufman,
R.S. Michalski, P. Pachowicz, H. Vafaie,
J. Wnek, and J. Zhang)
Assistant Professional 100.0% 81.25% 100.0%
(by B. Cestnik, I. Kononenko, and I. Bratko)
mFOIL 100.0% 69.2% 100.0%
(by S. Dzeroski)
ID5R 81.7% 61.8%
IDL 97.2% 66.2%
ID5R-hat 90.3% 65.7%
TDIDT 75.7% 66.7%
(by W. Van de Velde)
ID3 98.6% 67.9% 94.4%
ID3, no windowing 83.2% 69.1% 95.6%
ID5R 79.7% 69.2% 95.2%
AQR 95.9% 79.7% 87.0%
CN2 100.0% 69.0% 89.1%
CLASSWEB 0.10 71.8% 64.8% 80.8%
CLASSWEB 0.15 65.7% 61.6% 85.4%
CLASSWEB 0.20 63.0% 57.2% 75.2%
(by J. Kreuziger, R. Hamann, and W. Wenzel)
PRISM 86.3% 72.7% 90.3%
(by S. Keller)
ECOBWEB leaf pred. 71.8% 67.4% 68.2%
" plus inform.utility 82.7% 71.3% 68.0%
(by Y. Reich and D. Fisher)
Backpropagation 100.0% 100.0% 93.1%
BP + weight decay 100.0% 100.0% 97.2%
(by S. Thrun)
Cascade Correlation 100.0% 100.0% 97.2%
(by S.E. Fahlman)
=----------------------------------------------------------------------
=----------------------------------------------------------------------
1 The MONK's Comparison Of Learning Algorithms --
Introduction and Survey
S.B. Thrun, T. Mitchell, and J. Cheng 1
1.1 The problem 2
1.2 Visualization 2
2 Applying Various AQ Programs to the MONK's Problems: Results
and Brief Description of the Methods
J. Bala, E. Bloedorn, K. De Jong, K. Kaufman,
R.S. Michalski, P. Pachowicz, H. Vafaie, J. Wnek,
and J. Zhang 7
2.1 Introduction 8
2.2 Results for the 1st problem (M1) 9
2.2.1 Rules obtained by AQ17-DCI 9
2.2.2 Rules obtained by AQ17-HCI 10
2.3 Results for the 2nd problem (M2) 11
2.3.1 Rules obtained by AQ17-DCI 11
2.3.2 Rules obtained by AQ17-HCI 11
2.3.3 Rules obtained by AQ17-FCLS 13
2.4 Results for the 3rd problem (M3) 15
2.4.1 Rules obtained by AQ17-HCI 15
2.4.2 Rules obtained by AQ14-NT 16
2.4.3 Rules obtained by AQ17-FCLS 16
2.4.4 Rules obtained by AQ15-GA 17
2.5 A Brief Description of the Programs and Algorithms 17
2.5.1 AQ17-DCI (Data-driven constructive induction) 17
2.5.2 AQ17-FCLS (Flexible concept learning) 18
2.5.3 AQ17-HCI (Hypothesis-driven constructive induction) 18
2.5.4 AQ14-NT (noise-tolerant learning from engineering data) 19
2.5.5 AQ15-GA (AQ15 with attribute selection by a genetic algorithm) 20
2.5.6 The AQ Algorithm that underlies the programs 20
3 The Assistant Professional Inductive Learning System:
MONK's Problems
B. Cestnik, I. Kononenko, and I. Bratko 23
3.1 Introduction 24
3.2 Experimental results 24
3.3 Discussion 25
3.4 Literature 25
3.5 Resulting Decision Trees 26
4 mFOIL on the MONK's Problems
S. Dzeroski 29
4.1 Description 30
4.2 Set 1 31
4.3 Set 2 31
4.4 Set 3 32
5 Comparison of Decision Tree-Based Learning Algorithms on
the MONK's Problems
W. Van de Welde 33
5.1 IDL: A Brief Introduction 34
5.1.1 Introduction 34
5.1.2 Related Work 35
5.1.3 Conclusion 36
5.2 Experimental Results 40
5.2.1 ID5R on test set 1 43
5.2.2 IDL on test set 1 43
5.2.3 ID5R-HAT on test set 1 44
5.2.4 TDIDT on test set 1 44
5.2.5 ID5R on test set 2 45
5.2.6 IDL on test set 2 46
5.2.7 TDIDT on test set 2 48
5.2.8 TDIDT on test set 1 49
5.2.9 ID5R-HAT on test set 2 50
5.3 Classification diagrams 52
5.4 Learning curves 56
6 Comparison of Inductive Learning Programs
J. Kreuziger, R. Hamann, and W. Wenzel 59
6.1 Introduction 60
6.2 Short description of the algorithms 60
6.2.1 ID3 60
6.2.2 ID5R 61
6.2.3 AQR 61
6.2.4 CN2 62
6.2.5 CLASSWEB 62
6.3 Results 63
6.3.1 Training Time 63
6.3.2 Classifier Results 64
6.4 Conclusion 68
6.5 Classification diagrams 69
7 Documentation of Prism -- an Inductive Learning Algorithm
S. Keller 81
7.1 Short Description 82
7.2 Introduction 82
7.3 PRISM: Entropy versus Information Gain 82
7.3.1 Maximizing the information gain 82
7.3.2 Trimming the tree 82
7.4 The Basic Algorithm 83
7.5 The Use of Heuristics 84
7.6 General Considerations and a Comparison with ID3 84
7.7 Implementation 84
7.8 Results on Running PRISM on the MONK's Test Sets 85
7.8.1 Test Set 1 -- Rules 86
7.8.2 Test Set 2 -- Rules 87
7.8.3 Test Set 3 -- Rules 90
7.9 Classification diagrams 92
8 Cobweb and the MONK Problems
Y. Reich, and D. Fisher 95
8.1 Cobweb: A brief overview 96
8.2 Ecobweb 97
8.2.1 Characteristics prediction 97
8.2.2 Hierarchy correction mechanism 97
8.2.3 Information utility function 98
8.3 Results 98
8.4 Summary 100
9 Backpropagation on the MONK's Problems
S.B. Thrun 101
9.1 Introduction 102
9.2 Classification diagrams 103
9.3 Resulting weight matrices 105
10 The Cascade-Correlation Learning Algorithm on the MONK's Problems
S.E. Fahlman 107
10.1 The Cascade-Correlation algorithm 108
10.2 Results 109
10.3 Classification diagrams 112
------------------------------
End of Neuron Digest [Volume 9 Issue 5]
***************************************
