Copy Link
Add to Bookmark
Report
Pure Bollocks Issue 21_008
Wsi helps bring neural computers a step closer
Neural networks have been intensely researched during the past
few years and they are set to provide a new direction for
computing in the 1990s.
At a recent technology exhibition in London, engineers from
Hitachi's central research laboratories showed that by using
wafer scale integration (wsi) for the implementation they have
produced a system they believe is of unprecedented size and
speed.
The system is based on the wsi of digital neural network
circuits with up to 144 neurons on each wafer. The wafers are of
5in diameter with the neurons and their interconnections, or
synapses, being created using gate array methodology and 0.8um
technology. Eight wafers provide up to 1152 neurons.
Neural networks are modelled on the form of the brain. Each
neuron produces an output as a threshold function of the sum of
its weighted inputs. That output is then transmitted to the next
'layer' of neurons, where it is in turn weighted and summed with
other neural outputs.
The pattern of connection between the input layer and the
output layer and the weights assigned to each connection
determine the result produced from a given input. The ability to
alter weights in response to feedback allows a network to be
trained to generate an appropriate answer to a particular
problem.
Neural networks have shown promise in solving problems of
optimisation and recognition. But the inherent parallelism of
neural networks is not taken advantage of when run as software
simulations on conventional serial processing computers. This
results in much slower operations.
In hardware, researchers are pursuing analogue and digital,
and optical or electronic networks. The most general network
architecture is the Hopfield network, in which each neuron -
neurons can be disconnected by setting the synaptic weight to
zero. But where the synapse connections are permanent and
discrete their number rises with the square of the number of
neurons.
Each neuron must be able to store a synaptic weight for
connections from every neuron but is obviously unwieldy. The
solution to this chosen by Hitachi is a time multiplexed digital
bus.
Hitachi's engineers describe the system as a neurocomputer,
and envision it running as an accelerator or coprocessor for a
conventional computer or workstation host. The company hopes to
bring such a system to market within two years.
The engineers responsible for the neurocomputer feel that
their combination of digital neural networks and wsi has, for the
first time, produced a network of useful complexity and speed.
They say that moves to cell based layout and more aggressive
design rules could get them up to 1000 neurons per wafer quickly.
There is also scope to improve the efficiency of gate utilisation
from the currently required 2000 per neuron to 1000 per neuron.
The system is contained in a frame measuring 12 x 8.3 x 9in.
Operating from a 5v supply, each wafer dissipates 5W and the
system as a whole approximately 50W with forced air cooling used
to remove heat.
The neurocomputer is digital in operation with each neuron
producing 1 9bit result. Up to 64 synapse weights of 8 bits are
stored in local sram at each neuron. Each wafer contains 60
gate arrays of 100,000 gate complexity. Of these, 48 are used to
implement three neurons per gate array along with the associated
bus connections and memory. The remaining 12 are used for
buffering bus interconnections.
To provide a Hopfield network architecture, each neural gate
array contains its own global bus which links to two of its
nearest neighbours. This bus joins a main bus block at the
median line of the wafer, and from there it is taken off-wafer to
a backplane which connects all the wafers.
The bus is a 9 bit multiplexed bus allowing any neuron to be
connected with any other neuron in the system. The bus protocol
includes a broadcast facility allowing a neuron to be linked to
all others simultaneously.
This time multiplexed bus means that only one synapse
weighting circuit is required per neuron. Summation is performed
by adding the weighted connections on successive cycles. This is
much slower than a fully and discreetly connected network but
still allows neurons to operate in parallel, providing speed
benefits over the software simulation.
As for learning performance, Hitachi claims 2.3 billion
connection updates per second, and a prototype system has been
used to perform signature verification in 2s.
Neural networks are well suited to wsi as they are inherently
tolerant of individually defective neurons. Such a neuron can be
effectively excised from the network by setting all its
coefficients to zero.
Neural networks are notoriously difficult to control or
program but running one from a conventional host computer helps.
But don't expect a neurocomputer to cost less than $10,000 for a
minimal configuration. If such a configuration has a 1000
artificial neurons, what value does that put on the human brain
with about 10 billion neurons?
