VLSI FOR NEURAL NETWORKS AND THEIR APPLICATIONS

ABSTRACT : Most of the students of Electronics Engineering are exposed to Integrated Circuits (IC's) at a very basic level, involving SSI (small scale integration) circuits like logic gates or MSI (medium like out scale integration) parity circuits world multiplexers, there neural nets can be used to solve. Suppose that you are given 500 characters of code that you know to be C, C++, Java, or Python. Now, construct a program that identifies the code's language. One solution is to construct a neural net that learns to identify these languages. According to a simplified account, the human brain consists of about ten billion neurons -- and a neuron is, on average, connected to several thousand other neurons. By way of these connections, neurons both Neural networks are a new good very method at difficult of programming pattern using computers. They are exceptionally performing to program recognition and other tasks that are conventional techniques. Programs send of and receive varying very quantities energy. One

encoders etc. But there is a lot bigger involving miniaturization at levels so great, that a micrometer and a microsecond are literally considered huge! This is the world of VLSI - Very Large Scale Integration.

important feature of neurons is that they don't react immediately to the reception of energy. Instead, they sum their received energies, and they send their own quantities of energy to other neurons only when this sum has reached a certain critical threshold. The brain learns by adjusting the number and strength of these connections. The brain's network of neurons forms a massively parallel

that employ neural nets are also capable of learning on their own and adapting to changing conditions. Neural nets may be the future of computing .A good way to understand them is with a puzzle that

information processing system. This contrasts with conventional computers, in which a single processor executes a single series of instructions.

INTRODUCTION: VLSI stands for "Very Large Scale Integration". This is the field which involves packing more and more logic devices into smaller and smaller areas .Thanks to VLSI, circuits that would have taken broadfulls of space can now be put into a small space few millimeters across! This has opened up a big opportunity to do things that were not possible before. VLSI circuits are everywhere ... your computer, your car, your brand new state-of-the-art digital camera, the cell -phone, etc. MOST OF TODAYS VLSI

DESIGNS ARE CLASSIFIED INTO THREE CATEOGRIES: 1. chip The VLSI is also used for Neural Networks in many ways and applications. A neural network is a powerful data modeling tool that is able to capture and represent complex input/output relationships. The motivation for the development of neural network technology stemmed Analog 2.Application Specific on a

Integrated

Circuits 3.Systems

from the desire to develop an artificial system that could perform "intelligent" tasks similar to those performed by the human brain. The true power and advantage of neural networks lies in their ability to represent both linear and non-linear relationships and in their ability to learn these relationships directly from the data being modeled. Traditional linear models are simply inadequate when it comes to modeling data that contains non-linear characteristics. The most common neural network model is the multilayer perceptron (MLP). This type of neural network is known as a supervised network because it requires a desired output in order to learn. The goal of this type of network is to create a model that correctly maps the input to the output using historical data so that the model can then be used to produce the output when the desired output is unknown.

ARTIFICIAL NEURAL NETWORK: An artificial neural network (ANN) or commonly as Neural Network (NN) is an interconnected group of artificial neurons that uses a mathematical or computational model for information processing based on a connectionist approach to computation. In most cases an ANN is an adaptive system that changes its structure internal In more based on external that terms or information practical flows neural

through the network. networks are non-linear statistical data modeling tools. They can be used to model complex relationships between inputs and outputs or to find patterns in data.

system. In the field of neuroscience, it most often refers to a group of neurons from nervous systems that are suited for laboratory analysis. Biological Neural Network From "Texture of the Nervous System of Man and the Vertebrates". The figure illustrates the diversity of neuronal morphologies in the auditory cortex. In neuroscience, a neural network from is a bit of a of conceptual rigorously observed juggernaut: the conceptual transition neuroanatomy, discipline descriptive

structure, to the designation of the parameters delimiting a 'network' can be problematic. In outline a neural network describes a population of physically interconnected neurons or a group of disparate neurons whose inputs or signaling targets define a recognizable circuit. Communication Biological Neural Network between neurons often involves an electrochemical process. The interface through which they interact with A biological neural network is a plexus of connected or functionally related neurons in the peripheral nervous system or the central nervous surrounding neurons usually consists of several dendrites (input connections), which are connected via synapses to other neurons, and one

axon (output connection). If the sum of the input signals surpasses a certain threshold, the neuron sends an action potential (AP) at the axon hillock and transmits this electrical signal along the axon. In contrast, a neuronal circuit is a functional entity of interconnected neurons that influence each other (similar to a control loop in cybernetics). The neural network is divided into three different categories

the multiplication, and analog inputs must first be converted to digital. Multi-processor Chip: A far more elaborate approach is to put many small processors on a chip. Two architectures dominate such designs: single instruction with multiple data (SIMD) and systolic arrays. For SIMD design, each processor executes the same instruction in parallel but on different data. In systolic arrays, a processor does one step of a calculation (always the same step) before passing it's result on to the next processor in a pipelined manner. SIMD chips include the Inova N64000 and the HNC 100 NAP. All chips execute the same instruction and common control and data bases allow for multiple chips to be combined. Radial Basis Functions: RBF networks provide fast learning and straight-forward interpretation. The comparison of input vectors to stored training vectors can be calculated easily without using multiplication operations. Two commercial RBF products are now available: the IBM ZISC036 Computer) (Zero chip Instruction and the Set Nestor

DIGITAL: The digital neural network category categories architectures, encompasses SIMD many and subslice systolic including

array devices, and RBF architectures. For the designer, digital technology has the advantages of mature fabrication techniques, weight storage in RAM, and arithmetic operations exact within the number of bits of the operands and accumulators. digital digital slower From the users applications. operations are viewpoint, However, usually

chips are easily embedded into most

than in analog systems, especially in 5

Ni1000 chip. The ZISC036 contains 36

prototype-vector neurons, where the vectors have 64 8-bit elements, and can be assigned to categories from 1 to 16383. Multiple chips can be easily cascaded Region algorithm to of provide Influence using signum additional learning basis prototypes. The chip implements a

multi-layer networks, and loaded with small programs to direct the processing. Feed-forward processing reaches 40MCPS. At the other end of the complexity scale are the Hitachi Wafer Scale Integration chips. Both Hopfield wafers and have back-propagation been built. A

functions with radii of 0 to 16383. Recall processing takes for a 250k/sec pattern presentation rate. The Nestor Ni1000, developed jointly by Intel chip and has Nestor, two contains 1024 prototypes of 256 5-bit elements. The on-chip learning algorithms, RCE[21] and PNN[22], and other algorithms can be micro coded. The processing rate is about 40k patterns/sec with a 40MHz clock. Other Digital Designs: Some digital neural network chips don't quite fit into the above three sub-categories. Examples include the Micro Circuit Engineering MT19003 NISP Neural Instruction Set Processor [23] and the Hitachi Wafer Scale Integration chips [24]. The NISP is basically optimized a for very simple RISC of processor with seven instructions, implementation

neurocomputer with 8 of the backprop wafers, each with 144 neurons, achieved 2.3GCUPS [6]. ANALOG: Analog hardware networks can exploit physical properties to do network operations and thereby obtain high speed and densities. A common output line, for example, can sum current outputs from synapses to sum the neuron inputs. However, analog design can be very difficult because of the need to compensate for variations in manufacturing, in temperature, etc. Creating an analog synapse involves the complications of analog weight storage and the need for a multiplier linear over a wide range. While many designs use analog techniques to carry out conventional architectures like multi-layer feed forward networks, neuromorphic

designs, such as the Synaptic Silicon Retina, emulate biological functions as closely as possible. The first analog commercial 80170NW chip was the Intel ETANN (Electrically

the Bell core CLNN-32 chip has 5-bit weights loaded of digitally the but the with processing analog. The Neuro Classifier from the Mesa Research Institute at the University of Twente digital has 70 analog inputs, 6 weights. The feed-forward The final hidden and 1 analog output with 5-bit processing rate is an astounding 20ns, representing 20GCPS. output is without a squashing function so that multiple chips can be added to increase the number of hidden units. The use of pulse rates or pulse widths is another method to emulate nets in hardware. The first was commercial the Neural implementation network

Boltzmann style annealing is done in

Trainable Analog Neural Network) that contains 64 neurons and 10280 weights. The non-volatile weights are stored as charge on floating gates and a Gilbert multiplier provides 4quadrant multiplication. A flexible design, including internal feedback and division of the weights into two 64x80 banks (including 16 biases), allows for multiple configurations including 3-layers of 64 neurons/layer, and 2-layers with 128 inputs and 64 neurons. HYBRID: Hybrid designs attempt to combine the best of analog and digital techniques. Typically, the external inputs/outputs are digital to facilitate integration into digital systems, while internally some or all of the processing is analog. The AT&TANNA Artificial Neural Network ALU, for example, is externally digital but uses capacitor charge, periodically refreshed by DAC's, to store the weights. Similarly,

Semiconductor chip set with the SU3232 synapse unit and the NU32 neuron unit. The Ricoh Company has reported a pulse chip with a special back-propagation implemented on-chip. algorithm TheRN-

100contained only a single neuron with 8 inputs and8outputs.An array of 12 RN-100's learned to balance a 2-D pendulum in just30s.

APPLICATIONS

OF

NEURAL

Predictability of future indicator values based on past data and trend recognition.

NETWORKS IN MEDICINE: Network (NN) in medicine has attracted many researchers. A simple search by Machado (1996) in Medline for articles about computerbased NN between 1982 and 1994 resulted with more than 600 citations. Several applications was reviewed and evaluated based on the model used, input and output data, the results and project status. From the review, several research and applications of Neural Expert System in medical applications have been listed. Most of the research that employed NN yields between 70% to 80% accuracy. NN has been shown as a powerful tool to enhance current medical diagnostic techniques. Partridge et al. (1996) listed several potentials of NN over conventional computation and manual analysis in medical application: Implementation using data instead of possibly ill defined rules. Noise and novel situations are handled automatically via data generalization.

Automated real-time analysis and diagnosis. Enables rapid identification and classification of input data. Eliminates error associated with human fatigue and habituation. In this paper the discussion of applications of neural network in medical applications is divided into several domain that are applications in basic sciences, clinical medicine, signal processing and interpretation and medical image processing. Applications in Basic sciences: In basic sciences, NN helps clinician to investigate the impact of parameter after certain conditions or treatments. It supplies clinicians with information domain. Learning the time course of blood glucose .For example can help clinician to control the diabetes mellitus. Uses feed forward NN for predicting the time course of blood glucose levels from the complex about the risk or incoming circumstances regarding the

interaction

of

glucose

counter

Applications in Signal Processing and Interpretation: Signal processing and interpretation in medicine involve a complex analysis of signals, graphic representations, classification. experienced In of and Consequently, surgeon pattern even could (ECG) of could acute be

regulatory hormones and insulin. Multi-Layer Perceptron (MLP) with sigmoidal Feed-Forward and standard Back-Propagation algorithm forecaster was for (BP) employed learning as a

bacteria-antibiotic

interactions of infectious diseases. They conclude that the 1-month forecaster produces output correct to within occurrences of sensitivity. However, predictions for the 2-month and 3-month are less accurate. Applications in Clinical Medicine: Patient who hospitalize for having high-risk diseases required special monitoring as the disease might spread in no time. NN has been used as a tool for patient diagnosis and prognosis to determine patients survival. Bottaci and Drew (1997) investigate fully connected feed forward MLP and BP learning rule, were able to predict patients with colorectal cancer more accurately than clinico pathological methods. They indicate that NN predict the patients survival and death very well compared to the surgeons.

misinterpret or overlooked the data. electrocardiography the ECG readings by (Janet, monitoring and timing analysis for example, the complexity myocardial misjudged cardiologist ECG patient infarction even

experienced 1997). is the

Accordingly the difficulty faced in variability in Morphology patients and across of within patients,

normal and ventricular beats. (Lagerholm et al., 2000) employed Self-Organizing conjunction Neural Networks Basis (Self-Organizing Maps or SOMs) in with Hermite function for the purpose of beat clustering to identify and classify ECG complexes in arrhythmia. SOMs topological structure is a benefit in interpreting the data.

The

experimental

compared to conventional systems even with adjusted thresholds. Applications Processing: Image processing is one of the important applications in medicine as most of decision-making is made by looking at the images. In general the segmentation of medical images is to find regions, which represent single anatomical structures. Poli and Valli employed Hopfield neural network for optimum segmentation of 2-D and 3-D medical images. The networks have been tested on synthetic images and on real tomographic and X-ray images. Uses of two self-organizing maps (SOM) in two stages, self-organizing principal components analysis (SOPCA) and self-organizing feature map (SOFM) for automatic volume segmentation of medical images. They performed a statistical comparison of the performance of the SOFM with Hopfield network and ISODATA algorithm. The results indicate that the accuracy of SOFM is superior compare to both networks. In addition, SOFM was claimed to have in Medical Image

results were claimed to outperform other supervised learning method that uses the same data. Analysis of NN as ECG analyzer also proves that NN is capable to deal with ambiguous nature of ECG signal. Sloop and Marchesi use static and recurrent neural network (RNN) architectures for the classification tasks in ECG analysis for arrhythmia, myocardial ischemia and chronic alterations. Feed forward network with 8-24-14-1 architecture was employed as a classifier for ECG patient monitoring (Waltrous and Towell, 1995). The analysis indicated that the performance of the patient-adapted network was improved due to the ability of the modulated classifier to adjust the boundaries between classes, even though the distributions of beats were different for different patients. Multi layer RNN performance with 15-3-2 architecture had been studied and the performance with of NN is compared conventional

algorithms for recognizing fetal heart rate abnormality (Lee et al., 1999). The study reveals that the performance of NN is exceptional

10

advantage of ease implementation and guaranteed convergence. A neural network is an interconnected group of nodes, akin to the vast network of neurons in the human brain The brain, neural networks and computers: While historically the brain has been viewed as a type of computer, and vice-versa, this is true only in the loosest sense. Computers do not provide us with accurate hardware for describing the brain (even though it is possible to describe a logical process as a computer program or to simulate a brain using a computer) as they do not posses the parallel processing architectures that have been described in the brain. Even when speaking of multiprocessor computers, the functions are not nearly as distributed as in the brain. Neural networks, as used in artificial intelligence, have traditionally been viewed as simplified models of neural processing in the brain, between even this though model the and relation brain

debated. To answer this question, Marr has proposed various levels of analysis which provide us with a plausible answer for the role of neural networks in the understanding of human cognitive functioning. The question of what is the degree of complexity and the properties that individual neural elements should have in order to reproduce something resembling animal intelligence is a subject of current research in theoretical neuroscience. Historically computers evolved from Von Neumann architecture, based on sequential processing and execution of explicit instructions. On the other hand origins of neural networks are based on efforts to model information systems, processing which in are biological

primarily based on parallel processing as well as implicit instructions based on recognition of patterns of 'sensory' input from external sources. In other words, rather sequential processing and execution, at their very heart, neural networks are complex statistic processors. DISADVANTAGES:

biological architecture is very much

11

VLSI has been around for a long time, there is nothing new about it ... but as a side effect of advances in the world of computers, there has been a dramatic proliferation of tools that can be used to design VLSI circuits. The terms computation power, utilization of available area, yield. The combined effect of these two advances is that people can now put diverse functionality into the IC's, opening up new frontiers. Examples are embedded systems, where intelligent devices are put inside everyday objects. CONCLUSIONS: Progress in the fabrication of IC's has enabled us to create fast and powerful circuits in smaller and smaller devices. This also means that we can pack a lot more of functionality into the same area. All modern digital designs start with a designer writing a hardware description of the IC. NN have been successfully implemented in many applications including medicine. NN, which simulates the function of human

biological neuron, has potential of ease implementation domain. in The many main applications

consideration of NN implementation is the input data. Once the network is train, the knowledge could be applied to all cases including the new cases in the domain. Studies have shown that NN predictive capability is a useful capability in medical application. Such capability could be used to predict patient condition based on the history cases. The prediction could help doctor to plan for a better medication and provide the patient with early diagnosis.

REFERENCES: 1) Waserman - Neural computing. 2) Wayne Wolf Modern VLSI Design

12