A Learning Program
-
For decades there have been attempts to create computer programs that
can learn like people - Artificial Intelligence.
For example, how do you teach a child to recognize what a chair is? You
show him examples telling him "This is a chair ; That one is not a chair"
until the child learns the concept of what a chair is. In this stage, the
child can look at the examples we have shown him and answer correctly to
the question "Is this object a chair?". Furthermore, if we show to the child
new objects, that he didn't see before, we could expect him to recognize
correctly whether the new object is a chair or not, providing that we've
given him enough positive and negative examples. This is exactly the idea
behind the perceptron.
The Perceptron
-
The perceptron is a program that learn concepts, i.e. it can learn
to respond with True (1) or False (0) for inputs we present to
it, by repeatedly "studying" examples presented to it.
The Perceptron is a single layer neural network whose weights and biases
could be trained to produce a correct target vector when presented with
the corresponding input vector.
The training technique used is called the perceptron learning rule.
The perceptron generated great interest due to its ability to generalize
from its training vectors and work with randomly distributed connections.
Perceptrons are especially suited for simple problems in pattern
classification.
Our perceptron network consists of a single neuron connected to two
inputs through a set of 2 weights, with an additional bias input.
The perceptron calculates its output using the following equation:
P * W + b > 0
where P is the input vector presented to the network, W is the vector
of weights and b is the bias.
The Learning Rule
-
The perceptron is trained to respond to each input vector with a
corresponding target output of either 0 or 1. The learning rule has been
proven to converge on a solution in finite time if a solution exists.
The learning rule can be summarized in the following two equations:
For all i:
W(i) = W(i) + [ T - A ] * P(i)
b = b + [ T - A ]
where W is the vector of weights, P is the input vector presented to the
network, T is the correct result that the neuron should have shown, A
is the actual output of the neuron, and b is the bias.
Training
-
Vectors from a training set are presented to the network one after another.
If the network's output is correct, no change is made. Otherwise, the
weights and biases are updated using the perceptron learning rule. An entire
pass through all of the input training vectors is called an epoch. When
such an entire pass of the training set has occured without error, training
is complete. At this time any input training vector may be presented to
the network and it will respond with the correct output vector. If a vector
P not in the training set is presented to the network, the network will tend
to exhibit generalization by responding with an output similar to target
vectors for input vectors close to the previously unseen input vector P.
Limitations
-
Perceptron networks have several limitations. First, the output values of
a perceptron can take on only one of two values (True or False). Second,
perceptrons can only classify linearly separable sets of vectors. If a
straight line or plane can be drawn to seperate the input vectors into
their correct categories, the input vectors are linearly separable and the
perceptron will find the solution. If the vectors are not linearly separable
learning will never reach a point where all vectors are classified properly.
The most famous example of the perceptron's inability to solve problems
with linearly nonseparable vectors is the boolean exclusive-or problem.
Our Implementation
-
We implemented a single neuron perceptron with 2 inputs. The input for
the neuron can be taken from a graphic user interface, by clicking on
points in a board. A click with the left mouse button generates a '+'
sign on the board, marking that it's a point where the perceptron should
respond with 'True'. A click with the right mouse button generates a '-'
sign on the board, marking that it's a point where the perceptron should
respond with 'False'. When enough points have been entered, the user
can click on 'Start', which will introduce these points as inputs to the
perceptron, have it learn these input vectors and show a line which
corresponds to the linear division of the plane into regions of opposite
neuron response.
Here's an example of a screen shot:
Examples
-
Here are more examples of screen shots. The first two show succesful runs
of the perceptrons, the third is an example of how the perceptron fails
to classify a set of non linearly separable vectors.
Source Code
-
Here is the source code. In order to be able to run the program,
save all these files in a directory, type 'make', and use 'prcpt' to
start the system. The program should run under all platforms which
support Tk, though we tested it for UNIX SunOS only.
- prcpt - The Tk interface (8K).
- perceptron.c -
The actual Neural Network, C code (7K).
- readme - The readme file (very much like this file) (6K).
- makefile - The (trivial) makefile (<1K).
You can also grab all these files in one
tarred gzipped file (6K) .
Note: This code is free and is in the public domain. You may copy, use,
and change it as much as you like. However, the writers take no
responsibility for the results of using this program.
See Also
- NNUGA -
Neural Network Using Genetic Algorithms
