Introduction to Artificial Inteligence

Assignment 5

Programming assignment - learning with artificial neural networks 


In this assignment, you will implement neural networks and training neural
networks with a random-restart backpropagation algorithm.
The program will include two modules:
the neural network itself, and the training module, that modifies the
network weights so as to achieve optimal peformance on a given function.



The neural network structure is feed-forward,  specified by the number of
units per layer. Assume that the output of each unit at level k is connected
to the input of all units at level k+1. Specification will be by an ASCII
file, where the first line will contain just the number of layers, and
following lines will contain (one per line) the number of units at layer 1
(input), then layer 2, etc. For example, the following specifies
a network with 3 input units, 1 intermediate (hidden) unit, and 2 output
units.

3
3
1
2





The TASK for a neural network is an ASCII file containing a TRUTH TABLE
for some BINARY VALUED function. Each line in the file specifies the outputs
for each combination of input values. For example, a TASK file for the above
network might be:

F F F   F T
F F T   T T
F T F   F F
F T T   T F
T F F   T T
T F T   T T
T T F   T F
T T T   T F





The program that trains the network uses 10 random initial weight assignments,
and for each initial weight assignment runs backpropagation training,
where the stopping criterion is: near - 0 error
over the entire set of examples,
or 5 introductions of the entire example set with no decrease in the
number of errors. Note that if the former occurs, there is no
need for further re-starts! Use a training parameter alpha = 0.3, and initial
weights (and thresholds) uniformly generated in the range [0,1]. Use
sigmoid as the activation function (1/(1+exp(-x))). In computing
score for a network, a total (sum of absolute values)
error less than 0.1 can be considered 0 (it will never be really
0 for finite weights!



Deliverables:

    

  1.  Your program.

  2.  Report on a set of experiments - number of errors as a function of
number of training sessions, on a non-trivial learning example.





Deadline: February 3, 2002.


Note: this is a optional, bonus assignment - do not do it unless
desperate for a minor grade improvement and have spare time.
Worth 4 bonus points in the final grade for a perfectly delivered
assignment.