Introduction to Artificial Inteligence

Programming assignment - learning with artificial neural networks

In this assignment, you will implement neural networks and train them with a genetic algorithm. The program will include two modules: the neural network itself, and the training module, that modifies the network weights so as to achieve optimal peformance on a given function.

The neural network structure is feed-forward, specified by the number of units per layer. Assume that the output of each unit at level k is connected (only) to the input of all units at level k+1. Specification will be by an ASCII file, where the first line will contain just the number of layers, and then the number of units per layer, separated by white-space. The order is: the number of units at layer 1 (input), then layer 2, etc., and finally number of units at output layer. For example, the following specifies a network with 3 input units, 1 intermediate (hidden) unit, and 2 output units.

3 3 1 2

Note that input units do nothing but pass the input signal to the next layer. Thus, the number of layers will always be at least 2, and you may assume it is no more than 10. Also, each layer will never be required to have more than 10 units. The TASK for a neural network is the same ASCII file - a TRUTH TABLE for some BINARY VALUED function. Each line in the file specifies the outputs for each combination of input values. For example, a TASK file for the above network might be:

0 1
1 1
0 0
1 0
1 1
1 1
1 0
1 0

Note that the number of lines specifying the function is 2^m, where m is the number of input units. The number of data columns is equal to the number of output units, obviously. The file mapping is such that the first data line is for input 000, then for input 001, etc. (counting in binary), until 111. As shown above, since the input is implicit, it is not in the file - only the required output. Data lines have the numbers separated by white space, with a linefeed charcter between lines.

The program that trains the network starts with a population including random initial weight assignments. Use any reasonable variant of genertic algorithm to modify the population until a stopping criterion is reached. (Example: no errors for the best individual over the entire set of examples, or 1000 generations, or 10 generations without improvement, whatever comes first). Use initial weights uniformly generated in the range [-1,1]. Use a step function with a threshold of 1 as the activation function. After training, the program should print the score for the best network, its weights, and examples for which the network still produces incorrect results, if any.

Deliverables:

1. Your program.
2. Documentation on what your program does - specify which variant of GA you were using, how crossover and mutation were implemented, and the value of all parameters (such as mutation rate, etc.).
3. Report on a set of experiments - number of errors as a function of number of training sessions, on a non-trivial learning example.

Deadline: June 9, 2003. Extended to June 12.