Exercise 3
Box Moving Robot
Shira Gottlieb 040144594
Along Koza's paper I used Mahadevan and Connell's original paper
about experimenting reinforcement learning (for the definition of 'bump' and 'stuck' parameters).
Here is a short movie of their robot in action, pushing boxes…
Implementation
A gene is a scheme program built as a tree, where:
Terminals = {S00, S01, S02, S03, ... , S11, SS, (MF), (TR), (TL)}
Functions = {IFBMP, IFSTK, IFLTE, PROGN2}
Fitness function is:
(distance of center of mass of the box from nearest wall) * 4runs;
Where each run has a different robot starting point, and there is a limit of 350 time units (that is, if the robot doesn't move the box within 350 time units the evolution terminates).
- Selection is done by tournament selection, with k = 2 for all runs but the run with population size = 500, in which k = 3.
- Crossover is done by selecting one random node at each parent and replacing the sub tree whose root is the selected node with the sub tree of the other parent's selected node. pc = 0.7.
- Mutation is done by selecting a random node and replacing its sub tree by growing a new tree instead. The new tree is "grown" in the grow method, in order to avoid oversized "fat" trees. pm = 0.01.
- Population initialization is done by the 'ramped half and half method', with maximal depth = 5.
As stated in Mahadevan and Connell's paper, the Bump detector detects whether there is an object within 30 cm ahead of the robot's face angle (meaning that if the robot is moving forward it will bump the object).
The Stuck detector detects whether the robot got 'stuck' in something and he cannot move it or itself (that is – it got stuck in a wall).
Physics Model
The difficulty was calculating the rotation of the box, when the robot hits it not coaxial with the box's center of mass.
When the robot hits the box, the force he applies on it is divided to 2 components:
- force in direction of the center of mass of the box (which will cause the box to move).
- force orthogonal to the box's center of mass direction (causing it to rotate).
The calculation was done in the following way:
I calculated the point which the robot was suppose to move to from the moment he started moving until he reaches the box (in he's regular moving speed), and from the moment he reaches the box until his final destination (in a slower speed).
When moving along with the box (that is, pushing it; the time from reaching the box to the final destination), the force is divided to 2 components:
- The distance the box moves:
I'll define segment [bc], where b is the point the robot hit the box, and c is the point where the robot is supposed to move to,
and segment [ab], where a is the box's center of mass point.
Let alpha be the angle [abc], then [bc]*cos(alpha) is the distance the box's center is moving in [ba] direction.
(If [bc]*cos(alpha)<0 the box will not move but only rotate).
- Let [bd] be the projection of [bc] on an infinite line orthogonal to [ab].
Let beta be the angle [dab]. This is the box's rotating angle.
It is important to note that this is not a full physical model, since forces such as friction, acceleration etc were not included. However, it is realistic enough and emphasises the conditions in the problem.
Here is a live demonstration of the forces:
(In order to view the movies, download the movie to your computer and then open it with a suitable program)
a note about measuring units
Koza's paper uses feet as its measure unit. I adopted the metric system instead. Here are the room's parameters:
- room size: 30X30 m
- box size: 3X3 m
- robot speed, moving: 1 m per time unit
- robot speed, pushing: 1/3 m per time unit
Code
code
All the code is written in Java.
Test.java class is testing the genome (Lisp program) on different starting points.
I used MatLab to view stills images and create movies of my robot pushing boxes.
Experiment and Conclusions
I first ran the evolution on a small population (100 individuals), and let the fitness be calculated only from one run (with one starting point, and not as mentioned above - with 4 runs).
I found out that even if the robot had a good genome and successfully moved the box to a wall, it didn't do it when I gave the same genome a different starting point. That is, the robot wasn't "smart" enough to perform the task in a place he didn't know.
Below we see the robot moving the box from a point known to it.
Here is the same genome starting at point (28, 3), trying to move a box with no success.
Next, I tested the evolution on the same population size (100 individuals), but this time with 4 runs each time, starting in 4 different starting points for the robot (exactly as I mentioned above in the fitness description).
This time, the robot learnt to move the box to the wall from each the four starting points. However, it did not succeed in moving the box to a wall when its starting points were new and different from the ones the evolution was given.
The starting points were: (5, 5), (25, 5), (18, 25), (28, 3).
Here is the robot successfully moving a box starting at point (28,3):
Here is the same genome starting at (15, 9):
Perhaps the population was too small, and there was not much diversity.
I then enlarged the population size to 500, k = 3 (the size of the group in tournament selection), added elitism with 2% rate and ran the algorithm with 4 different starting points each run.
The best genome is:
(IFLTE (MF) (IFLTE (IFSTK (S00) (IFLTE (IFLTE (S08) (S08) (S07) (S06)) (IFBMP (IFSTK (SS) (IFLTE (PROGN2 (S02) (SS)) (IFSTK (TL) (IFSTK (SS) (IFLTE (PROGN2 (S02) (IFSTK (S08) (S01))) (IFLTE (IFSTK (S00) (IFLTE (IFLTE (S08) (S08) (S07) (S06)) (IFBMP (IFSTK (SS) (IFLTE (PROGN2 (S08) (S02)) (IFSTK (TL) (IFSTK (SS) (IFLTE (PROGN2 (S02) (S02)) (IFSTK (TL) (S07)) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S08)) (IFSTK (TL) (S08)) (S05) (IFLTE (IFLTE (S02) (S08) (IFSTK (IFLTE (IFSTK (S08) (S01)) (IFSTK (S10) (S08)) (IFLTE (S02) (S00) (S08) (S05)) (IFLTE (IFLTE (S07) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (S08) (IFSTK (S08) (S01))) (S00)) (S10) (IFBMP (PROGN2 (S02) (S10)) (SS)))) (S01)) (S08)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (SS)))) (S02) (IFLTE (IFSTK (S07) (S01)) (IFSTK (TL) (IFSTK (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (S10)) (IFBMP (PROGN2 (PROGN2 (S02) (S10)) (IFLTE (S08) (IFSTK (TL) (S08)) (IFSTK (SS) (IFSTK (IFSTK (S08) (S06)) (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (S07) (S00) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S00)) (S06)) (IFBMP (IFSTK (SS) (S08)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS)))))) (IFLTE (IFLTE (S07) (S08) (IFSTK (S01) (S01)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (SS))) (S08))) (S05) (S06)) (S05)) (PROGN2 (IFLTE (S08) (S08) (S07) (S06)) (S10))))) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (S07) (S00) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S01) (S01)) (S06)) (IFBMP (IFSTK (SS) (S01)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (S02) (IFLTE (IFSTK (S07) (S01)) (IFSTK (S08) (S08)) (S05) (PROGN2 (S08) (S10))) (S05)) (PROGN2 (S10) (S10)))) (S00)) (IFSTK (S10) (MF)) (IFBMP (S10) (S05)))) (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (IFLTE (S08) (S08) (S07) (S06)) (IFBMP (IFSTK (SS) (IFLTE (PROGN2 (S08) (S02)) (IFSTK (TL) (IFSTK (SS) (IFLTE (PROGN2 (S02) (S02)) (IFSTK (TL) (S07)) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S08)) (IFSTK (TL) (S08)) (S05) (IFLTE (IFLTE (S02) (S08) (IFSTK (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFSTK (TL) (S08)) (IFLTE (IFLTE (S07) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (S08) (IFSTK (S08) (S01))) (S00)) (S10) (IFBMP (PROGN2 (S02) (S10)) (SS)))) (S01)) (S08)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (SS)))) (S02) (IFLTE (IFSTK (S07) (S08)) (IFSTK (TL) (S00)) (S05) (S06)) (S05)) (PROGN2 (IFLTE (S08) (S08) (S07) (S06)) (S10))))) (IFLTE (IFSTK (IFSTK (S02) (S01)) (IFLTE (IFSTK (IFSTK (S08) (S01)) (S01)) (IFSTK (TL) (S08)) (IFLTE (S07) (S00) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S01) (S01)) (S06)) (IFBMP (IFSTK (SS) (S01)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (S02) (IFLTE (IFSTK (S07) (S01)) (IFSTK (S08) (S08)) (S05) (PROGN2 (S08) (S10))) (S05)) (PROGN2 (S10) (S10)))) (S00)) (IFSTK (S10) (MF)) (IFBMP (S07) (IFSTK (S11) (S10)))) (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (S07) (S08) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S08)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS)))) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (IFLTE (S07) (S00) (S10) (S05)) (PROGN2 (S02) (S10))) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (SS)) (S05) (IFLTE (IFLTE (S02) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (SS)))) (IFSTK (S08) (S01)) (IFLTE (IFSTK (S07) (S08)) (IFSTK (S07) (S00)) (S05) (PROGN2 (S00) (PROGN2 (S06) (IFSTK (S08) (S01))))) (S05)) (PROGN2 (IFLTE (S08) (S08) (S07) (IFLTE (IFLTE (S02) (S08) (S07) (S10)) (IFBMP (IFSTK (SS) (IFLTE (PROGN2 (S02) (S02)) (IFSTK (TL) (S08)) (IFLTE (IFSTK (IFSTK (SS) (IFLTE (PROGN2 (S02) (S02)) (IFSTK (TL) (S07)) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (S07) (TL) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S02) (S01)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (S08)))) (S02) (IFLTE (IFSTK (S07) (S08)) (IFSTK (TL) (S00)) (S10) (PROGN2 (S05) (S10))) (S05)) (PROGN2 (S06) (S10)))) (IFLTE (IFSTK (S08) (S07)) (IFSTK (PROGN2 (S02) (S10)) (S08)) (IFLTE (S07) (S00) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (S02) (IFLTE (IFSTK (S07) (S08)) (MF) (S05) (PROGN2 (S05) (S10))) (S05)) (PROGN2 (S06) (S10)))) (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (S07) (S00) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S01) (S01)) (S06)) (IFBMP (IFSTK (SS) (S01)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (IFSTK (S10) (S01)) (IFBMP (S08) (IFSTK (S11) (S10))))) (IFSTK (S08) (S01)))))) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (IFLTE (S08) (S08) (S07) (S06)) (IFBMP (IFSTK (SS) (IFLTE (PROGN2 (S08) (S02)) (IFSTK (TL) (IFSTK (SS) (IFLTE (PROGN2 (S02) (S02)) (IFSTK (TL) (S07)) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S08)) (IFSTK (TL) (S08)) (S05) (IFLTE (IFLTE (S02) (S08) (IFSTK (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFSTK (TL) (S08)) (IFLTE (IFLTE (S07) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (S08) (IFSTK (S08) (S01))) (S00)) (S10) (IFBMP (PROGN2 (S02) (S10)) (SS)))) (S01)) (S08)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (SS)))) (S02) (IFLTE (IFSTK (S07) (S08)) (IFSTK (TL) (S00)) (S05) (S06)) (S05)) (PROGN2 (IFLTE (S08) (S08) (S07) (S06)) (S10))))) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (S07) (S00) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S01) (S01)) (S06)) (IFBMP (IFSTK (SS) (S01)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (S02) (IFLTE (IFSTK (S07) (S01)) (IFSTK (S08) (S08)) (S05) (PROGN2 (S08) (S10))) (S05)) (PROGN2 (S10) (S10)))) (S00)) (IFSTK (S10) (MF)) (IFBMP (S07) (IFSTK (S11) (S10)))) (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFLTE (IFSTK (S08) (S02)) (IFSTK (TL) (S08)) (IFLTE (S07) (S08) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S08)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS)))) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (S06) (IFLTE (IFSTK (S07) (S01)) (IFSTK (TL) (S00)) (S05) (PROGN2 (S08) (S10))) (S05)) (PROGN2 (IFLTE (S08) (S08) (S07) (S06)) (S10)))) (S00)) (IFSTK (S10) (MF)) (IFBMP (S08) (IFSTK (S11) (S10))))) (IFSTK (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (SS) (S07)) (S00)) (IFSTK (MF) (S10)) (IFBMP (PROGN2 (S02) (IFLTE (S07) (IFSTK (TL) (S10)) (IFLTE (S07) (S00) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S02) (S00)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (S08))))) (SS))) (S08)) (IFLTE (S07) (S00) (S10) (S05)) (PROGN2 (S02) (S10))) (S10) (TL))
This time the it studied all 4 points successfully; Here is the robot moving the box from all 4 starting points:
Starting at (5, 5):
Starting at (25, 5):
Starting at (18, 25):
Starting at (28, 3):
And even from points he never saw before:
(1, 1):
(11, 13):
Some movies of the best evolved robot pushing boxes
Best Genome's Evolution
Here are some plots of the best genome in several generations along the evolutionary run:
Generation 0
(IFLTE (MF) (IFLTE (PROGN2 (S03) (S11)) (S08) (S08) (S09)) (IFBMP (TR) (S00)) (TL))
... and what it is able to do:
Generation 10
(IFLTE (IFLTE (IFSTK (S11) (S10)) (PROGN2 (S06) (S09)) (IFBMP (S11) (S03)) (MF)) (PROGN2 (PROGN2 (MF) (S10)) (IFBMP (S10) (S03))) (IFLTE (IFLTE (S08) (S08) (S07) (S06)) (IFBMP (S07) (S00)) (IFSTK (S01) (MF)) (IFBMP (SS) (S09))) (IFLTE (PROGN2 (S03) (TL)) (IFSTK (SS) (S09)) (PROGN2 (S08) (S06)) (PROGN2 (S05) (TR))))
Generation 45
(IFLTE (MF) (IFLTE (IFSTK (S00) (IFLTE (IFLTE (S08) (S08) (S07) (S06)) (IFBMP (IFSTK (SS) (IFLTE (IFSTK (IFSTK (S00) (S10)) (S01)) (IFSTK (TL) (S08)) (IFLTE (S07) (S02) (IFLTE (IFSTK (IFSTK (MF) (S10)) (S01)) (IFSTK (TL) (S00)) (IFLTE (S07) (S00) (S08) (S05)) (PROGN2 (S00) (S10))) (S05)) (PROGN2 (S01) (S10)))) (S00)) (IFSTK (S01) (MF)) (IFBMP (S08) (IFSTK (S11) (S10))))) (IFSTK (TL) (S08)) (IFLTE (S07) (S00) (S10) (S05)) (PROGN2 (S02) (S10))) (S07) (TL))
Generation 120
(IFLTE (MF) (IFLTE (IFSTK (S00) (IFLTE (IFLTE (S08) (S08) (S07) (S06)) (IFBMP (IFSTK (SS) (IFLTE (PROGN2 (S02) (SS)) (IFSTK (TL) (IFSTK (SS) (IFLTE (PROGN2 (S02) (S02)) (IFSTK (TL) (S07)) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (SS)) (S05) (IFLTE (IFLTE (S02) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (SS)))) (IFSTK (S08) (S01)) (IFLTE (IFSTK (S07) (S08)) (IFSTK (S07) (S00)) (S05) (PROGN2 (S00) (PROGN2 (S06) (IFSTK (S08) (S01))))) (S05)) (PROGN2 (IFLTE (S08) (S08) (S07) (IFLTE (IFLTE (S02) (S08) (S07) (S06)) (IFBMP (IFSTK (SS) (IFLTE (PROGN2 (S02) (S02)) (IFSTK (TL) (S08)) (IFLTE (IFSTK (IFSTK (SS) (IFLTE (PROGN2 (S02) (S02)) (IFSTK (TL) (S07)) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (S07) (TL) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (IFLTE (IFLTE (S10) (S08) (S07) (S06)) (IFBMP (IFSTK (SS) (IFLTE (PROGN2 (S02) (S02)) (IFSTK (TL) (S07)) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S00) (S01)) (IFSTK (S08) (S08)) (IFLTE (S07) (S00) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (S02) (IFLTE (IFSTK (S07) (S08)) (IFSTK (TL) (S00)) (S05) (PROGN2 (S00) (S10))) (S05)) (PROGN2 (IFLTE (S08) (S08) (S07) (S06)) (S10)))) (S00)) (IFSTK (S00) (MF)) (IFBMP (S01) (S01)))) (SS))))) (S02) (IFLTE (IFSTK (S07) (S08)) (IFSTK (TL) (S00)) (S10) (PROGN2 (S05) (S10))) (S05)) (PROGN2 (S06) (S10)))) (IFLTE (IFSTK (S08) (S01)) (IFSTK (PROGN2 (S02) (S10)) (S08)) (IFLTE (S07) (S00) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (S02) (IFLTE (IFSTK (S07) (S08)) (MF) (S05) (PROGN2 (S05) (S10))) (S05)) (PROGN2 (S06) (S10)))) (S00)) (IFSTK (S10) (S01)) (IFBMP (S08) (IFSTK (S11) (S10))))) (S10))))) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (IFLTE (S08) (S08) (S07) (S06)) (IFBMP (IFSTK (SS) (IFLTE (PROGN2 (S08) (S02)) (IFSTK (TL) (IFSTK (SS) (IFLTE (PROGN2 (S02) (S02)) (IFSTK (TL) (S07)) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S08)) (IFSTK (TL) (S08)) (S05) (IFLTE (IFLTE (S02) (S08) (IFSTK (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (S02) (S00) (S08) (S07)) (IFLTE (IFLTE (S07) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (S08) (IFSTK (S08) (S01))) (S00)) (S10) (IFBMP (PROGN2 (S02) (S10)) (SS)))) (S01)) (S08)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (SS)))) (S02) (IFLTE (IFSTK (S07) (S08)) (IFSTK (TL) (S00)) (S05) (S06)) (S05)) (PROGN2 (IFLTE (S08) (S08) (S07) (S06)) (S10))))) (IFLTE (IFSTK (IFSTK (S08) (S01)) (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (S07) (S00) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S01) (S01)) (S06)) (IFBMP (IFSTK (SS) (S01)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (S02) (IFLTE (IFSTK (S07) (S01)) (IFSTK (S08) (S08)) (S05) (PROGN2 (S08) (S10))) (S05)) (PROGN2 (S10) (S10)))) (S00)) (IFSTK (S10) (MF)) (IFBMP (S08) (IFSTK (S11) (S10)))) (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFLTE (IFSTK (S08) (S01)) (IFSTK (TL) (S08)) (IFLTE (S07) (S08) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S08)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS)))) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (S02) (IFLTE (IFSTK (S07) (S01)) (IFSTK (TL) (S00)) (S05) (PROGN2 (S08) (S10))) (S05)) (PROGN2 (IFLTE (S08) (S08) (S07) (S06)) (S10)))) (S00)) (IFSTK (S10) (MF)) (IFBMP (S08) (IFSTK (S11) (S10))))) (IFSTK (IFLTE (IFLTE (S08) (S08) (IFSTK (S08) (S01)) (S06)) (IFBMP (IFSTK (SS) (S07)) (S00)) (IFSTK (MF) (S10)) (IFBMP (PROGN2 (S02) (IFLTE (S07) (IFSTK (TL) (S10)) (IFLTE (S07) (S00) (S08) (S05)) (IFLTE (IFLTE (S08) (S08) (IFSTK (S02) (S00)) (S06)) (IFBMP (IFSTK (SS) (S09)) (S00)) (IFSTK (S01) (MF)) (IFBMP (PROGN2 (S02) (S10)) (SS))))) (SS))) (S08)) (IFLTE (S07) (S00) (S10) (S05)) (PROGN2 (S02) (S10))) (S10) (TL))
Best run graph:
Discussion
- Using only one starting point in the evolution tends to create robot that can't operate properly from different starting points
- A bigger population makes it more likely for the robots to learn to solve the general problem and be able to push the box starting from 'unseen' points
- It is interesting to see a relatively big improvement of the best individual at around generations 126, and a few generations later, at generation 132 a noticeable improvement in the average fitness. I guess the reason for that was that in generation 126 a "good" individual has evolved (either by crossover or mutation), with a major improvement in its box-moving skills. Being selected for crossover more often (and automatically copied to the next generation due to the elitism), this "better" robot has spread its genes to the population, causing an improvement in the average fitness a few generations after the first emergence of the original "good" robot