The scientific method – lego robots edition

At the University of Sheffield we’ve been teaching psychology using lego robots. This isn’t as peculiar as it might sound. You can learn a lot about your theories by trying to build them into a machine or computer programme. But while teaching the course, I discovered that you can also learn a lot about the methods used in experimental psychology by trying them out on robots.

This is one of the lego robots we were using. They are built using a Lego Mindstorms set and inspired by a book by Valentino Braightenberg called ‘Vehicles: Experiments in Synthetic Psychology‘.

The robot has a light sensor on each side and a nose-bumper which tells it when it has hit something. A simple brain connects these sensors with two independently powered wheels. Here’s the robot in action:

The suprising thing, and a crucial point of Braitenberg’s book, is that you can get what looks like complex behaviour (in this case line following) from simple rules. All that governs this robot’s behaviour is a positive connection between each light sensor and the wheel on the same side. This makes the robot turn away from lighter floor patches, so in this environment it traces the edge of the patten. An additional ‘fixed action pattern‘ makes it spin around and start in another direction if it bumps into something.

Many people look at these robots and over-intepret the complexity of their behaviour. You need skepticism and controlled experiments to discover exactly how simple the rules controlling the robot are. However, while I was trying to use the robots to teach this to my class, the robots and the class conspired to teach me something.

In a more advanced class I put a simple learning rule in the robot’s brain so that they could learn to slow down before hitting walls (actually, it is only true that I put the rule in the robot’s brain in the sense that Hitler invaded Poland. In truth I made a grad student programme the rule into the robot. Thanks Stuart!).

The task I set the class was simple, I thought: run an experiment to see the robot learn over successive trials. Because I’d programmed the rule into the robot I thought I’d be able to predict the robot behaviour. The predicted learning curve of the robot looked like this:

The results from the groups looked like this

Since each robot was identical – same body, same brain in the way only robots can be – and all the groups were doing the same experiment, I expected to get the same results from each group. No luck there! Some get an increase, but with some the line stays almost flat. Some it goes up smoothly, some get wild swings in performance up and down.

And this got me to thinking. If the results are this variable with experimental subjects which we understand completely – their simple bodies are made of lego for goodness sake! the brains are identical and programmed by us! – how unreliable will results be if you experiment on real people? Noisy humans have bodies and brains which are both vastly more complex than lego robots, and each body and brain is unique. With so many sources of variability between individuals it ios amazing that experimental psychologists ever get any results at all.

The moral is that experimental work is hard, really hard. You’d better be sure your experiment reduces sources of variability as much as possible because there will be enough uncontrollable variability without you adding any more.

Fortunately there is a light at the end of the tunnel, in the form of statistics. If you average the different noisy group results you get something a bit more like the underlying pattern I knew to be there:

Trying a simple experiment with the lego robots gave me a new respect for the experimental method, and the difficulty psychologists face when trying to discover the rules underlying the wonderous variety in human behaviour.