The New York Times has an excellent article on IBM’s ‘Watson’ project which is an artificial intelligence system designed to answer natural language queries to the point where it can beat humans at Jeopardy! quiz show questions – where contestants are given an answer and they have to come up with the question.
Natural language questions are traditionally very difficult for computers because they involve a lot of assumptions. For example, take the question “How many people work in a bank?” To answer the question you need to understand that ‘bank’ refers to a financial institution and not a river bank.
Answering this question needs pre-existing knowledge and, computationally, two main approaches. One is constraint satisfaction, which finds which answer is the ‘best fit’ to a problem which doesn’t have mathematically exact solution; and the other is a local search algorithm, which indicates when further searching is unlikely to yield a better result – in other words, when to quit computing and give an answer – because you can always crunch more data.
If you’re not familiar with it, the quiz show Jeopardy! is a a particularly difficult version of this because it gives people answers and they have to provide correct question: such as “A singer who was touched for the very first time and became the material girl” – the winning contestant would be the first to respond with “Who is Madonna?”
In a major advance for artificial intelligence IBM have developed a system that can beat humans at the quiz. Although the ability to publicly trounce puny humans in quiz shows is not necessarily the greatest contribution to humanity, this is just a way of testing the system which could be deployed to answer unprepared question based on large datasets.
Watson applies computational linguistics to extract knowledge from text – a technique sometimes known as text mining and then applies constraint satisfaction and local search algorithms to produce reasonable answers quickly.
This could be very useful for asking questions of large datasets which someone may not have necessarily asked before – such ‘which drug shows the best promise for treating tuberculosis?’
The article has lots of great insights into the difficulties of artificial intelligence. I particularly liked this section:
To avoid losing money — Watson doesn’t care about the money, obviously; winnings are simply a way for I.B.M. to see how fast and accurately its system is performing — Ferrucci’s team has programmed Watson generally not to buzz until it arrives at an answer with a high confidence level. In this regard, Watson is actually at a disadvantage, because the best “Jeopardy!” players regularly hit the buzzer as soon as it’s possible to do so, even if it’s before they’ve figured out the clue. “Jeopardy!” rules give them five seconds to answer after winning the buzz. So long as they have a good feeling in their gut, they’ll pounce on the buzzer, trusting that in those few extra seconds the answer will pop into their heads. Ferrucci told me that the best human contestants he had brought in to play against Watson were amazingly fast. “They can buzz in 10 milliseconds,” he said, sounding astonished. “Zero milliseconds!”
Buzzing just on a ‘gut feeling’ is an example of what psychologists called ‘metacognition‘ or a little more crudely ‘thinking about thinking’. More specifically in this case its an example of humans relying on their ‘feeling of knowing‘.
‘Feeling of knowing’ is used a little differently in memory and decision making research, but it essentially boils down to the feeling that you know something, without necessarily having to bring the thing to mind. In some ways, it’s similar to when you look at something and decide whether you can lift it or not, without actually having to try and pick it up.
In other words, its being able to manage your mental resources based on estimations. This has become one of the core problems of artificial intelligence.
Computation is easy. Meta-computation, it turns out, is a bitch.
Link to NYT piece ‘What Is I.B.M.’s Watson?’
Mind Hacks 