Build Watson: An Overview of DeepQA for The

Jeopardy! Challenge
David Ferrucci, PhD
IBM Research
Hawthorne, NY
ferrucci@us.ibm.com

Abstract
Computer systems that can directly and accurately answer peoples' questions over a broad domain
of human knowledge have been envisioned by scientists and writers since the advent of computers
themselves. Open domain question answering holds tremendous promise for facilitating informed
decision making over vast volumes of natural language content. Applications in business
intelligence, healthcare, customer support, enterprise knowledge management, social computing,
science and government would all benefit from deep language processing. The DeepQA project
(www.ibm.com/deepqa) is aimed at illustrating how the advancement and integration of Natural
Language Processing (NLP), Information Retrieval (IR), Machine Learning (ML), massively
parallel computation and Knowledge Representation and Reasoning (KR&R) can greatly advance
open-domain automatic Question Answering. An exciting proof-point in this challenge is to
develop a computer system that can successfully compete against top human players at the
Jeopardy! quiz show (www.jeopardy.com). Attaining champion-level performance Jeopardy!
requires a computer to rapidly answer rich open-domain questions, and to predict its own
performance on any given category/question. The system must deliver high degrees of precision
and confidence over a very broad range of knowledge and natural language content and with a 3-
second response time. To do this DeepQA generates, evidences and evaluates many competing
hypotheses. A key to success is automatically learning and combining accurate confidences across
an array of complex algorithms and over different dimensions of evidence. Accurate confidences
are needed to know when to “buzz in” against your competitors and how much to bet. Critical for
winning at Jeopardy!, High precision and accurate confidence computations are just as critical for
providing real value in business settings where helping users focus on the right content sooner and
with greater confidence can make all the difference. The need for speed and high precision
demands a massively parallel compute platform capable of generating, evaluating and combing
1000’s of hypotheses and their associated evidence. In this talk I will introduce the audience to the
Jeopardy! Challenge and describe our technical approach and our progress on this grand-challenge
problem.

Categories & Subject Descriptors: Information Search and Retrieval, Software

Architectures, Natural Language.

General Terms: Algorithms, Performance, Design, Experimentation, Human Factors

Bio
The speaker is a Research Staff Member and Department Group Manager at IBM’s T.J. Watson
Research Center. He received his PhD from Rensselaer Polytechnic Institute. Dr. Ferrucci’s
primary technical interests are in natural language processing and knowledge representation and
reasoning.

Copyright is held by the author/owner(s).

PACT’10, September 11–15, 2010, Vienna, Austria.
ACM 978-1-4503-0178-7/10/09.