Watson Takes a Turn on Wall Street

By Michael Feldman

April 7, 2011

In the wake of Watson’s dominant performance on Jeopardy last month, IBM has taken the technology on the road to showcase it to anyone who’ll listen. On Monday, Watson — or rather, its keepers — headlined the opening session of the High Performance Computing Linux Financial Markets Conference in New York City. There was even a Watson demo at the conference, attracting crowds in IBM’s exhibitor booth.

If presenting a research project to a Wall Street crowd seems unusual, keep in mind that IBM does not intend to keep Watson in the lab forever. Commercialization of the technology is clearly in the company’s plan.

Following its Jeopardy win, the supercomputer’s next task will be to apply its analytic smarts to healthcare applications. IBM and Nuance Communications in collaboration with Columbia University Medical Center and the University of Maryland School of Medicine are looking for ways to use Watson to help doctors with patient diagnoses in a real-world medical setting.

Since I was in town for the HPC Wall Street conference, I got the opportunity to chat with two of the IBM’ers that spoke on the topic — Jean Staten Healy, director of the Cross-IBM Linux group, and Edward Epstein, manager of Unstructured Information at IBM Research — and ask them about the how the technology could be applied to financial services.

First though, I wanted to find out more about the Watson design and how it evolved over the three-year project. Since Epstein was one of the primary developers of the Watson software, he was able to give me a rundown on the supercomputer’s path to Jeopardy stardom.

According to him, Watson had a rather unimpressive start. In its first incarnation, it took two hours to spit out the answer to a question (or rather the question to the answer), which obviously wouldn’t do for a prime-time game show. The IBM engineers soon realized they had to do a serious redesign of the 750,000 lines of code if they were ever to be competitive on Jeopardy.

First off, all the data (dictionaries, encyclopedias, historical texts, etc.) had to be placed into RAM. Waiting precious milliseconds for disk reads is a performance killer, so everything got stuffed into memory for lightening-fast access.

But most of the initial effort to boost execution speed involved scaling out the software such that the hundreds of analytics algorithms and natural language processing (NLP) code could be run in parallel. The algorithms were parallelized across the analytics framework — in this case Apache UIMA (Unstructured Information Management Architecture), an open source information management environment that was at the heart of Watson’s software. Also, the search algorithms that looked up data references were distributed across the available cores of the Watson cluster. When the initial scale-out effort was done, there were about 200 Java processes as well as an additional 200 C++ processes running in parallel on Watson’s hardware.

According to Epstein that effort reduced the average answer time to just over 14 seconds. Since, in Jeopardy, you need have the answer in just a few seconds — in most cases just a fraction of a second right after the clue is read — they still needed another four-fold performance boost. Most of that was achieved by precomputing the deep NLP analysis of the pre-canned text and by hammering on every computation outlier. With that accomplished, the average answer time was trimmed to 3.6 seconds — on par with a human Jeopardy champ.

The software development work and the initial sparring matches for Watson were done on an IBM x86 blade cluster, outfitted with Xeon Nehalem CPUs. That system had the ability to store intermediate results, so that during test runs, the software team could execute a partial scenario, and return to it later to run a new calculation based on those intermediates. Also during development, it was important to run thousands of questions simultaneously, rather than a single question for fast real-time execution. So the system was scaled differently than the final Power 750 cluster that was used in the Jeopardy match.
 
The x86 development cluster had much less powerful processors, less memory, and most importantly less memory bandwidth compared to the Power 750 machine. Fundamentally, Watson is a big data app that feeds large amounts of information through a complex framework of analytics software. The fact the this needs to be done interactively puts particular constraints on performance.

According to Epstein, they needed the performance of the Power 750 to be competitive in Jeopardy. Fortunately, porting the software from the x86 blades system to the Power cluster was fairly straightforward, given that the software stack is all based on portable technology (Java, C++, Linux, and UIMA).

A single 750 node has four 8-core 3.5 GHz Power7 CPUs, and the entire system consisted of ninety such nodes, encapsulating 2,880 CPUs and 16 TB of RAM. The peak performance of the Jeopardy system is estimated to be about 80 teraflops.

The Watson software team added a number of Power7 optimizations to bump up the performance a bit more. Most of that involved using NUMA control to pin software processes to specific resources in the machine. “If you’re really trying to get that last edge in performance, then you do things like that,” said Epstein.

The ninety cluster Power7 was probably a bit of overkill for the Jeopardy match. Epstein estimates that CPU utilization was in the neighborhood of 30 percent during the clue processing (So theoretically, Watson could have been playing two additional Jeopardy matches simultaneously.) In any case, it was Epstein’s task to win the match at any cost, CPU utilization be damned. “I had the luxury of having enough hardware to do this job for Jeopardy,” he explained.

So what is Watson doing on Wall Street? IBM might be looking to attract some willing partners for a Watson-style financial analytics project analogous to the aforementioned healthcare research initiative. Big Blue is obviously proud of the technology and believes the system can be applied to all sorts of deep analytics work.

Epstein himself is currently working in the group involved in the healthcare project, but there are a number of individuals who are exploring “other opportunities.” One group is specifically focused on the financial application space.

IBM’s Healy believes a major focus for the technology in the financial arena will involve risk management. The idea is to provide results that will enable investors and money managers to make very fast decisions based on market conditions. Healy said it would not just involve spitting out a single answer like in Jeopardy, but also provide metrics of confidence about that answer, as well as some sort of evidence trail of its analysis.

Healy also suggested the possibility that Watson could serve as a resource for individuals making personal investments decisions. One could envision a sort of “Ask Watson” application that could serve thousands or even millions of investors simultaneously (assuming the machine was scaled appropriately). For this type of work, Watson might have to solicit information from the user based on the specific investment question. In that sense, Watson couldn’t just be an answer machine; it would need some rudimentary conversational skills as well. While Healy concedes the technology is still in the research stage, from her perspective, it has many applications going forward.

I suspect Watson will show up at a lot of conferences this year as IBM tests the waters for the technology. Deep analytics is broadly applicable to many domains and this has all the makings of a high-margin business for IBM. They just need to gather some proof points.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

MLPerf Inference 4.0 Results Showcase GenAI; Nvidia Still Dominates

March 28, 2024

There were no startling surprises in the latest MLPerf Inference benchmark (4.0) results released yesterday. Two new workloads — Llama 2 and Stable Diffusion XL — were added to the benchmark suite as MLPerf continues Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing power it brings to artificial intelligence.  Nvidia's DGX Read more…

Call for Participation in Workshop on Potential NSF CISE Quantum Initiative

March 26, 2024

Editor’s Note: Next month there will be a workshop to discuss what a quantum initiative led by NSF’s Computer, Information Science and Engineering (CISE) directorate could entail. The details are posted below in a Ca Read more…

Waseda U. Researchers Reports New Quantum Algorithm for Speeding Optimization

March 25, 2024

Optimization problems cover a wide range of applications and are often cited as good candidates for quantum computing. However, the execution time for constrained combinatorial optimization applications on quantum device Read more…

NVLink: Faster Interconnects and Switches to Help Relieve Data Bottlenecks

March 25, 2024

Nvidia’s new Blackwell architecture may have stolen the show this week at the GPU Technology Conference in San Jose, California. But an emerging bottleneck at the network layer threatens to make bigger and brawnier pro Read more…

Who is David Blackwell?

March 22, 2024

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HBM3E memory as well as the the ability to train 1 trillion pa Read more…

MLPerf Inference 4.0 Results Showcase GenAI; Nvidia Still Dominates

March 28, 2024

There were no startling surprises in the latest MLPerf Inference benchmark (4.0) results released yesterday. Two new workloads — Llama 2 and Stable Diffusion Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing po Read more…

NVLink: Faster Interconnects and Switches to Help Relieve Data Bottlenecks

March 25, 2024

Nvidia’s new Blackwell architecture may have stolen the show this week at the GPU Technology Conference in San Jose, California. But an emerging bottleneck at Read more…

Who is David Blackwell?

March 22, 2024

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HB Read more…

Nvidia Looks to Accelerate GenAI Adoption with NIM

March 19, 2024

Today at the GPU Technology Conference, Nvidia launched a new offering aimed at helping customers quickly deploy their generative AI applications in a secure, s Read more…

The Generative AI Future Is Now, Nvidia’s Huang Says

March 19, 2024

We are in the early days of a transformative shift in how business gets done thanks to the advent of generative AI, according to Nvidia CEO and cofounder Jensen Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, codenamed Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Leading Solution Providers

Contributors

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

Intel Won’t Have a Xeon Max Chip with New Emerald Rapids CPU

December 14, 2023

As expected, Intel officially announced its 5th generation Xeon server chips codenamed Emerald Rapids at an event in New York City, where the focus was really o Read more…

IBM Quantum Summit: Two New QPUs, Upgraded Qiskit, 10-year Roadmap and More

December 4, 2023

IBM kicks off its annual Quantum Summit today and will announce a broad range of advances including its much-anticipated 1121-qubit Condor QPU, a smaller 133-qu Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire