DeepStack, a computer program developed by scientists from the University of Alberta in collaboration with researchers from both Czech Technical University and Prague’s Charles University, became the first AI to best real poker players at no-limit Texas Hold’em.
The news came more than a month after Carnegie Mellon University’s poker bot, Libratus, defeated a team of accomplished poker professionals, including Jason Les and Daniel McAuley, for 20 days straight.
Libratus may have been the first AI publicized to conquer no-limit hold’em, but DeepStack was the first to actually do it in December 2016. But the public announcement, it turns out, was put off until the Czechs’ findings could be properly peer-reviewed.
Besting the Simple Humans
According to a research paper published in Science, a ten-member team, led by Michael Bowling, recruited 33 poker pros representing 17 countries from the International Federation of Poker to take on DeepStack in 3,000 games over four weeks. Granted, the competition was neither as accomplished nor experienced as the Libratus challengers, likely the result of less prize money being offered.
Just 11 of the players managed to complete the 3,000 games, and all of them lost to the AI program, which took an average of just three seconds to act. Over 44,000 hands (compared to the Libratus sample size of 120,000 hands), DeepStack defeated the competition with statistical significance.
“Poker has been a longstanding challenge problem in artificial intelligence,” Bowling said in a University of Alberta announcement. “It is the quintessential game of imperfect information in the sense that the players don’t have the same information or share the same perspective while they’re playing.”
Despite its complexity, DeepStack can operate on your run-of-the-mill laptop using a Nvidia graphics processing unit.
Mastering the Subtleties of the Game
The DeepStack algorithm was actually the next step in AI evolution. In 2008, the same team developed “Polaris,” which bested top poker players in heads-up limit hold’em. Seven years later, they would solve that particular poker variant with the introduction of “Cepheus.”
Unlike games like checkers and chess, no-limit hold’em is a game of incomplete information, something AI has had a problem contending with in the past. DeepStack was designed with this in mind, by avoiding a set strategy and instead picking an applicable one each time it acts.
“We train our system to learn the value of situations,” Bowling continued. “Each situation itself is a mini poker game. Instead of solving one big poker game, it solves millions of these little poker games, each one helping the system to refine its intuition of how the game of poker works. And this intuition is the fuel behind how DeepStack plays the full game.”
While the study was centered on poker, the hope is the results will help outside the game, specifically with real-world problems involving incomplete information like medical, tactical negotiation, and defense situations.
It’s also important to note both the Libratus and DeepStack AI programs were designed for heads-up no-limit hold’em. Researchers have yet to tackle much more complicated poker scenarios, such as either six or nine-handed play.
