Secrets of Libratus Revealed: Paper Unmasks AI Behind Triumphant Poker Bot

The team behind Libratus, the poker bot that beat a team of pros for more than $1.7 million, has finally revealed how they created their artificial intelligence (AI) card player.

Artificial intelligence schematics for poker bot Libratus

Developers Noam Brown and Tuomas Sandholm reveal how poker bot Libratus used the concept of “Subgame solving” to beat a team of pros for more than $1.7 million. (Image: Science)

Outlining their raison d’être and subsequent methods in a research paper published in the journal Science, authors Noam Brown and Tuomas Sandholm of Carnegie Mellon University described the mammoth task of creating AI capable of cracking the code of a game as complex as no-limit Texas hold’em.

Cards Against Humanity

For those with short memories, Libratus stuck it to four poker pros back in January 2017. After playing Dong Kim, Jason Les, Bjorn Li, and Doug Polk heads-up for 120,000 hands, the poker bot won by a margin of $1,776,250.

Beyond the convincing victory, the performance did what many assumed was either impossible or, at least, many years away from happening: AI mastery of no-limit hold’em.

Although Libratus hasn’t “solved” the game by any imaginative stretch, the program showed that AI is now capable of beating some of the world’s top pros.

In a paper titled, “Superhuman AI for heads-up no-limit poker: Libratus beats top professionals,” Brown and Sandholm explain the rise of Libratus, and acknowledged tackling the lack of perfect information in the game as the toughest obstacle to overcome.

“Hidden information makes a game far more complex for a number of reasons,” the authors wrote. “Rather than simply search for an optimal sequence of actions, an AI for imperfect-information games must determine how to balance actions appropriately, so that the opponent never finds out too much about the private information the AI has.”

When a lack of direct information about a player’s hand is coupled with unrestricted betting, the number of variables in play becomes extremely vast. This necessitated a three-pronged approach to creating an advanced poker bot capable of competing at a higher level.

Subgame Solutions

First, Libratus was programed with an abstraction of the game to give the bot a basic understanding of general scenarios and game-theory for optimal play. This information formed the basis of the software’s moves in early rounds, when its human opponents didn’t have a read on how it was going to play.

Second, as the game develops, Libratus uses nested subgame solving. This basically takes a more detailed abstraction of subgames (i.e. the specific game in real-time rather than poker in general) and conducts an analysis of what’s going on in the moment. This information is then compared to the general abstractions defined in the first instance.

The major innovation in this area was how Libratus was able to add new information to its calculations to adapt, in real time, for when a player made an unexpected move.

Finally, Libratus was able to self-improve. After taking general and game-specific information, the bot could teach itself and refine its database of knowledge. Another way of looking at it is that Libratus could take information obtained in the first level and rewrite it based on new information and calculations from level two.

While the nuances of Libratus and how it beat four pros are a little more detailed, a key takeaway from the recently published paper is that AI is now hugely advanced, and capable of advancing itself. By having the ability to take general principles and refine them at speed based on new information is the reason why bots are now, in this instance, better poker players than humans.

Written by
Daniel Smyth
Dan Smyth is a poker media journeyman who politely reminds CardsChat readers that poker is played all around the world, not just America.


Jere Rivard wrote...

Dam BOTs !

scb1983 wrote...

The scariest thing about this post was that “…AI is now hugely advanced, and capable of advancing itself.” Computers and technology have an exponential curve towards advancement. This coupled with a ‘consciousness’ and ‘learning’ capabilities is extremely unsettling.

albosaltenio wrote...

impressive as artificial intelligence is avansando without limits, and how poker sites are going to have work to stop these bots. I imagine that there must be a lot of these bots working full time, With these things, little hope remains to be able to really progress in online poker.
It would be very good to understand the logic it use and be able to use it, Since it is part of the game and use that logic without machines I do not think it is illegal since it would be part of the player’s intellect to be able to solve those logic.

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Did you know about our poker forum?

Discuss all the latest poker news in the CardsChat forum