flapflip9 t1_ivkuf7s wrote on November 8, 2022 at 6:30 PM

Reply to comment by icosaplex in [Project] Rebel Poker AI by Character_Bluejay601

Wouldn't the gametree get too large to store on GPU memory for poker? Unless of course you start making abstractions and compromises to fit into hardware constraints. I used to rely on PioSolver (a commercially available Nash equilibrium solver) a lot in my younger years, a shallow stacked post-flop tree could maybe be squeezed into 64GB ram and could be computed in a few seconds. But the entirety of the game tree, with preflop gameplay.. my superstitious peasant brain is telling me you can't trim your model down to small enough size to make it work. On the flip side.. Given how crazy well these large NLP/CV models are doing, learning poker should be a breeze.

icosaplex t1_ivn5lha wrote on November 9, 2022 at 4:15 AM

Yep, it would be very large if you stored the entire game tree. But as I understand it, using a neural net in the right way, you don't have to any more, the same way that AlphaZero doesn't have to store the entire astronomically large game tree for Chess. Instead you rely on the neural net to learn and generalize across states.

Doing this in imperfect information games like Poker in a theoretically sound way (i.e. one that would converge to a true equilibrium in the limit of infinite model capacity and training time) obviously requires a lot more care, and plus you presumably also get the other practical challenges of neural function approximation - e.g. having to make sure it explores widely enough, doesn't overfit, etc. But it's still good enough apparently to be superhuman, and apparently if done right you can throw away practically all abstractions and just let the neural net learn on its own how to generalize between between all those states.