by Jason Parker (Ragachak)
This is something I wanted to talk about in brief about a month ago, but the last month was very hard for me, mentally and physically. But, it was a pretty good month for gaming. In particular, Race for the Galaxy received another expansion, Brink of War. I reviewed the game earlier last year, and as a near-perfect replication of one of the best strategy/card games you can play physically, what makes this version stand out, is that you can play alone! You don’t have to find another friend who plays to get a good match in (but you should. You should get people playing it if it’s your bag. It’s incredible). You can play on Steam, Mobile Device, or Tablet, and you’ll have the same incredible experience on all of them. But before I get into the meat of it, let’s talk about the latest expansion too, because it’s worth noting.
The Brink of War is the latest expansion, and as its name suggests, the galaxy is getting a little rowdier. Prestige is key and is another way to gain the victory of the PC or your friends. It includes thirty-six new developments to build and worlds to settle, new goals and a new action in “Search”. Need that one card that can seal doom for your opponent? Search just might be the command you need in order to get going. I highly recommend it for this, for cross-play, and insanely deep tactical gameplay that you can normally only receive if you and your friends are all geniuses at this game. Replayability is so key though, and having a complex, powerful AI can give players a new experience every single time.
One of the things I found interesting, was when I was doing some reading on their AI system, is that their system (temporal difference neural network), requires no human input to generate training data. This means it doesn’t require a gigantic expensive team to make it work, and that this small team of dedicated fans of the boardgame can make it happen on their own with limited resources. This is not their first game designed this way though; they started with Ascension, using a heuristic-driven AI. The AI will favor moves that they deem superior. EG: It will buy cards with high VP-to-cost ratios, or trashes low-value militia cards before others. This, however, was designed by people, and it’s based on what they think the best plays are. This requires the people who developed the game to either be experts at it or to seek out those who truly are the best at Race for the Galaxy. This AI is smart, but people will get one over on it reliably eventually. It’s still far more enjoyable to beat real people, but you can still practice and get better at the game with the AI and not feel like it’s a disappointment. Theresa Duringer had this to say about their AI network:
“Instead, it learns by playing randomly, making predictions at the turn level on which player is winning, and updating the weights in its multilayer perceptron architecture such that the delta between predictions from one turn to the next is diminished. Using this method on over 30,000 training games, it’s learned the black box function that best represents the relationship between input (the state of the game) and output (prediction of who’s winning) for the neural network that drives our AI. It’s a pretty incredible method, first used for boardgames by Gerald Tesauro who created TD Gammon and now applied to Race for the Galaxy by our AI expert, Keldon Jones.”
I think it’s absolutely fascinating that this AI system is designed around a 5,000 year old game: Backgammon. Gerald Tesauro developed TD Gammon, and it was one of the most cutting-edge games of all time possibly. This AI system was designed around this, and Tesauro’s work changed how Backgammon is/should be played. It doesn’t come with pre-conceived notions of what is a smart play and instead has zero initial knowledge. Basically, the TD Gammon system is teacherless, which is interesting because after all, human error is not really that uncommon. Letting the engine determine what is smart and what will ultimately have a higher win rate.
I will spare you the mathematics, but the details can be found here, for those interested. With that in mind, this AI system for Race for the Galaxy only needs to know the “rules” of the game. It will make determinations from there. Over thousands of training games, it develops strategies and tactics such as “Piggy Backing”. To explain from Theresa, “For example, here the AI chose Trade, even though it has no goods to trade. I fell into the trap, chose settle, the AI can settle a world that comes with a good, effectively piggybacking my settle, and trade it. The tactics employed can be nuanced and context-sensitive, unlike a heuristic driven AI.”
Why do I think this is so interesting? When I first started playing Race for the Galaxy, I lost to the AI almost every time. I was getting frustrated, because I had not played the actual game in years, and was more than rusty. I was losing all of the time because the AI is incredibly smart. But they did add some noise to this AI system for new players, so it won’t always make the best, smartest choices. You can adjust the difficulty, but they all feel suitably challenging. The bottom line for me, is that the AI is smart enough that you can get tons of replayability out of it, being forced to change strategies and tactics against the AI. In lots of strategy games, simply playing against the AI is not rewarding or fun, because it’s usually too easy. They’re still working on new technology and strategies to make to improve performance and require fewer neural network evaluations a turn (instead of 500,000+). Do you love strategy games that will challenge you? This is the one you need.