Dota 2 ai developments

How AI beat the best esports team in Dota 2

Expectations were great and tensions were high when the OpenAI Five Finals began in San Francisco. Team OG, who had lifted the world’s most covered esports prize by taking the no. 1 spot at The International, were confident thanks to their practice and skill.

Their opponent: OpenAI Five.

Artificial Intelligence has found its niche in several domains from science, medicine to live casino gaming. Take an example of the first basic poker AI software called Poker Bots which was created to compete in the World Series of Poker tournament. It had the capability to play against 6 human players. How cool is that! Similarly, OpenAI had been testing its Dota 2 AI for months. It had ‘practised’ with numerous real players behind doors and had even beaten the legendary Dota player Danil “Dendi” Ishutun at the Key Arena earlier. Fans and attendees were excited to see the AI in action, but what followed next was simply jaw-dropping.

The Competition

Dota 2 is a highly complex strategy game. It involves more than 100 unique characters, deep skill trees and item lists, and a dizzying number of interactions at any given moment in the match.

The game began with each side banning some heroes and picking from 17 available ones in order. OpenAI had also disabled summoning and illusion features because the system wasn’t trained for it, yet.

In the first match, OpenAI completely surprised OG and came out victorious by using several aggressive tactics. One move that nobody had anticipated was the decision to instantly revive heroes upon death early on in the match. It is considered a wrong move, but the AI’s decision paid off and gave it an edge OG simply could not overcome during the 30-minute plus match.

In the second match, OpenAI Five steamrolled OG with even more aggressive gameplay. It gained an early advantage in the first few minutes and then played off of it to win in barely over half the time it took to beat OG in the first game. Experts said the AI had started strong by picking four of the top five heroes ranked by net worth. But OG had also had some shortcomings as they were doing little to disrupt the AI across the map.

How did it achieve this?

As OpenAI put it, they didn’t teach the bot how to play; they taught it how to win. After that, the bot was meticulously trained over thousands of games of data; during the event, OpenAI revealed that the bot had had 45,000 human hours worth of training in the game.

Their training philosophy is straightforward too — virtual practice with as many variants as possible and only encourage the models that win. This means that even losses for the bot are wins and lead it to make moves that baffle players and viewers alike.

What’s next?

OpenAI confirmed it was ‘retiring’ its OpenAI Five team from any competitive matches after the event. But their main focus will remain on using the technology to find innovative applications in other areas of need.

Thanks to ‘Collaboration’, an OpenAI initiative, players can now play with and against bots to learn and explore their favourite game further. Regarding the future of Dota, there is no doubt that OpenAI completely changed the way the game will be played from now on.

Artificial Intelligence Beating Pros In Dota 2 Could Pose ‘Devastating’ Threat To Esports

AI and Dota 2

When AI developer called OpenAI – co-founded and chaired by Elon Musk – announced in mid-August that its bot had managed to beat some of the world’s best players in head-to-head match-ups, it sent shockwaves through the esports community.

Writing on the OpenAI blog, the team behind the bot said it had learned the game from scratch by self-play and did not use imitation learning.

“This is a step towards building AI systems which accomplish well-defined goals in messy, complicated situation involving real humans,” they told the world.

‘Devastating’ to esports?

Reacting to the developments, Daniel Sahl, associate director at the Center for Gaming Innovation at the University of Nevada Las Vegas, said AI had the potential to be “devastating” to esports competition.

“Computer assistance has always been a huge challenge to the viability of competitive video games,” he said. “It can be a real problem if, like in a game of Starcraft back in the day, people can see ‘the whole map’.”

While game developers can try their best to counter cheats, “it’s like an arms race.” “There will always be those looking to get an advantage and use game assists and AI is going nuclear on stuff like that.”

The OpenAI bot took on top Dota 2 players in best-of-three one-versus-one matches. Sahl drew comparisons with the effect that bots have had in online poker should AI become prevalent in esports games.

“Is online poker as enjoyable to play today as it was 10 or 15 years ago?” he questioned. “When you have all these people using programs to help them, playing ten hands at once, you have bots with auto-fold or auto-discard. How does that affect the player pool?”

Exponential improvements in Dota 2 AI

The OpenAI team points out in its blog that in the span of a month between early July and early August this year, the system went from barely matching a high-ranked player to beating the top pros.

It has continued to improve since then, due to the nature of the AI self-learning process:

March 1: OpenAI sees first “classical reinforcement learning” results in a simple Dota environment
May 8: Human tester reports getting better at the game than the bot
Early June: Bot beats novice human tester
June 30: Bot wins games against more experienced testers
July 8: “Barely” wins game against semi-pro tester
Aug. 7: Beats former pros ‘Blitz’, ‘Pajkatt’ and ‘CC&C’
Aug. 9: Beats top pro Arteezy
Aug. 10: Beats top 1v1 player Sumail – also plays Aug. 9 bot and wins 2-1
Aug. 11: Beats former world champion and “old-school favorite” Dendi 2-0 – bot also has 60 percent win rate vs. Aug. 10 bot

Sahl says bookmakers would be unlikely to be caught out in the initial stages of AI bots becoming prevalent in esports games.

“If you have a team that has figured out a way to use AI undetected, that will be reflected by their win rate” Sahl says.

“But the risk is much more severe if AI becomes too rampant, then the games will lose the player pool base,” he adds.

An ‘ocean of complexity’ in mutiplayer

What happens when whole teams of AI bots take to the games is a matter for conjecture. The rate of advance being made by OpenAI is likely to overtake the initial assumptions about its impact. As the team that built the bot explains, while 1v1 is “complicated,” a multiplayer 5v5 game is an “ocean of complexity.”

“We know we’ll need to further push the limits of AI in order to solve it,” they suggest, before pointing out exactly how they think they will tackle the task.

“One well-established place to start is with behavioural cloning,” says the blog posting.

Dota 2 has about a million public matches a day, and the replays for these matches are stored on Valve’s servers for two weeks. “We’ve been downloading every expert-level replay since last November, and have amassed a dataset of 5.8 million games (where each game is about 45 minutes with 10 humans),” the blog continues.

“If the publishers can’t stop OpenAI from interfering with their games, then there will be huge disruption in the popularity of competitive esports,” says Sahl. “The audiences derive from people feeling passionate about the gameplay. So if you see AI invade the space, it ruins the experience.”

The impact on betting

Traditional betting would presumably go the same way, but Sahl does see an upside for the games publishers – and by implication a potentially large role in the future for skin betting.

“OpenAI could be a real problem for competitive player-versus-player games, but at the same time AI will make the games more immersive, story-telling games far more compelling. People might not care whether they are playing against AI.”

He concludes: “The worlds we enter will become more real. We’re already seeing it with skin betting. The skins have a tangible market value.”

In this brave new world, esports betting operators will be in a better position than operators of traditional gambling activities.

“This means nothing good for casinos,” he forecasts. “When people have these virtual worlds, the thrill of winning at a casino might begin to pall.”