Dota 2 elder titan 6 867 5309

Thread: CMsgBotWorldState, Protobufs, and You

Thread Tools

Display

• Linear Mode
• Switch to Hybrid Mode
• Switch to Threaded Mode

I wrote a quick enum which you can include at the top of your. proto file to name all the abilities based on id. This enum is auto-generated by a quick python parser I wrote.

You also have to replace all the “uint32 ability_ .

Last edited by nostrademous; 09-26-2017 at 01:55 PM .

So for ML purposes, I’m thinking of doing slightly higher-level reinforcement learning than what was shown by OpenAI or some others by using the following information:

Now, all of the above is just my white-boardng of the world-state representation data I hope to provide to my machine learning implementation. The next thing is to now figure out a set of allowable actions we can take based on our observations of world state. These I white-boarded below.

The Fight Actions can probably use more flashing-out and review.

Thoughts from any of you that are much more well versed at ML and RL than I am. since I know nothing?

So I’m not an expert in the field, but I’d at least advice you to start with a much smaller and still much higher level action pool. Some actions would already require intense training on their own, such as the “Go to Location(X, Y,Z)”. Having it figure out a combination of X, Y,Z on top of figuring out that it should move at all is enough of a challenge that you could put a research team on it. Muddying the waters further by also allowing rarely necessary actions such as “lock combining” is going to massively increase the training time necessary. And I’m assuming you don’t have a supercomputer available. You’d be lucky if your system manages to play at the level of passive bots after years of training, unless you figure out some insane new ML technique.

Honestly, Dota is excessively difficult. Like, really, extremely difficult. I’d estimate that among all humans alive, given 500 hours of training, perhaps 5% would be able to play at the level of a 1k mmr player. A solid 50% of people would given a week and a computer be unable to install and launch the game, start a match, run to a lane and stay alive there for 5 minutes against nothing but creeps and towers. And these are humans, loaded with incredible brains full of life experience. They have the ability to adapt, reason, visually process the images from the screen, intuition and the capacity to create their own rewards.

If you’re going to start entirely from scratch using machine learning, you don’t get any of those for free. Neural nets are dumb, extremely so. The only reason they’ve come anywhere is because they’ve been given amazing training regiments. State of the art NNs are all given insane amounts of training time in terms of humans. And unless you come up with something new, you’re going to need that too. If you don’t have a supercomputer and the ability to run games on it, starting from scratch just isn’t going to work. The action space of dota is so huge, the volume of important content is so gigantic, proper rewards are so sparse and incomplete information is so difficult that no current NN architecture can be trained to competence on a simple home computer. Hell even DeepMind with all its resources is currently getting essentially nowhere trying to beat starcraft using machinelearning (and Dota is probably similar in terms of machinelearning difficulty)

If your goal is to educate yourself on ML or to try and contribute to machine learning research, have a blast. But if your goal here is to build a better bot, I’d strongly advice you to do one of three things:
– Build it the classical way
– Try to join OpenAI, a competitor, or at least type of company that gives you free supercomputer access
– Use machine learning only in very limited sections of your program

The last is probably the most interesting to talk about here. While working from scratch is probably impossible, integrating some learning in small sections should be doable. In fact you code up trivial ‘learning’ systems that merely calibrate some weights in your code pretty easily. A simple evolutionary system applied to some weight, like say something that determines to what degree it should prioritize lasthitting over denying, can be done by just randomly trying out a couple of options. Simply record the wins/losses and after a couple of hundred games, assuming the value actually impacts the game significantly, one or two values will probably stand out as the best.
To some degree it’s what evolution based neural nets are like. They’re just calibrating a boatload of tiny weights, which together play the entire game.
Now instead of one weight for lasthitting/denying, you’re probably going to want to do something that’s more fun, more impactful and which takes more data. Dota 2 elder titan 6 867 5309 Perhaps the decision making process around ‘when should I be aggressive’ is a good target. This is really hard to code manually, but a neural net may do that task pretty well, given the right inputs and enough training.
That kind of target is much more attainable than doing the entire thing from scratch. Outputting a single confidence float value is something NNS are pretty good at generally. You’ll also be able to select some obvious inputs such as health, herotype, distance, etc. The reward system will be relatively easy too, and the bots will actually engage in the interesting behavior reliably. A big problem with a lot of game based machinelearning projects is that the bots never even get far enough in the game for them to be able to stumble into the things that matter, if by pure chance a starcraft bot without coaching manages to build a barracks in one game, it’s not going to be more likely to win the game unless it also builds units and sends them to the enemy base. It might take it millions of games before the barracks building action contributes to a victory and billions before it realizes the two are related. According to the OpenAI reports, it took their bot ages (Some on the order of 100,000 – 10,000,000 games) to figure out that going to lane was a good idea. Dota 2 elder titan 6 867 5309
You’d be avoiding that kind of issue if you only work on limited domains like this.

So, tldr:
Don’t use anything even resembling the actions you picked unless you want to waste a couple of months.

Last edited by Siesta Guru; 10-12-2017 at 09:21 AM .