Procrastination
The main issue is my unyielding determination to procrastinate. Pretty much since the start, I intended to implement the universal "algorithm" to get good at any game. Alpha Zero. Or by now, Mu Zero.
The method requires no human input, just a computer program trying to beat itself indefinitely. Nothing but the game rules. Plus some machine learning magic. It's younger brother Mu Zero works even without rules.
Now, what did I actually do in those 6 months? I implemented the Monte Carlo Tree Search algorithm from scratch, wrote a simulation of the Splinterlands Battle Simulator. The first part is needed for Alpha Zero, the second is optional. But for some reason, I didn't program the rest. Probably because of my irrational distaste of neural networks.
But now I've started working on that part. Pretty much because I can't do anything else. Classic Temporal Difference Learning- doesn't work. Breadth-first initialisation - nope. RAVE - double nope. Thompson Sampling - comparable.
I hope I'll finish soonish because @wonsama with his Deep Reinforcement Learning approach plays pretty successfully. He did pretty good work on his bot.
Current Season
In the current season, Aicu performs as always. Somewhere between rank 110 and Rank 50.
Win streaks followed by loss streaks.
Some work out:
https://steemmonsters.com/?p=battle&id=5ff9d7f40ac52835c6631a18ff198bcc7f09cb6c&ref=aicu
Some matches are close:
https://steemmonsters.com?p=battle&id=b73020be7ab6fb4b7dab59b5e3eedb49a64d62f1&ref=aicu
Some just fail miserably:
https://steemmonsters.com/?p=battle&id=d75bee665c448829d70b7ed9c8093ab8d8587481&ref=aicu
Anyway, lot's of works to be done until rank 1.
Giveaway
Onto the giveaway:
 |
 |
 |
The giveaway works as usual. Comment down below with your Splinterlands username. And after seven days I'll draw the winners randomly.
Such an interesting project. I'm a bit afraid of the future capabilities of aicu now you've stopped procrastinating, lol. I suspect it's going to beat us all ;0)
@monstermother
Never underestimate the power of procrastination ;) especially if if kicks in again.
But for now aicu remains beatable as always.
And good luck in the giveaway :)
Posted using Partiko Android
I didn't realise @wonsama was also a bot. Did you find the new cards affect the AI at all? Interesting project, looking forward to future developments :)
He said so in the discord, and I've had a short chat with him about it :)
And the new cards created some issues. The main issue is, that with my algorithm the more cards are available the harder it gets to find a good solution. I had to find a way to deal with that. But now it works fine.
That's where a machine learning model would come in handy. It could estimate which cards to look at first for a given position. Especially neural networks work great for that.
But there's a speed tradeoff. Without a machine learning model it can look at more combinations than without. By doing that it can discover more promising teams by "accident", which it would ignore with the model. The model would help looking at promising cards first. Right now I added a simple statistical model to guide it. Because earlier experiments with ML models showed that most models are too slow. But I'm working on that. And Thanks :) are you taking part in the giveaway?
Posted using Partiko Android
Sure thing, of course I'll put my name in... username is @synrg... see 'you' in the arena!!! ;)
Great giveaway thanks! Wish I could code. It sound fun. Splinterlands username @gillianpearce.
I think learning to code is worthwhile :) but you need a high frustration tolerance. Thank you, and good luck :D
Posted using Partiko Android
It will be upvoted on by some friends and will be in the next edition of our newsletter. Congratulations on the great work.
Oh wow, thank you for picking my post :) where can I find your newsletter?
Posted using Partiko Android
@gratisuntamed
Good luck :)
Posted using Partiko Android
As always: Interesting updates, great giveaway!
@handtalk5
Posted using Partiko Android
Thanks, glad you like it :)
and good luck :)
Posted using Partiko Android
ugh, programming... I'd rather get a root canal again. More power to ya with your bot-work. I would have given up long ago. See ya in-game (same name as here)
That sounds like there a story behind you and programming :D
I think programming is interesting. Just like solving many ever changing puzzles.
Posted using Partiko Android
It's been a while since I checked in to see how the bot was getting on. You're still doing far better than I am.
I'll take cards off your hands - @welshstacker
I think it's mostly due to the deck :) with a high level deck you'd be playing in the same league as well :) and good luck in the giveaway !
Posted using Partiko Android
I was going to try and mimic Alpha Zero!
Well, then may the better implementation win :p
And: how is your Alpha Zero coming along?
Posted using Partiko Android
Very interesting to apply machine learning to all of this. Damn it, I'll have to start programming something if I want to climb in the top leagues with my rewards cards.
My user is @darthgexe, thanks for the giveaway.
Give it a go, Splinterlands is a fun playground for that.
It's simple enough, that many "traditional" techniques work, but complex enough to still get a benefit from more sophisticated approaches :)
And good luck in the giveaway :D
@yonilkar, hahaha your bot is crazy, new cards are frustrating him.
Bots are also just humans.... sort of :D good luck :)
Posted using Partiko Android
Hey there!
Even though I do not necessarily understand the hardcore tech bit of your posts it is definitely interesting to see what goes on with making a bot. I think already being able to hover around the 50 - 110 ranks is quite spectacular. Do you think it is the case with the last few ranks that stuff like artistry and finesse (the same difference with a player on twitch with 4k hours a year versus a "pro" doing 2500 hours a year) comes in and makes it that much more difficult to have a bot beat them?
@Rymlen for the giveaway :)
Hey, glad you enjoyed my article.
And regarding beating the top players: you're heading in the right direction with your theory. But there are two/three sides to the story.
One is, that people start to develop an intuition which moves work and which don't. That intuition comes from repetition and challenges. Then that intuition is combined with logical planning. And that's how you reach a team selection as a human.
My bot uses a logical approach without any intuition. It tries to learn an optimal behavior from scratch against an opponent by trial and error. That puts it at a disadvantage. Imagine you trying to learn a game withing just two minutes. That's what my bot is essentially doing.
The better approach would be to use intuition combined with that logical approach. That's what Google did with alpha zero. Their system played for weeks against itself and started to develop an intuition how to play.
They achieved that by training a machine learning model on the "discoveries" of the logical planning system and used the estimates to guide the planning. With that approach they beat a famous go champion and the leading chess engine stockfish. (Which already played much better than any human)
That system is what I'm going to work on next. With that I think beating aicu might become pretty difficult. It's still possible due to the probabilistic nature of the game. But it should be much harder.
Another aspect is, that there are a couple optimal teams for each ruleset /mana cap situation. They don't win every time, but they win frequently.
Just by using those consistently you can reach the top 50. Plus a max level deck. But that approach seems boring to me. That's why I've never really looked into it.
Another major one is a bit more technical, but it has to do with probabilities. Some matches can depend on a single successful dodge. And learning a optimal behavior for a non deterministic game is harder than for a deterministic game like go or chess. Especially within the time limit.
That's pretty much all there is to it. Good luck with the giveaway :D
Good work @bauloewe You are on a good way...
This bot is kicking my a... time to time... :)
Thanks :) it's still has a long way to go, because you're still kicking its a* as well :D
Posted using Partiko Android
Giveaway is now closed. I'll draw the winners later today. :)
Posted using Partiko Android
And the winners are drumroll:
Chain Golem: @tillysfamilyfarm
Darkest Mage: @darthgexe
Fire Elemental: @yonilkar
Posted using Partiko Android
Gee, thanks a lot for the card, I really appreciate it.
hi @bauolewe, please could you help me understand how the secret and team_hash fields are generated by sending the customJSON to the blockchain.
Hi @ydavgonzalez :) I do have an example in one of my github repositories for python. https://github.com/Bauloewe/SMBotFramework/blob/master/Requests.py
Check the submit team function. It's essentially an md5 hash over your team's uids and the secret. The secret is a string of 10 random ASCII characters and numbers.
Thank you very much, sorry for the delay in responding.
Congratulations @bauloewe! You received a personal award!
You can view your badges on your Steem Board and compare to others on the Steem Ranking
Do not miss the last post from @steemitboard:
Vote for @Steemitboard as a witness to get one more award and increased upvotes!