Monday, September 21, 2009

Designing Emergent AI, Part 5: Don't Squeeze a Handful of Sand

In the fourth installment of this series, I talked about the asymmetrical nature of the AI in AI War. That was intended to be the final article in the series, unless more questions came up that I needed to address. Recently a discussion arose on the Arcen Games forums, however, which I think really helps to illuminate an important point that I never managed to cover in this article series before now.

The Question, From User "dumpsterKEEPER"

I really like the AI behavior that allows for retreats when faced with overwhelming odds. On a number of occasions however, I've noticed this general scenario:

The AI has a strike force on one of my worlds when I jump a fleet in to defend it. As it's a sizeable defense force, the retreat logic kicks in and the AI splits off into multiple groups heading towards exit wormholes. One or more of these groups travel to an undefended wormhole and exit the system without a scratch. However, another group will head straight towards a turreted/defended wormhole and get completely wiped out in their attempt to leave the system.

I would suggest that when retreating, the AI should attempt to prioritize exit wormholes that are undefended or lightly defended. This would make sense with their retreating posture and would leave more AI ships alive to attack again in the future.

My initial response
I thought about this, but it leads to some undesirable ways to then further trick the AI -- leave a REALLY big force on the other side of the undefended wormhole, and then the AI pops out and gets slaughtered. This is one of those places (like the gap in the wall logic) that leads to too-predictable AI in other games. I agree, it will often make the AI act slightly non-ideal in this case, but it also protects it from being trapped into REALLY non-ideal situations, if that makes sense.

One of the forum moderators then suggested that I might consider simply having the AI probe undefended wormholes to see if it's safe. My response to that:

Probing undefended wormholes requires

A) time, during which the AI ships have to do something (and during which time they might well just be getting killed, or giving the human player time to come up with a counter)

B) a way to evaluate if the other side is safe (is it safe if the wormhole is clear, but there are massive fortifications on the other planet? what about if it is a long string of planets, and the danger is somewhere further down? do we get into situations where the AI just runs back and forth between some planets because it can't find an acceptable exit to punch through at either end?).

C) a lot of coordination between the AI ships, which isn't super compatible with the whole emergent AI thing.

This is one of those times where the emergent AI will make a slightly nonideal choice sometimes, but that slightly nonideal choice is actually better in the long term than always making the predictable 100% best choice. I intentionally avoid coding in hard rules like that, because as soon as the AI is too predictable, even if it is very smart, the players can formulate some second-order strategies to counter it. I know this because my play group (the AI War alpha group) is expert at finding these strategies in pretty much all RTS games. We never did find anything too exploitative for RoL or RoN thanks to the lack of walls, but we did for AoE2, AoE3, Empire Earth, SupCom, and all the various expansions -- and the SupCom AI mods, as well, though that took longer.

This really goes to the fundamental nature of the AI in AI War, and why it is in the main better. Will it sometimes do things like this that are tempting to "fix?" Yes. When there is a direct way to evaluate this sort of thing on a per-ship basis, I try to do that while also still making sure it remains fuzzy. A lot of the recent minor AI rules updates have been that sort of thing. But when something requires a lot of intentional ship coordination, or a lot of looking-ahead or scouting or what have you, that's where I start getting very nervous and staying away. The premise of this sort of AI is to stay away from those sort of things, because even though they fix the direct problem, they often cause a whole raft of other problems and exploits down the line...

More from dumpsterKEEPER
Ah, that makes sense. I hadn't thought about it from the potential exploits perspective. Thanks for explaining your thinking though, that's helpful to understand why the AI sometimes behaves the way it does. I can understand the hesitation to add specific rules to the AI as eventually you'd essentially end up with a decision tree AI.

In regards to this issue in particular, I don't think it's a huge deal, it just sometimes strikes me as odd that a group of AI ships will impale themselves on obviously placed defenses. On the other hand, occasional sub-optimal decisions do sometimes catch me by surprise and make the AI feel more "human."

My closing notes
No problem! Glad it's not too big an issue. And, for sure, sometimes those groups of ships will, in the process of impaling themselves, do something important. Or sometimes they'll have enough strength to break through into the adjoining planet and do some real damage on the other side. One of the players in my game on Saturday actually lost his home planet to something like that.

When I was working on the AI code to start out, originally everything was rules-based. And when I switched to a more emergent style of AI code, I thought I'd have to build in more rules there, too. That's why I was so surprised how quickly the AI started being effective, and from that stage on I became careful of second-order effects. I think that's the unique bit of knowledge that I stumbled across by accident, which leads to more effective AI designs in general.

Effective AI is like holding a handful of sand: some sand will trickle uselessly from your hand no matter what, this can't be prevented, but most game AI programmers squeeze the sand more tightly to try and save it, and wind up losing so much more sand in the process.

Next Time?
Once again, we come to the "end." Unless readers bring up more topics that they'd like me to address, this will probably be the final article in my AI series. So, I'm sure that another AI-focused article will come up at some point, in other words, but I don't have any idea when or what the specific subject matter will be. In the meantime, I do have other articles planned on other game-related subjects!

AI Article Index
Part 1 gives an overview of the AI approach being used, and its benefits.
Part 2 shows some LINQ code and discusses things such as danger levels.
Part 3 talks about the limitations of this approach.
Part 4 talks about the asymmetry of this AI design.
Part 5 is a transcript of a discussion about the desirability of slightly-nonideal emergent decisions versus too-predictable "perfect" decisions.
Part 6 talks about player agency versus AI agency, and why the gameplay of AI War is based around keeping the AI deviously reactive rather than ever fully giving it the "tempo."

Wednesday, September 16, 2009

Why Rebalance A Released Game?

The main reason for making these changes (and balance changes in general, you don't have to click those links to understand this article) in AI War is to increase the opportunity cost for a number of actions, and to provide a more rich set of strategic options in general. Some expert players had concerns with the above linked changes that they would basically reduce the viability of certain advanced strategies, but the second link demonstrates why I feel like the strategies in question are all still very valid (but no longer abusable). However, these come with some cost.

But for purposes of this blog article, this isn't a discussion of any specific changes, but is rather a discussion of what makes for good changes versus bad changes. Some external commentators have noted that I am "faffing" about with game balance, which I take great offense to -- I know exactly what I am doing, in terms of my long-term goals, and my actions have all been purposeful and productive. I'm building a longterm game environment, of the sort you normally only see in MMOs. This requires some ongoing thought and commentary from expert players who find tricky ways to abuse the mechanics. But I'm getting ahead of myself.

Why Nerf Strategies?
When an advanced strategy comes up that I feel like is too exploitative of subtle unit interactions or too abusive of the game rules, I often will add a counter (either in AI logic, or in the game rules/mechanics themselves) to counter this. To some, on the surface this seems counter to the goal of having a strategically rich environment. Is it my goal to have all players playing the game exactly the same way, with minor variations? Of course not.

In general, the only reason I ever nerf a given strategy is if it gives too great a benefit at too low of a cost. There have been some really challenging issues of late with players taking too few planets and doing all sorts of clever things, which really causes the AI to be less effective and lowers the difficulty in an artificial way. My response to this has been partly to teach the AI some new behaviorlets, and partly to reduce the benefits and increase the costs for these more esoteric strategies.

Preserve A Rich Decision Space
When I look at AI War, or any game for that matter, the main thing I am looking at is the "decision space." When a single strategy or group of strategies are too effective, the decision space effectively shrinks because expert players would be fools to use any other strategy. This becomes a failing of the game which I have to address through balance updates and new/updated game mechanics in some cases. Individual ship balance is only the beginning, because how players use all the myriad types of ships in concert, plus how they plan their overall strategy, can have an even more complicated effect on game balance.

My goal is not to make all strategies exactly equal (because then the decision space is shrunk by nature of the fact that any strategy is as good as the next, so it doesn't really matter what you do). Having no interesting deviations in strategy is just as much of a game-killer as having one best strategy is.

Instead, my goal is to make strategies that are generally all within a standard deviation of one another, so that players with different playstyles can play as they wish, but also which are context-specific to a degree, so that the truly expert players will adjust their strategy very heavily depending on the specific circumstances of a given scenario. This not only adds to the richness of the strategy of the game, it adds to the replay value.

Of course, when players play below their true difficulty level, they have more latitude to just use their favorite strategy and have done with it. But when things are really neck-and-neck, players should have to make appropriate evaluations of the map and act accordingly, rather than being able to artificially lower the difficulty through exploitative tactics.

Balancing For Very Long-Term Play
Will this annoy some players who rely on these tactics to play at a higher difficulty level? Of course it will, and that is an unfortunate side effect. Any balance shift in any RTS game seems to annoy someone, while (hopefully) the majority rejoice. You might assume that because AI War is not a competitive pvp affair that these sorts of balance issues are not important. To a certain extent this is true, it is certainly much less important that the unit balance be perfect because of a number of facets of the AI War design. However, the overall strategic balance is critical for the longevity of the game.

When I play any RTS game, I am going so solo or co-op against the AI in skirmish mode. That's the only way I play. I basically can get 6-12 months of biweekly play out of most of the better RTS games, and that's the point at which I get bored with the game because I have figured out some sort of killer best strategy that the AI can't counter and that I can't top. At that point there's no other way that I really want to play the game, and I've lost interest playing the game using that best strategy, so there's pretty much nothing left for me to do with the game and I move on.

That's all well and good if you are trying to sell a huge series of RTS games, but with AI War I intend to grow and build it as a series of expansions, not sequels. That means that the core game had better be extraordinarily rock solid, with absolutely no best-paths that people discover after however many months of play. There are always tricky things that players figure out, of course, and so that makes an ongoing balance load for me. This is not unexpected -- Starcraft is still getting balance patches some 11 years after its release, from what I hear, and it is regarded as supremely well balanced.

As with the Starcraft balance updates, my goal is not to quash player innovation -- I applaud it. However, my goal is to keep all strategies within essentially a single standard deviation of the norm, and also to add as much context-sensitivity to the grand strategies as possible. The kiss of death for an RTS game, in my opinion, is when all the games start feeling basically the same to expert players. That's when it's time to move on and find a new game to play. My goal is to keep that from ever happening with AI War, because that's the only way I'll maintain my own interest in the game, let alone the interest of anyone else.

That sort of outlook will annoy a few players as I go, unfortunately -- and I need to be very careful to listen to player feedback and not do something that pisses people off for no reason, or which is hated by a majority of the playerbase. In general I'm pretty averse to doing things that players don't like, which I think is a crucial attitude for game designers to have (just "doing your own thing" or having a "take it or leave it" attitude is stupid and is suicide). However, it's impossible to please everyone when making any given change, and so player feedback has to be weighed against the longterm health of the game. Rebalancing a game that has already been released is always a tricky proposition, but you only have to look at examples such as Starcraft or World of Warcraft to see how incredible the results can be in the long term if care is taken.

Tuesday, September 8, 2009

Monday Depression In A Job You Love

I'm not a doctor, obviously, and I'm not an expert on the human mind or all the various chemical and psychological processes that are ongoing in our bodies and brains. However, like a lot of people, I've noticed that humans in general seem to be pretty cyclical in many of our moods. Obviously this is the most pronounced in manic depressive / bipolar disorder patients, but from observation it seems like they just have a (much) more extreme version of what every person goes through to some greater or lesser degree. I think that OCD patients are also just very heavily exaggerated symptoms of some of the underlying impulses that we all have.

Why am I bringing this up? Well, because it's the first day of the week, and I'm dealing with my usual bout of depression that brings. Today is Tuesday, but yesterday was a holiday in the US so I wasn't working. Usually the depression hits me on Mondays, but since I was off it is instead hitting me today. In the movie Office Space, one of my favorite films, they refer to this as "A case of the Mondays," and that seems about right to me. Certainly when I was working in a physical office, you could notice a distinct atmosphere difference on Monday mornings in all the staff, even those who intensely loved what they do.

So what gives? What on earth do I have to be depressed about right now, for instance? Things are going pretty well overall, I'm mostly doing what I want to be doing and seem to be moving in the right direction for being able to do it fulltime, and I've just had some really happy news in my personal life this morning. Plus, even just yesterday, I was excited about the work I would get to do today. Why on earth should I instead be feeling like things are so bad, and so much is wrong with the world?

Oh -- wait, are you waiting for me to give some big reveal as to why that is? Sorry to disappoint, I have no freaking clue. I'm sure a psychologist or a neurologist could give a better answer. All I know is that it happens, and I don't have clinical depression or anything (the rest of the week I'm generally fine, except when something happens that would make anyone a bit down for a few hours). So the question, for me, is not why this happens but what to do about it.

I had thought that if I was more into my work, that I would not have this trouble. But, clearly that's not the cure because I get this even when I'm working on game stuff right on Monday morning. I think it might have to do with the transition. Even if I'm looking forward to the work, getting back into the work can be a bit daunting. You start staring at that huge forest, rather than just the individual trees, and it can be depressing just how much there is to do.

It might have to do with having too many options and too many decisions to make, I don't know. Normally during the week, I have a short list of items that I am going to hit next. Sometimes new items get slotted into that list ahead of the original items, but that's no bother -- you just roll with it. But somehow at the start of the week it feels like there are more options.

Maybe that's what it is, I don't know. The solution, for me, is simply to start small. Just work on some little thing, important or not, that I can finish in 5-10 minutes. There are plenty of tweaks of that size in my queue. That usually gets me going, but if it doesn't then I just do a few more of those. Within an hour or so, I've completed several small tasks, am in the right mindset, and am ready to move on with the rest of my bigger tasks. The feeling of being slightly off then tends to persist for a few hours, sometimes the rest of the day, but after that initial ramp-up period I'm back in the saddle and at least not hampered by it. And the rest of the week, I'm perfectly fine.

This is perhaps a bit of an odd topic to touch on, especially considering my usual subject matter, but I think this is an important issue. At first this cyclical start-of-the-week depression had me worried that something was wrong, that maybe this was not the right profession for me after all (despite all of the evidence to the contrary). I don't know how many other game designers also have this problem. I think it's a common thing amongst people in general, but especially the "creative types." I'm convinced it doesn't mean anything, it's just some sort of biological problem such as needing sleep/food, or having low blood sugar or what have you. Do what you need to do to move past it as quickly as possible, so that it doesn't cost you a whole day of productivity (who can afford to lose 1/5 of their work week every week to this phenomenon?), and don't read too much into it.

The social tendency to not whine and talk about things like this (just "suck it up") is unfortunate, I think, because it tends to make people feel very alone when they have this sort of issue. But, anecdotally, it seems to be pretty universal to me. Most people just assume it is because there is actually something wrong, such as a job they don't love. That makes people who normally love their job worry even more about things like this. To that I say: let it go. There is some sort of process going on inside us that neither you nor I likely fully understand, but it doesn't mean anything. "Looks like someone has a case of the Mondays" is, for whatever reason, not limited to people who hate their jobs. Once you realize that, the phenomenon is not nearly so alarming.

Wednesday, September 2, 2009

On Writing For Games

A few players and reviewers have commented on the fact that AI War does not have much lore, or much story to it. Some players see this as a detriment, or at the very least as a noncritical oversight that they wish I would fix. Most players don't care either way, of course, and some have even made comments to the effect that they are happy they don't have to wade through a bunch of backstory just to play the game.

Given the above, you might assume that my motivation in having such a limited amount of story (15 lines of intro on the main menu) was simply for time-saving reasons since the majority of players don't care. Or you might assume that I don't like to read or write. In fact I love to write, and I'm an avid reader, so the truth is a lot more complicated.

It comes down to what I'm trying to accomplish with the game. I really do believe that Less is More in the case of certain games. I don't want more story than what your average Zelda game gives, for instance. I very much enjoyed the story in Super Mario RPG, but I find the stories in a lot of the Paper Mario games to be too longwinded. I enjoy Final Fantasy but find most western RPGs to be too longwinded. Etc.

In so many games, there is an abundance of story that is not all that original, not all that interesting, and kind of redundant and bloated in its execution. With some good cutting (and I mean cutting over 50%), you'd arrive at something more the quality of a novel, which I think is important. I prefer reading over television, and I've been an avid writer my whole life even though I never quite made it into print, so this isn't just the opinion of someone who doesn't want to read.

As a great example of Less Is More, you have Braid. A lot of people criticize the little story bits as being unintelligible or whatever else, but I thought it was well done and poignant. And with a lot of the older NES games, where text space was limited, a very limited amount of text was able to convey so much. Zelda 2 is absolutely one of my favorite games, partly just because of the atmosphere of exploration and mystery that is there. With pages and pages of text, I think it would have been a much less compelling experience.

As an example from film, I think that in a lot of movies it is important to not show the monster. Look at all of Hitchcock's films. Less really is more in those cases, because not knowing, not seeing, is what makes it significant. I liked the M. Night movie Signs, but the worst thing in that was when they showed the aliens. Bad, bad idea. The original Star Wars movies were so much better than the later ones also partly because they were more restrained in their storytelling, and followed the "show don't tell" rule (amongst so many other things they did better, but I digress).

Looking around at other mediums, take the comic strip Calvin and Hobbes. There is something called "The Noodle Incident" that Watterson never explains, though it is referred to several times. In one of his books about the strip, he talked about how that was intentional -- any prosaic thing he could come up with could never compare to the crazy half-formed ideas that immediately spring to mind in all the readers for what this "noodle incident" could possibly be.

Sometimes it is good to be explicit. Dialogue, characters, etc, are important in some games. Other times it is better to be more subtle. Show, don't tell -- this is a chief rule of good writing, but not all game writers bother to follow it. Games are a visual medium, and like with great movies, great games don't always have to have a lot of talking. The lore is right in front of you, staring you in the face. You are playing it, creating new lore as you do so. The half-formed stories in your head are vastly superior to the stories that you or I or anyone could actually put down in the game itself to make people read. When you put it on paper, it's showing the monster.

Some movies need to show the monster. Some games need to have a lot of text. But I think an important skill of a disciplined writer is to know when to write and when to hang it up. An amateur writer will always spill out pages and pages of narrative for even the slightest prompt, never pausing to think how original or interesting it really is. This is what you get in a lot of games, and I'm so against that. Ico is a game that is immensely evocative, and tells a wonderful story with hardly any dialogue at all. The latest Prince of Persia game has rafts of snappy dialogue between Elika and the Prince, but it's mostly pretty forgettable. They have some great lines in there, and if they had cut down to just those it could have been much more poignant, but as it was the Prince and Elika could ramble on for what felt like hours.

I guess I subscribe more to an older school of thought, which is not in vogue right now. Suspense movies have been supplanted by horror movies, after all, and I feel like that's a crying shame. A lot of remakes of older movies are actually inferior to the originals because they forget to show not tell.

In the end, what does this really mean? Does this mean I will look down on anyone who wants more story, or who wants to create fanfiction? Of course not. There are loads of games and stories where I want more, and where I considered writing fanfiction (or did write it). However, something I have realized through my trials and tribulations as a writer is that this very state of "wanting more" is in fact a good thing. Leaving the reader loving what was there but wanting more is where you want them -- if you satisfy every last craving for information about the universe of the story, you wind up with a lot of the more modern book series that I am not as much a fan of. It cheapens them.

Some people will disagree with that assessment, and will always clamor for more, or write fanfiction. That's cool with me. Just don't expect me to add to the official cannon for AI War beyond the minimum needed to create and maintain atmosphere. And the bulk of that atmosphere, as in the 8-bit classics of yore, is created via gameplay and gameplay alone. Will every game I code be like this? No. I intend to write some very story-heavy RPGs in the future. But I'm quite proud of how this one turned out.