Adaptive Artificial Intelligence for Zombies

[melancholymastermind 33]

I have spent years researching and creating artificial intelligence for autonomous robots and for mods and indie games.

 

I would love to see adaptive AI being applied to the zombies in this game to keep them fresh and a challenge as the game gets older.

 

I am happy to share any adaptive AI concepts with you if it something that interests you for the game and not already on the table.

[Twitchy040 2]

how would you figure the zombies would/could adapt?

Edited January 10, 2014 by -Scootaloo-

[enforcer1975 1111]

How can brainless adapt? I mean that's wgat distinguishes zombie games from cod....ok nie that i think about it the cod ai acts just the same...

[Twitchy040 2]

How can brainless adapt? 

 thats exactly what i was wondering, popculture sees a zombie as a mindless shell of a human focused on only one thing, feeding.

so i was interested to see his take on how they could,would adapt

[Publik 404]

I have spent years researching and creating artificial intelligence for autonomous robots and for mods and indie games.

 

I would love to see adaptive AI being applied to the zombies in this game to keep them fresh and a challenge as the game gets older.

 

I am happy to share any adaptive AI concepts with you if it something that interests you for the game and not already on the table.

I'd be interested in stuff you know in general.

[melancholymastermind 33]

Sorry about that everyone I was at work when I started this but Im home now and have time to explain properly.  In hindsight yes initially it sounds silly to have intelligence in zombies.  And that is not really what I am talking about.  Think more along the lines of a swarm of ants.

 

zombies are so mindless that the decisions on where they move and what they do without player interaction is most likely controlled through group based artificial intelligence that has a single ai unit controlling a large number of zombies (even all of them).

 

There are many scenes from almost all zombie movies that hint at a possible either new intelligence or remnants of the old intelligence still residing in the zombie.  In the very first episode of "the walking dead" the little girl zombie stops to pick up a teddy bear and keeps it.  Later on in the series that guy with his son in the house.  The wife zombie walks up to the door of her old house and attempts to turn the handle.   You see things like this all the time.

 

But regardless to whether they are have smarts or not, I am talking about adding adaptive choices to the artificial intelligence that controls how the entire zombie population or large hordes of zombies act in the game world, rather than the individual zombie.

 

Note : I am still editing this post with more information, but I wanted to answer that now

[Rufert 146]

I very much like the idea, but considering how wonky and limited arma is for AI especially melee AI, I don't really see how any advanced stuff would be possible without deserving the server.

[DropBearChick 1216]

Maybe they get hot so they steal your hats or bored standing around so they head to the nearest cinema or library

[Twitchy040 2]

the idea of a wandering horde in SA would be great and i know that ( without searching) it has been suggested before would be a great idea. add a new element to those mindless runs between towns inland. any incarnation of that would have my full support. i just misunderstood the post it seems and apologise for my earlyer sceptasisum

[mike miller 14]

I have spent years researching and creating artificial intelligence for autonomous robots and for mods and indie games.

 

I would love to see adaptive AI being applied to the zombies in this game to keep them fresh and a challenge as the game gets older.

 

I am happy to share any adaptive AI concepts with you if it something that interests you for the game and not already on the table.

 

you should try to contact Rocket or Bohemia since DayZ is selling very successfully they might give you a project or two ;)

[SunTechNick 3]

Hope you join Rocket's team. But i realy doubt that it can be implemented in this engine. 

[Monkfish (DayZ) 339]

I very much like the idea, but considering how wonky and limited arma is for AI especially melee AI, I don't really see how any advanced stuff would be possible without deserving the server.

The zombie AI is being rewritten from the ground up. The current AI is just a placeholder as the new AI isn't done yet.

[thomas-tk@live.dk 39]

Since I think they are considered infected and not "braindead" zombies, I suppose they could adapt to new circumstancse. 
Deends where Rocket and histeam is going in regards to the infected/zeds/zombies/ those guys who wants to kill you but arent players

[ambiguousFoo 12]

This can be seen in hordes.  Consider as the zombie apocalypse drags on, individual zombie encounters will become less prevalent, while zombies that migrate into hordes will be 'more successful'.

[melancholymastermind 33]

Sorry about that everyone, again.    I was oncall and would you believe it got a call. So Im posting a reply instead of updating.

 

 

I did not create this thread to just ask the devs to add adaptive AI into the game (which I hope they do whichever established or personal route they take) but to adapt this new AI concept I have developed in particular into the game.

 

this could get confusing but its futile to explain in depth everything behind this now without diagrams to assist. I advise to read slowly

 

This artificial intelligence works on a 3 layer structure designed to maximize complexity while minimising processing time.  The Sony Aibo robot I own was released 13 years ago and contains a risc processor that runs at 50mhz!  In other words very little processing power is required to implement this particular concept into a game and favours having a "dumb" AI already in place to directly control the zombies.  I favour the modular approach to coding and having looked into ARMA modding before the coding is similar (I am using that loosely) to OPEN-R for the Aibo.  In other words Im confident it will work in DayZ if they chose to implement it.

 

Firstly before I even get into the AI itself I need to cover the adaptive part so that you understand how it fits together.

Each level of the AI has functions that either control the ai level below them, or the current AI in the game for the lowest level.

Each function has a probability rate, that decides how likely it will be "randomly" chosen to be in a queue for its level.

Each function has requirements that dictate success or failure. This is used to alter the functions probability rate up or down accordingly.

For example a function for moving towards a target entity would have the condition that the group became closer to the target entity to be successful.

If a function needs a target but the ai has none, that function would fail without being executed. Most functions would have some value/limit or condition that would cause them to immediately fail, but not all have to.

Each function controls the below AI by having one or more preset queue's of lower level function(s) that will successfully complete this functions requirements. (that's a mouthful)

Each queue has a success/failure ratio stored and modified when the chosen queue's last lower level function succeeds or fails. Preset queues start with 100% success rate as they are set by the programmer.

Finally each function effects any number of the monitored statistics for succeeding and/or failing.

 

If the AI is particularly "content" (doing whatever it is designed to be doing in other words) it has a very small chance of creating a "mutated" queue in the function that just completed. Mutation is simply copying the queue and randomly replacing one lower level function in the queue with another randomly chosen one. That's where the probability rate comes in.  all functions have a chance of being chosen, regardless of probability rate, but it is more likely to choose a function with a high probability rate than one with a low probability rate. 

The new queue has success/failure rate set to 50%.

Most of the mutations will be abysmal failures and quickly the ratio will drop extremely low. 

So if there are more mutated queues than a preset number, when the function that queue belongs to fails the queue is checked to see if it has the lowest ratio among mutated queues and is removed if it does.

Conditions for a function to even start are key to failing a queue before taking any action and waisting ai time but they are not required.  There is more chance of seeing the AI do seemingly idiotic and completely unrelated actions to a situation if requirements are not set.  This can lead to some interesting and probably hilarious outcomes however so like I said not all functions need requirements.

 

 

Levels of the AI

 

Top Level - Emotional/Physical state reactive ai

This AI monitors and alters things like desires (how badly the zombies want brains for example), excitement (blood or death in nearby), stamina (if they have it), company ( used to help grouping or mega hordes), being attacked by players and how many etc. 

Its an averaging of the units under its control which would preferably be a single horde but could be all at once.

It would react off already in-place functions that return states of zombies or positions and other data already provided by the ARMA engine.

Finally this AI also controls the single goal (top level function) of the group. This might be "move to city" "move to forest" or of course "find flesh to eat".

Once a goal is started it remains until the middle level AI has completed its queue, or an event occurs that causes a new goal (such as group members being killed or one finds a player)

 

Middle Level AI - Activities

This level of AI takes the queue chosen by the top level AI and evaluates the functions in turn to see if they are valid (required target type exists, not too full, not too hurt etc.) and if they are passes each function in the queue in turn to the lower level AI for execution.

It waits for the low level AI to return that it has either succeeded or failed before moving on the to next function in the queue. If the low level function fails it immediately reports to the top level AI that it failed and does not execute further functions from the queue.

Middle Level Functions would include things like "move towards target"  "circle target" "wonder around" "eat target" "group together" etc

 

Low Level AI - Actions

Things are most complex here for the AI.  The functions that are passed to this AI will directly control what the zombies do in the game world.

This AI's purpose is to evaluate the function passed by the middle level to see if it can be executed (like before) and then execute it.

Functions at this level would be things like "walk to position" "crawl to position" "run to position" "stand attack" "crouch grab" "push" all the individual actions a zombie is capable of doing.  The low level function contains the code that uses the top level AI's current state to decided which zombies under its control are ordered to do what.  They are almost exclusively timer based that gives the AI in game its finite order then waits, evaluates and then reorders until the conditions for success or failure are met.

So for example moving towards a city or forest would likely want the whole horde moving, but attacking the player may only want to initially attack with the closest few zombies.

 

 

To see it as a whole think of it like an organisation chart.

At the top you have Top Level Ai functions or Goals.

Below them are all the Middle Level Functions or Activities.

Finally across the bottom are the Low Level Functions or Actions.

 

As the AI runs, it begins by following the preset queues created for it.  These will be generally successful as they are designed by the programmer to work. As time passes and the AI completes actions and activities and goals the probability rates of the various functions fluctuate according to what is happening in the world, including player interaction (a valid activity could easily fail thanks to a player killing some of the zombies in the group for example). This will change which functions are chosen by the AI.  The ever changing inputs of a live server gameworld and player interactions would also help to allow mutated queues to succeed that would otherwise fail in a test. Mutation and Evaluation of these queues will create simple adaptive emergent behaviour from the AI.

 

The speed at which new behaviours emerge and are tested is very slow, and could require hundreds of hours for non preset actions to start actually occurring (at the choice of the programmer of course, fast learning is possible with stricter conditions on functions)

 

 

Finally before posting this mammoth thing I will give an example

 

There is a zombie horde in a lucky position where regularly bambies come along and get eaten.  This keeps the AI content as they are eating regularly and not getting hurt (silly bambi's).  Every time a player comes near or attacks a zombie with an axe/weapon the goal is switched from whatever it is now to attacking back and trying to eat them.  As this happens over and over (silly silly bambi's) a mutated queue is created for the attack player functions (hey its happy) which has the AI run to the forest before the attacking the player. Due to their current world position this works, and is valid and succeeds multiple times.

Later on other players who are facing a horde may see that instead of heading towards them as normal, this horde reaches them by heading towards the forest and then towards them.  The more it succeeds the more it happens.

On the flipside if you are fortified somewhere and a huge hordes comes down on you but you hold them off, this will cause the functions in use to fail, lowering the chance they will try that again either now or later on.

 

 

It is entirely posssible that this entire thing only makes sense to me, and Im sorry if thats true LOL

Edited January 11, 2014 by TemplarGFX

[Damnyourdeadman 1045]

Reallity is that i don't know a thing about the AI in Arma.But you explained in much detail-debth of the engine's functions.

Seeing as this,is a big project,i can't help myself feel intrueged by it.If you are positive that you are willing to see it implemented,have the skills and you are up for the task,you should totally contact the team.Regarding myself i can only give you my  :beans:  and a  :thumbsup:

[Publik 404]

I've got a game I'm working on with an enemy AI that functions similarly but without the layers or learning.

 

Basically, roll a d4. If you get 1, attack the lowest HP target.

If you don't get a 1, roll another d4 and if you get 1 attack the target with the least armor.

If you still don't get a 1, attack a random target.

 

With OP's method, the die rolled changes to have fewer sides when the previous rolls succeed, and the die gains sides when the previous roll fails. Whatever works gets more likely, whatever doesn't work gets less likely. Additionally, there are different layers the AI works on. At the bottom are basic movements, like "move forward", "move left", "crawl forward", etc. The middle section is made of these basic movements and represent activities, like "go to player 1", which is made up of "move forward" and so on. The top level is for goals, like "kill player 1", which includes the middle level commands "go to player 1" and "attack player 1". Another example is a survivor behind a door. The "kill player 1" goal might fail, because the player is behind a door. The zombies would eventually learn that when the "kill player 1" goal fails (because "go to player 1" fails), they need to do something else, for example adding "break door" to the front of the goal ("break door" -> "go to player 1" -> "kill player 1").

 

Is that about right?

 

Would they do better if the system doesn't evolve and the initial percentages are determined in closed simulations? The only negative thing I could think of is that this assumes that zombies are relatively permanent/never despawn, which might not match the server restart pattern that's currently here, and if the servers are so put out by a few hundred zombies how are they going to get a system like this in?

[Gezus 5]

I actually "sorta" like this Idea.. Maybe not ALL zeds.. but maybe some.. 

 

This is not an unheard of experiment in zombie culture. In fact George A. Romero's "Land of the Dead" is a movie that explores the idea of regardless of "Dead".. perhaps ALL things "learn" and "adapt" to their surroundings. 

 

Also .. for the record. The last time I checked.. the zeds in DayZ are NOT walking corpses.. they are in fact "infected" humans.. 

 

Link for reference.

http://www.imdb.com/title/tt0418819/?ref_=nm_knf_i3

Edited January 11, 2014 by Gezus

[melancholymastermind 33]

 

 

I actually "sorta" like this Idea.. Maybe not ALL zeds.. but maybe some.. 

 

This is not an unheard of experiment in zombie culture. In fact George A. Romero's "Land of the Dead" is a movie that explores the idea of regardless of "Dead".. perhaps ALL things "learn" and "adapt" to their surroundings. 

 

Also .. for the record. The last time I checked.. the zeds in DayZ are NOT walking corpses.. they are in fact "infected" humans.. 

 

Link for reference.

http://www.imdb.com/title/tt0418819/?ref_=nm_knf_i3

 

 

 

This particular approach should not really be thought of as any form of intelligence in zombies, The AI concept here applied the way I am thinking would instead give you a game where after yo play for 100 hours and think you have seen it all and know all the exploits,  the AI will come out with slight alterations to its most successful actions which it tests and gains confidence in over time.   This happens over and over theoretically constantly showing adjusting behaviours to keep it fresh.

 

And think of adding the public hive into the equation and having all zombies share the top mutations across all servers. That would be very interesting

 

 

 

I've got a game I'm working on with an enemy AI that functions similarly but without the layers or learning.

 

Basically, roll a d4. If you get 1, attack the lowest HP target.

If you don't get a 1, roll another d4 and if you get 1 attack the target with the least armor.

If you still don't get a 1, attack a random target.

 

With OP's method, the die rolled changes to have fewer sides when the previous rolls succeed, and the die gains sides when the previous roll fails. Whatever works gets more likely, whatever doesn't work gets less likely. Additionally, there are different layers the AI works on. At the bottom are basic movements, like "move forward", "move left", "crawl forward", etc. The middle section is made of these basic movements and represent activities, like "go to player 1", which is made up of "move forward" and so on. The top level is for goals, like "kill player 1", which includes the middle level commands "go to player 1" and "attack player 1". Another example is a survivor behind a door. The "kill player 1" goal might fail, because the player is behind a door. The zombies would eventually learn that when the "kill player 1" goal fails (because "go to player 1" fails), they need to do something else, for example adding "break door" to the front of the goal ("break door" -> "go to player 1" -> "kill player 1").

 

Is that about right?

 

Would they do better if the system doesn't evolve and the initial percentages are determined in closed simulations? The only negative thing I could think of is that this assumes that zombies are relatively permanent/never despawn, which might not match the server restart pattern that's currently here, and if the servers are so put out by a few hundred zombies how are they going to get a system like this in?

 

 

That is basically whats going however it is not learning simply adapting through randomisation.  In other words it does not know which action should be chosen for the particular function that is being mutated and whether it will work or not.

It randomly chooses one and then that mutated queue is available for selection next time the function runs, which could be hours/days away. 

 

Server restarts would hopefully be reduced later on in development so that servers run for many days or even weeks.  However this could either be saved to the hive or saved to disk so that a single server/installation continues to adapt over many restarts

[Steak and Potatoes 13480]

 

Sorry about that everyone, again.    I was oncall and would you believe it got a call. So Im posting a reply instead of updating.

 

 

I did not create this thread to just ask the devs to add adaptive AI into the game (which I hope they do whichever established or personal route they take) but to adapt this new AI concept I have developed in particular into the game.

 

this could get confusing but its futile to explain in depth everything behind this now without diagrams to assist. I advise to read slowly

 

This artificial intelligence works on a 3 layer structure designed to maximize complexity while minimising processing time.  The Sony Aibo robot I own was released 13 years ago and contains a risc processor that runs at 50mhz!  In other words very little processing power is required to implement this particular concept into a game and favours having a "dumb" AI already in place to directly control the zombies.  I favour the modular approach to coding and having looked into ARMA modding before the coding is similar (I am using that loosely) to OPEN-R for the Aibo.  In other words Im confident it will work in DayZ if they chose to implement it.

 

Firstly before I even get into the AI itself I need to cover the adaptive part so that you understand how it fits together.

Each level of the AI has functions that either control the ai level below them, or the current AI in the game for the lowest level.

Each function has a probability rate, that decides how likely it will be "randomly" chosen to be in a queue for its level.

Each function has requirements that dictate success or failure. This is used to alter the functions probability rate up or down accordingly.

For example a function for moving towards a target entity would have the condition that the group became closer to the target entity to be successful.

If a function needs a target but the ai has none, that function would fail without being executed. Most functions would have some value/limit or condition that would cause them to immediately fail, but not all have to.

Each function controls the below AI by having one or more preset queue's of lower level function(s) that will successfully complete this functions requirements. (that's a mouthful)

Each queue has a success/failure ratio stored and modified when the chosen queue's last lower level function succeeds or fails. Preset queues start with 100% success rate as they are set by the programmer.

Finally each function effects any number of the monitored statistics for succeeding and/or failing.

 

If the AI is particularly "content" (doing whatever it is designed to be doing in other words) it has a very small chance of creating a "mutated" queue in the function that just completed. Mutation is simply copying the queue and randomly replacing one lower level function in the queue with another randomly chosen one. That's where the probability rate comes in.  all functions have a chance of being chosen, regardless of probability rate, but it is more likely to choose a function with a high probability rate than one with a low probability rate. 

The new queue has success/failure rate set to 50%.

Most of the mutations will be abysmal failures and quickly the ratio will drop extremely low. 

So if there are more mutated queues than a preset number, when the function that queue belongs to fails the queue is checked to see if it has the lowest ratio among mutated queues and is removed if it does.

Conditions for a function to even start are key to failing a queue before taking any action and waisting ai time but they are not required.  There is more chance of seeing the AI do seemingly idiotic and completely unrelated actions to a situation if requirements are not set.  This can lead to some interesting and probably hilarious outcomes however so like I said not all functions need requirements.

 

 

Levels of the AI

 

Top Level - Emotional/Physical state reactive ai

This AI monitors and alters things like desires (how badly the zombies want brains for example), excitement (blood or death in nearby), stamina (if they have it), company ( used to help grouping or mega hordes), being attacked by players and how many etc. 

Its an averaging of the units under its control which would preferably be a single horde but could be all at once.

It would react off already in-place functions that return states of zombies or positions and other data already provided by the ARMA engine.

Finally this AI also controls the single goal (top level function) of the group. This might be "move to city" "move to forest" or of course "find flesh to eat".

Once a goal is started it remains until the middle level AI has completed its queue, or an event occurs that causes a new goal (such as group members being killed or one finds a player)

 

Middle Level AI - Activities

This level of AI takes the queue chosen by the top level AI and evaluates the functions in turn to see if they are valid (required target type exists, not too full, not too hurt etc.) and if they are passes each function in the queue in turn to the lower level AI for execution.

It waits for the low level AI to return that it has either succeeded or failed before moving on the to next function in the queue. If the low level function fails it immediately reports to the top level AI that it failed and does not execute further functions from the queue.

Middle Level Functions would include things like "move towards target"  "circle target" "wonder around" "eat target" "group together" etc

 

Low Level AI - Actions

Things are most complex here for the AI.  The functions that are passed to this AI will directly control what the zombies do in the game world.

This AI's purpose is to evaluate the function passed by the middle level to see if it can be executed (like before) and then execute it.

Functions at this level would be things like "walk to position" "crawl to position" "run to position" "stand attack" "crouch grab" "push" all the individual actions a zombie is capable of doing.  The low level function contains the code that uses the top level AI's current state to decided which zombies under its control are ordered to do what.  They are almost exclusively timer based that gives the AI in game its finite order then waits, evaluates and then reorders until the conditions for success or failure are met.

So for example moving towards a city or forest would likely want the whole horde moving, but attacking the player may only want to initially attack with the closest few zombies.

 

 

To see it as a whole think of it like an organisation chart.

At the top you have Top Level Ai functions or Goals.

Below them are all the Middle Level Functions or Activities.

Finally across the bottom are the Low Level Functions or Actions.

 

As the AI runs, it begins by following the preset queues created for it.  These will be generally successful as they are designed by the programmer to work. As time passes and the AI completes actions and activities and goals the probability rates of the various functions fluctuate according to what is happening in the world, including player interaction (a valid activity could easily fail thanks to a player killing some of the zombies in the group for example). This will change which functions are chosen by the AI.  The ever changing inputs of a live server gameworld and player interactions would also help to allow mutated queues to succeed that would otherwise fail in a test. Mutation and Evaluation of these queues will create simple adaptive emergent behaviour from the AI.

 

The speed at which new behaviours emerge and are tested is very slow, and could require hundreds of hours for non preset actions to start actually occurring (at the choice of the programmer of course, fast learning is possible with stricter conditions on functions)

 

 

Finally before posting this mammoth thing I will give an example

 

There is a zombie horde in a lucky position where regularly bambies come along and get eaten.  This keeps the AI content as they are eating regularly and not getting hurt (silly bambi's).  Every time a player comes near or attacks a zombie with an axe/weapon the goal is switched from whatever it is now to attacking back and trying to eat them.  As this happens over and over (silly silly bambi's) a mutated queue is created for the attack player functions (hey its happy) which has the AI run to the forest before the attacking the player. Due to their current world position this works, and is valid and succeeds multiple times.

Later on other players who are facing a horde may see that instead of heading towards them as normal, this horde reaches them by heading towards the forest and then towards them.  The more it succeeds the more it happens.

On the flipside if you are fortified somewhere and a huge hordes comes down on you but you hold them off, this will cause the functions in use to fail, lowering the chance they will try that again either now or later on.

 

 

It is entirely posssible that this entire thing only makes sense to me, and Im sorry if thats true LOL

 

 

You sir, have earned my beans :beans: love the idea.