Our current project Infinite Fleet is an MMO strategy game, in which there will be a huge number of units, including things like spaceships, turrets, bullets, etc. So, we decided to use ECS in our project to avoid performance bottlenecks and reach our ambition.
Though what is the current state-of-the-art in terms of AI compatible with the lastest Unity DOTS? Are the common Artificial Intelligence (AI) plugins and techniques we use with the legacy Unity framework also available for Unity ECS and DOTS?
In that article, I reintroduce to you a brief history of the most common game AI techniques used and then share to you the solutions that appear the most viable to me to use with Unity DOTS.
Let’s start digging into the history.
Common game AI techniques history
In the beginning, we had spaghetti hard-coded If/Else AI
This is the most straight-forward thinking solution and below there is a piece of code that illustrates the idea.
if (IsOnGround())
{
if (IsHungry()) DoEat();
else if (IsTired()) DoSleep();
// else if ...
}
else if (IsInAir())
{
if (IsHungry()) DoHunting();
else if (IsTired()) DoLanding();
// else if ...
}
// else if ...
With hundreds and hundreds of statements, it will quickly start to eat your brain…
Then, we had Finite State Machines (FSM)
An FSM is a directed graph that represents states of a system and its possible transitions between states. It will continue to execute its active state until an event happens (or some conditions are met) and then the active state will transition to another state.
Typical Finite State Machine.
Here:
- S stands for state,
- and E stands for event that will trigger a change of state.
To reduce complexity we can also separate certain behaviors into substates, using then what is called Hierarchical Finite State Machines.
Typical Hierarchical Finite State Machine.
Being able to transition from any state to any other state by specifying conditions (events), makes it very easy to design FSMs for AI behavior. But it becomes unmanageable as they grow in size and complexity. Damian Isla outlined this in detail during his GDC talk from 2005 discussing the AI of Halo 2: Link.
Then, Behavior Trees (BT) were born
BT are trees meant to describe complex behaviors and make decision on the behavior to adopt. At each tick, the whole tree is evaluated, and if another behavior (action) is chosen other than the last invoked, then the execution state will change, otherwise, the current behavior continues to be executed.
Typical Behavior Tree.
Starting with a root node, depth-first traverses the tree and executes the action node.
- Blue: composite node, parent node to control the workflow of child nodes, generally either execute children in a sequence or in a fallback manner (named selector).
- Green: conditional and decorator node, help to control the workflow.
- Red: action node, the action to be executed.
But like the Hierarchical FSM, we can have hierarchy using subtrees in a BT.
Complex Behavior Tree.
BT in many cases provides a framework for designing more comprehensible and easier-to-read AIs than Hierarchical FSM.
But for very large BT, the costs of evaluating the whole tree can be prohibitive. Consequently, the idea of subtrees has been introduced. The idea is that the subtree can continue executing without invoking the whole tree until some condition is met to exit the subtree. However, this reintroduces the same problems as with FSM.
Nowadays, Utility AI (or Goal Oriented Action Planning) gives us new possibilities
It works by identifying options available to the AI and selecting the best option by scoring each option based on the circumstances.
AI that rise from it will have emergent behaviors, it can make reasonable decisions in situations that the game designer did not necessarily foresee, and it can express a large range of behaviors similar to what would be expected by a human player.
Example of utility curves.
Curves can be assigned to options to make it an overall better decision-making method.
Utility AI is simple to design, easily extendable, and better quality. The ability to make smart decisions requires tweaking the scorers to ensure correct behavior, and this requires developer-time and game testing. Alternative technologies to avoid these limitations are in the direction of machine learning.
More detail can be found on this Gamasutra article Are Behavior Trees a Thing of the Past
AI For Unity DOTS
We can obviously reuse If/Else AI in Unity DOTS, but I do not recommend it anyhow. So below let me introduce a solution for each of the left cases: FSM, BT, and Utility AI.
Finite State Machines in Unity DOTS
When we implement FSM in traditional Object Oriented Design (OOD) used with MonoBehaviours, normally we first define some states and then use the FSM to do the transition between the states.
The transitions depend on the logic of each state (for example, a function like ShoudExit()).
As long as there is no transition triggered, we just keep executing the update loop of the active state (in OnUpdate() function for example).
A simple FSM framework in OOD.
But in Data Oriented Design (DOD) with ECS, all logic should be handled in systems. Which means we should have a system per state (StateASystem, StateBSystem, …) and one for the FSM itself (FiniteStateMachingSystem).
And the data of each state and FSM should be stored in components: one per state (StateAComponent, StateBComponent, …) and one for the FSM (FsmComponent). We also need a component to indicate a transition of state: FsmStateChangedComponent, which has two fields indicating the transition was from which state to which state.
Then the structure should look like:
A simple FSM framework in DOD.
Alright, this was a simple way to convert FSM from traditional OOD/OOP to DOD/ECS. I uploaded a demo with a simple FSM implementation in both MonoBehaviour (OOP) and DOTS (ECS) to ECS_FSM. Feel free to have a look.
Also as a reference, another FSM framework can be found here. It’s implemented using Entitas. It has more complicated systems and a unit-test system to let you start your test. Take a look if you are interested.
Behavior Trees in Unity DOTS
There is an implementation for Behavior Trees that can be found at https://github.com/quabug/EntitiesBT.
This is the solution I would recommend because it is based on DOTS and is a good example of using Behavior Tree in ECS.
Utility AI in Unity DOTS
I didn’t find any implementation for Utility AI in ECS, so I’d like to try to implement it myself, using DOTS.
Please note that here I only want to show the concept. There will not be any Editor support features.
In my opinion, Utility AI is about calculating a score for each possible action, then perform the action with the highest score.
We can consider an action scoring system, probably one job for one action, and find the highest score after all jobs finished. Then we flag that action to be executed.
To execute the action, we can think about that one action has a corresponding system to perform it. Once an action got an execution flag, its corresponding system will handle it.
For example: assume we have a cat entity that will:
- Eat when hungry,
- Go to sleep when tired,
- Play when not too hungry or tired. And while playing, it becomes hungry and tired.
So first we can define the actions and their score component:
enum ActionType
{
Null,
Eat,
Sleep,
Play,
}
struct EatAction: IComponentData
{
public float hungerRecoverPerSecond;
}
struct SleepAction: IComponentData
{
public float tirednessRecoverPerSecond;
}
struct PlayAction: IComponentData
{
public float hungerCostPerSecond;
public float tirednessCostPerSecond;
}
struct EatScore: IComponentData
{
public float score;
}
struct SleepScore: IComponentData
{
public float score;
}
struct PlayScore: IComponentData
{
public float score;
}
Maybe also have a component to flag our cat:
struct Cat: IComponentData
{
// This component have no fields cause we just want it to be a flag
}
And also we need components to define the hungriness and tiredness, so any entity with them can feel hungry and tired. A component to indicate the current action is needed too:
struct Hungriness : IComponentData
{
public float value; // 0: not hungry, 100: hungry to death
}
struct Tiredness : IComponentData
{
public float value; // 0: not tired, 100: tired to death
}
struct Decision : IComponentData
{
public ActionType action; // current action to perform
}
So, we can specify that an entity is a cat by adding the Cat component on it:
Entity catEntity = EntityManager.CreateEntity(new ComponentType[] {
new Cat(),
new Hungriness() { value = 0f },
new Tiredness() { value = 0f },
new Decision() { action = ActionType.Null },
new EatScore(),
new SleepScore(),
new PlayScore(),
});
Once an Entity has some of these Eat / Sleep / Play actions, the corresponding system will perform the action.
So, for each action, we can define its corresponding system like:
class EatActionSystem: SystemBase
{
void OnUpdate()
{
float deltaTime = Time.DeltaTime;
Entities.ForEach((ref Hungriness hunger,
in EatAction eatAction) =>
{
hunger.value -= eatAction.hungerRecoverPerSecond * deltaTime;
}).ScheduleParallel();
}
}
class SleepActionSystem: SystemBase
{
// ...
}
class PlayActionSystem: SystemBase
{
// ...
}
To add the flag component to an entity, we need the score system to calculate the scores of the actions:
class ActionsScoreSystem: SystemBase
{
protected override void OnUpdate()
{
//>> Calculate scores
Entities.ForEach((ref EatScore eatScore,
in Hungriness hunger,
in Decision decision) =>
{
if (decision.action == ActionType.Eat)
{
// Once it starts to eat, it will not stop until it's full
eatScore.score = hunger.value <= float.Epsilon ? 0f: 1f;
}
else
{
var input = math.clamp(cat.hunger * 0.01f, 0f, 1f);
// We use exponential curve for this score
eatScore.score = ResponseCurve.Exponential(input, 2f);
}
}).ScheduleParallel();
Entities.ForEach((ref SleepScore sleepScore,
in Tiredness tired,
in Decision decision) =>
{
if (decision.action == ActionType.Sleep)
{
// Once it starts to sleep, it will not wake up until
// it have enough rest
sleepScore.score = tired.value <= float.Epsilon ? 0f: 1f;
}
else
{
var input = math.clamp(cat.tiredness * 0.01f, 0f, 1f);
// This curve used for sleep is first raises fast,
// and then raises slow
sleepScore.score = ResponseCurve.RaiseFastToSlow(input, 4);
}
}).ScheduleParallel();
// PlayScore rely on hunger and tiredness
Entities.ForEach((ref PlayScore playScore,
in Hungriness hunger,
in Tiredness tired) =>
{
// The play score has two considerations
// The cat will play when it feels neither hungry or tired
// Let's say it hates to be tired more (love to sleep),
// so the sleep consideration get more weight
// sleep weight: 0.6, eat weight: 0.4
var eatConcern =
ResponseCurve.Exponential(
math.clamp(hunger.value * 0.01f, 0f, 1f)
);
var sleepConcern =
ResponseCurve.RaiseFastToSlow(
math.clamp(tired.value * 0.01f, 0f, 1f)
);
// tiredness bothers it to play more than hunger do
// (It's a lazy cat ^_^)
var concernBothersPlaying = sleepConcern * 0.6f + eatConcern * 0.4f;
playScore.score = math.clamp(1f - concernBothersPlaying, 0f, 1f);
}).ScheduleParallel();
//<<
this.CompleteDependency();
}
}
We use ResponseCurve.Exponential or ResponseCurve.RaiseFastToSlow for scoring, because that we do not want the score are always linearly changed. We want to "make it an overall better decision-making method".
The curves are like:
ResponseCurve.Exponential
ResponseCurve.RaiseFastToSlow
You can change the curves to get the different scores for each action.
After getting the scores, we need a system to select the action based on the score:
public class ActionSelectionSystem: SystemBase
{
EndSimulationEntityCommandBufferSystem endSimulationEcbSystem;
protected override void OnCreate()
{
base.OnCreate();
endSimulationEcbSystem =
World.GetOrCreateSystem<EndSimulationEntityCommandBufferSystem>();
}
protected override void OnUpdate()
{
// Choose action base on the highest score
var ecb = endSimulationEcbSystem.CreateCommandBuffer().ToConcurrent();
Entities.ForEach((Entity entity,
int entityIndex,
ref Decision decision,
in EatScore eatScore,
in SleepScore sleepScore,
in PlayScore playScore) =>
{
float highestScore = 0f;
ActionType actionToDo = ActionType.Play;
if (eatScore.score > highestScore)
{
highestScore = eatScore.score;
actionToDo = ActionType.Eat;
}
if (sleepScore.score > highestScore)
{
highestScore = sleepScore.score;
actionToDo = ActionType.Sleep;
}
if (playScore.score > highestScore)
{
highestScore = playScore.score;
actionToDo = ActionType.Play;
}
// Perform the action change
if (decision.action != actionToDo)
{
decision.action = actionToDo;
switch (actionToDo)
{
case ActionType.Eat:
ecb.RemoveComponent<SleepAction>(entityIndex, entity);
ecb.RemoveComponent<PlayAction>(entityIndex, entity);
ecb.AddComponent<EatAction>(entityIndex, entity);
ecb.SetComponent(entityIndex, entity, new EatAction() {
hungerRecoverPerSecond = 5f
});
break;
case ActionType.Sleep:
ecb.RemoveComponent<EatAction>(entityIndex, entity);
ecb.RemoveComponent<PlayAction>(entityIndex, entity);
ecb.AddComponent<SleepAction>(entityIndex, entity);
ecb.SetComponent(entityIndex, entity, new SleepAction() {
tirednessRecoverPerSecond = 3f
});
break;
case ActionType.Play:
ecb.RemoveComponent<EatAction>(entityIndex, entity);
ecb.RemoveComponent<SleepAction>(entityIndex, entity);
ecb.AddComponent<PlayAction>(entityIndex, entity);
ecb.SetComponent(entityIndex, entity, new PlayAction() {
hungerCostPerSecond = 2f,
tirednessCostPerSecond = 4f
});
break;
}
}
}).ScheduleParallel();
endSimulationEcbSystem.AddJobHandleForProducer(this.Dependency);
}
}
Each time we change the action, we just need to remove other action component and add the selected action component to the entity. Then the corresponding system such as EatActionSystem will do their jobs.
Let’s check it out:
Cat start from playing.
Cat then sleeps when it feels tired.
Cat starts to eat when it feels more hungry than tired.
Because we used ResponseCurve.RaiseFastToSlow for SleepScore, the cat tends to sleep more than eat. So you can see it switching actions between Play and Sleep several times before finally deciding to Eat.
All right, that is the basic idea about Utility AI for DOTS.
For reference, I uploaded my test project here, please feel free to test it: https://github.com/kaminaritukane/ECS_UtilityAI.git.