I'm now working on the AI for the upcoming diplomacy features. I've already implemented the diplomacy systems themselves: what war and peace mean, the ability to negotiate agreements and send ultimatums, and so on. But of course AI empires need to be able to interact with these systems: responding to players' diplomatic offers and making their own, and also conducting diplomacy between themselves.
Aims
The diplomacy system in Airships has a number of design goals mostly informed by my frustrations with the shortcomings of diplomacy in other games, in particularly the Civilization series and Paradox grand strategy games. (Which, to be clear, I love and have played hundreds of hours of.) In no particular order:
I want AI behaviour to make sense to the player. I want them to be able to understand why the AI makes the diplomatic decisions it does. Without this, diplomacy becomes frustrating as AIs will declare war or make peace seemingly at random. There may be a sophisticated system behind those decisions, but if the player cannot understand it, it might as well be random.
I want the diplomacy system to allow for deals between empires that consist of multiple parts. Modern strategy games generally all have this, but older Civ games do not. For example, you want to be able to sweeten a non-aggression pact by including a good trade deal.
Defensive pacts! All the mutual protection of an alliance with none of the being dragged into wars by the AI. My least favourite version of this is when your AI ally drags you into a war you don't want and then makes peace with the enemy again, a peace that does not include you. And you find yourself unable to make peace yourself, so now you're stuck in this war.
It should be possible for the player to ignore diplomatic niceties, making war on whomever they like. Of course, they will miss out on the safety and benefit of treaties.
Relatedly, no system of claims or war goals like in Paradox games. You can declare war on anyone at any time (though the diplomatic consequences may be dire) and when you conquer something, it's yours.
Finally, I want rich communications between humans and AIs. Of course, humans can use language to negotiate an agreement in detail and then put it through the diplomacy system, but how do you have this kind of conversation with an AI player? In part, this is helped by the first point, having understandable AI behaviour. You don't have to guess whether the AI wants a trade treaty - the game will tell you whether it does, and why.
The other thing I put in to make this work are ultimatums, which allow you to make a demand backed by a threat. The demand is an agreement that is against your victim's interests (give me tribute, dissolve this defensive pact I don't need anymore), and the threat is some unilateral action you could take that would be even worse (or I will declare war, or I will stop paying you tribute). This way you can actually communicate what you want and what you're prepared to do to get it.
AI Logic
These are nice goals for a diplomatic system, but how can I teach the AI to interact with them?
Again, there are things I want to avoid: AI players that love or hate you for no clear reason, AI that is not actually playing to win, AI whose decisions are so opaque as to feel random. For example, Civ 6 has the concept of AI player agendas, where e.g China wants to build world wonders and gets annoyed if you build them first. This is a cool idea in theory, but you're not actually going to avoid building wonders just because of that, right? And the AIs also have some (initially) hidden agendas, which in practice mean they just randomly like or dislike you.
So instead of an AI that likes or dislikes you, it should act according to strategic situation. If another empire is threatening, can the threat be reduced with a non-aggression pact? Or do we share a common enemy where an alliance would make sense? Can another empire be bullied into paying tribute?
The real-world term for this is "realpolitik" - a theory of international relations that's based on the relative power of states behaving "rationally" - that is, with little regard to fairness or past history. Your former enemy can become your friend if it makes sense for them to be so, and vice versa. If you are in a position of strength, you exploit that strength. The 19th century, on which the world of Airships is approximately based, was also the heyday of realpolitik, so it's a good fit.
And the AI knows what victory is in terms of the game and tries to win. Victory is achieved by conquering all cities, being in an alliance that controls all cities, or having your ruler crowned emperor. (The latter two victory conditions can also be toggled off.) This makes sense in-universe as the empires are attempting to achieve dominance over one another.
To figure out what it wants, the AI assigns each diplomatic situation (war, peace, trade treaty) a score and then tries to get to a higher-scoring situation. This means that if it really wants a non-aggression pact with you (+22) but doesn't especially want a trade treaty (-5), offering it both in the same deal will have it accept.
And these numbers get shown in the GUI for making diplomatic offers to AI players, so you don't have to guess.
To make diplomatic offers to human players, the AI can put together a package of terms that it likes and that it thinks you will like too. That second part is needed because otherwise the AI would start every negotiation with "and of course you should pay me tribute", because why not, it's nice for them. Instead, it will send you an offer that, while perhaps not entirely in your favour, is at least somewhat reasonable.
The diplomacy GUI also allows you to modify offers and send them back, so you can always change it to something more to your liking.
To come to agreements with other AI empires, the game looks at the relative strength of two empires and picks an agreement that they both like, but that's tilted towards the stronger partner. This is meant to simulate the result of a bunch of negotiations.
Ultimatums
Then there's ultimatums. A good ultimatum demands something that's in your favour and threatens something scary to the target that's also an acceptable outcome for you. The classic ultimatum is "pay me tribute or I declare war", where of course you'd like tribute, but you're powerful enough that you'd also be happy with war. Meanwhile, for your victim, tribute is unpleasant, but not as unpleasant as being conquered.
And this produces a pretty clear mathematical definition for what an ultimatum is. To construct an ultimatum, the AI can look through things it wants but its intended victim does not, and things it could do to its victim that the victim would really dislike. Those are the demand and threat of the ultimatum.
To determine whether the AI should cave in to an ultimatum - whether sent by a human player or an AI - we can also look at the difference between the demand and the threat scores. Let's say the demand is to pay tribute and enter a trade treaty, rated at -22. But the threat is war, rated at -60. So the demand is 38 points better than the threat. This is then directly converted into the likelihood that the AI will agree to the demand, by default by multiplying it by 2. So this particular ultimatum has a 76% chance of being accepted.
Unlike as with normal diplomatic offers, where you know right away whether the AI will accept, ultimatums have a probability of acceptance to make them a gamble. If they reject your ultimatum, you have to either follow through with the threat or lose a chunk of reputation.
So ultimatums can be sent from humans to humans, humans to AIs, and AIs to humans, and should be a pretty flavourful way of conducting diplomacy and getting what you want.
AI Personalities
Of course not every AI is exactly the same. All the numbers governing these evaluations can be tweaked (and modded), resulting in different personalities. Upstanding empires that don't break agreements or do ultimatums. Bullies that try to extract tribute from you but then back down from the threatened war. Lone wolves who simply want to be left in peace but will relentlessly pursue war when provoked. Diplomats who try to embed themselves into a web of pacts for safety. Traders who are mostly peaceful but won't pass up an opportunity to invade a sufficiently weak neighbour. Weaklings that will readily cave in to ultimatums. Overly principled empires who would rather be conquered than ever agree to anything they don't want. Even omnicidal maniacs who will pretty much ignore diplomacy and just attack at will.
And sometimes, you also want AIs that are more emotionally driven. There already is some of this in the system I'm building, because it does consider diplomatic grievances and reputation levels in its actions.
On top of that, AI personalities can have an attitude to other empires based on their past relationship. Were you at war? Have you been in a long-term stable pact? Do you own some land they consider theirs? In and of itself this isn't enough: geopolitics provide much more interesting motivations, and at the start, no one has any history with anyone else. But it can be a good additional layer for some personalities that deviate from this pure realpolitik concept.
What Remains
So yes, all this is what I've been working on. Things are going pretty well, with the AI able to make and respond to diplomatic messages. There's still a bunch of problems to work out. For example, if you successfully demand tribute through an ultimatum, the AI will immediately send you a diplomatic offer that just consists of cancelling the tribute, because after all, that would be to its advantage.
And I'm working on making the user interface for this as good as I can, showing the right information in the right places, plus generous quantities of tooltips.
I'm also still considering additional types of diplomatic interaction on top of the ones that currently exist, which are:
- War - A state of conflict. You may freely attack each others' fleets and conquer each others' cities.
- Truce - Formal cessation of hostilities. Automatically turns into a state of peace after five months.
- Peace - A neutral relationship. You may not send ships to each others' cities, and doing so is a declaration of war.
- Non-Aggression Pact - A promise not to attack each other. You can freely send ships to each others' cities.
- Defensive Pact - A promise to defend each other. If the other empire is attacked, you automatically declare war on the attacker. If they declare war, you stay neutral.
- Alliance - A full alliance sharing in all wars. Your ally's allies automatically become your allies too.
- Trade Treaty - Gain 5% of each others' incomes.
- Research Treaty - Gain 10% of each others' research output. Technologies the other has researched are 40% faster to research.
- Tribute - Receive 10% of their income.
In particular, I'm thinking of adding the following items. To be clear, there's guarantee that these will happen.
Transferring towns. This can make sense, especially as part of a truce, but it might also be abusable, so the AI's evaluation of these transfers would have to be very careful.
Monetary payment. It would be good to be able to sweeten a deal that the AI is nearly willing to accept, but not quite. On the other hand, it would be silly if every time you did a deal with the AI, you'd ask for exactly as much money as you could with the deal still being acceptable to them. Or constantly sending ultimatums to other empires demanding amounts of money so small that they're guaranteed to give them to you to avoid war. Both of these would damage suspension of disbelief and add a bunch of tedious work for optimal play. The one place where I think demanding money would make sense is as war reparations during truce negotiations, where it would be an automatically calculated amount paid out over time, a kind of temporary tribute.
Transferring ships. As with money and towns, this is tricky because these are the things that bring you victory. Towns produce money, which is used to build ships, which is used to conquer cities, which are what you need for victory. So giving away any of those three is reducing your chances at victory. On the other hand, being able to buy ships from other human players would be kind of fun.
Insults. The ability to send insults to other empires was brought up on the Discord as a flavour / role-playing thing, but in keeping with my desire to make human/AI communication as good as possible, I do think insults could have a specific diplomatic effect. Right now I'm thinking that the way it'd work is that the insulted party can either return the insult, worsening relations, or stay silent, and lose a bit of reputation for looking weak.
Diplomatic delegations - essentially the opposite of insults, delegations cost some money and are an attempt to reduce diplomatic grievances between two empires. If accepted, both parties gain some reputation. If rejected, you lose some.
Finally, if you want to read more about realpolitik in games, or are interested in history and strategy games in general, I can highly recommend ACOUP, a blog on those topics, which has for example just covered Europa Universalis IV and Victoria II from a historian's perspective.