It's been nearly four weeks since the launch of The Kraken Wakes: an interactive adaptation of John Wyndham's much-loved novel. In the game, you play as a journalist, interviewing, conversing and debating with the characters, typing or speaking your replies. There is no multiple choice. And our worst-kept secret? We can see what you write.
Our (anonymized!) analytics are available instantly, and over the last few weeks, we've had a lot to read through. Node based data shows us exactly what players are replying to specific interactions, and end-to-end transcripts show us a player’s experience across the whole game. Time stamps mean we can see how long players spend in particular places, or with interactable objects.
Per-node analytics listing what players have said to advance from a node asking “How has your day been?”
Transcript: Script-like playthroughs of the player and character dialogue, including action nodes attached to interactables
We can also see how many hits certain paths get, highlighting which are popular, or under-explored, and when choices are balanced.
Playthrough hits: Numbers on connectors show us how many times a player has taken that route
Combined, the analytics give us a comprehensive overview of how people are playing our game. The process is a narrative designer’s dream in itself, and informs our future work, but best of all? We can implement quick-win changes immediately. We don't need to push a new build to Steam, or even open Unreal Engine. All we need to do is make the changes to the nodes in the Charisma graph, test, and click publish, and they will seamlessly appear in the live game.
This iterative process is a vital part of development. In the past 30 days, we have republished Kraken 14 times and continue to do so. Many of the changes are small refinements, but some are more substantial. For example, as of last week, we have very carefully deployed a small amount of generated character dialogue into the live game using a Large Language Model (LLM). The analytics have been crucial in telling us where and how to strategically deploy this technique, and it’s already proving to be a meaningful improvement to player experience.
We have long been exploring the effects of AI generated content in storytelling. In 2021 we collaborated on and wrote a white paper detailing the concept of using LLMs to create a Universal Character Model . This was followed up in 2022 with an outline of our group's first steps using LLMs to generate consistent character dialogue. We're aware of possibilities that these technologies offer us. We're also aware of its downfalls. And so, The Kraken Wakes only utilizes these tools when the analytics have shown we really need it.
When people play Charisma stories, they want to find the edges of the experience. They try to push the characters to say odd (that’s putting it lightly) things. We prepare for this, front-loading our early interactions with as much content as possible. Eventually, if people trust that the system really does work, they cool off. Admittedly, the Kraken analytics revealed that this cooling-off period is longer and more forceful than we expected, and we needed to find ways to handle it. We’ve deployed a few carefully positioned, and strictly author influenced generated lines of dialogue, which are handling these challenges really well. Our intention here is not to let the player write their own story with the characters. Our intention is to keep the ball afloat, to protect the suspension of disbelief, and to keep the player within the realms of the story we are telling.
The Kraken Wakes is not a dynamically generated story. Character dialogue is written by us (there are 10,846 lines of author-written dialogue!), and any generated dialogue is heavily influenced by author intent:
The above is an example of the player “testing” the system. Our NLP matching system has not found an author-written response that suits (shocker!), despite there being over 30 at this interaction point. Instead, it opts to generate dialogue for Freddy’s response. Here the prompt is specifically told to link back to literary genres, keeping the conversation on track.
By using LLMs in this very targeted way, we can see the generated dialogue pick up on a Freddy speech quirk (”Ah”), reference her personal preferences (does not like war stories) and involve the story context (”popular in the current climate”). Our analytics give us visibility of every line that has been generated, and what prompt it used to get there. This means that these too will go through a process of refinement.
Our analytics have been the guiding light to post-launch edits. We hope you've enjoyed playing The Kraken Wakes, and we can't wait to see what you’ll say next!