As we get closer to the end of a project like Mask of the Rose, we shift our focus from making new things to improving the quality of what’s already present. Some of that takes the form of subjective polish; for example, tweaking a character’s art a little to make them more expressive (or dreamy 😉). On the writing side, we might make changes such as altering the pacing of the game in response to a playtest.
However, a large (perhaps larger, by time spent) aspect of quality is less subjective; bugs! QA is particularly important when it comes to getting to grips with this latter category. During a project we test work continuously, typically performing testing of each new feature or chunk of writing, along with periodic regression testing (playing through the game to check if anything has broken). Normally some of the issues detected during development can be deferred (i.e. don’t have to be solved immediately) because they are dependent on other features that will be done later or because some functionality is intended as a placeholder. We still record everything, because we don’t want bugs to slip through the net; but later on in the project is when programmers tend to focus more and more on eliminating bugs, and spend less time on new features.
In Mask of the Rose, for example, all of our planned feature programming is complete. For example; all the UIs are in place; the game save system is implemented; and an audio system manages sounds and music.
That does not mean the work is over! For the last couple of months our core Mask programmer, Séamus has been working closely with Lesleyann, our Principal QA Tester, to tackle bugs and improve performance. In fact, their work has been centred on a very specific purpose; Mask is the first Failbetter game that will launch simultaneously on PC/Mac and console (Switch).
It’s been important for us from early on in this project to always maintain a working build of the game for the Switch. There are a few reasons for this.
The Switch, being effectively a mobile platform, acts as a hardware baseline; even the more modestly specced PCs and Macs that Mask targets have more usable memory, hard drive capacity and CPU power than the Switch. This next point is oversimplified, but in essence; if we can get Mask running well on Switch, Mask will also run well on lower-end desktops and laptops. Therefore it “keeps us honest” about how complex features like graphical elements can be.
It also keeps us honest with regard to gamepad support. Historically, Failbetter titles have been primarily developed for mouse and keyboard control, with gamepad support either added later or not given the same level of focus. In Sunless Skies, we weren’t happy with how gamepad support turned out in the initial PC release, so when we set out to publish our own console ports (also a first for us) we were keen to take the improvements from console and make sure they made it back to PC and Mac in the Sovereign Edition. However, because we hadn’t given gamepads equal weighting from the start of Sunless Skies, there were technical compromises we did have to make to give gamepad and console players a better experience. In Mask, maintaining a build on Switch forces us to focus on the quality of both the gamepad and the M+KB experience.
A final reason to build on Switch early is because shipping a game on console is a unique challenge. Platform holders have extensive lists of technical requirements that games must fulfil. For example, most consoles limit the frequency to which we can write to the storage memory, to prevent physical wear to the drives. Another requirement might be to only use specific, authorised terminology to refer to the console’s controllers.
Before we can release the game we must check that it is compliant with these requirements, fix any issues, test everything, and then submit builds to the platform holder to perform their own testing. That means the release process on console must start earlier than on PC, and this is what Les and Séamus have been focusing on recently; getting to the point we can submit a build to Nintendo that gets the famous “Seal of Quality” from them!
Our other area of increased QA focus is writing. Here, also, Mask presents new challenges. In previous Failbetter titles, writing was very modular. In Sunless Skies, for example, a story can be tested as a stand-alone unit. This is because the player has lots of discrete, self-contained experiences as they visit different ports across the High-Wilderness. Stories tend to only act on each other indirectly; such as via the resource economy, or the completion state of an earlier story. It is also quite unlikely for a writer to accidentally trap the player in a dead end. Because of these properties, we know that bugs introduced by adding a new story will normally be confined to that one story, and the states of the playthrough that could affect the story are less numerous, making testing simpler.
But if Sunless Skies is like a short story collection, then Mask of the Rose is closer to a novel; every character and storyline is interwoven. Mask is also the first time we have used the Ink scripting language instead of our in-house content management system. The Ink script in Mask is fiendishly complex, reactive and programmatic; this is the most player-responsive title we’ve ever made.
Furthermore, with no way to “escape” faulty content (think about how you can simply leave a port in Skies if a desired option is unavailable), we have to work hard to spot the type of critical writing bug that can cause the game to “dead end”.
This has meant that we’ve had to rethink how we perform narrative testing, and are deploying automated narrative testing for the first time. Séamus has developed a tool that allows the writer to run a piece of content thousands of times outside the game engine before it’s integrated. This acts as a first line of defence against the dreaded dead ends, because we can tell if one of our randomised, automated runs gets stuck on a particular version of the script.
A graph illustrating the player’s routes through Mask of the Rose (simplified)
This approach isn’t bulletproof, because there can be differences between how the testing tool behaves and how the game engine behaves; there are often serious bugs at the interface between tech and writing. A purely stochastic tool will also make arbitrary decisions, which result in many runs that represent an atypical player experience. To bolster our approach we’re also performing extensive manual, in-game testing, and are working on a few ways to extend the capabilities of our autotest tool; for example, to allow writers to specify different “player profiles” that make the computer player focus on certain human-like behaviours such as pursuing a particular romance option.
Every game comes with its own set of challenges for the developers, not least in QA. However it’s easy to overlook this discipline from the outside, because QA work is invisible until something goes wrong. I hope this blog gave you a little peek under the mask! Thank you for coming on this journey with us as we extend the capabilities of our studio to tell a very different tale to the stories we’ve told before.