Icarus Week Ninety One Update | Fixes for hitching, stutters + new wind turbines

This weeks update we release the work we have been doing towards performance improvements. We have been working on these improvements for some time, and they represent a major change in how a number of assets and parts of the map are loaded. Most specifically these are targeted to address hitching and stutters caused by level streaming and asset loading when moving across the ma, but in many cases they also give modest general FPS improvements as well.

We’ve done a big write up to explain the changes we’ve made and our logic for it also, to give you an idea of the background work that’s going in to improving the experience for all players as we add more content to the game in the future.

This week also brings the long awaited wind power system, with new wind turbines giving you a third sustainable power source, and a sneak peek at our next improvement to the electricity system, batteries.

We’ve also got an update on Hypatia, our next big free update, and even a look at what is coming next week to Icarus.

Jump in and have a read.

Important Fixes

• Fixes to outposts so they all spawn creatures as intended
• Fixes to voxel mining audio which was causing a machinegun like sound effect that would not stop
• Fix for Deployable Destructible not being cleaned up unless the host / server restarted
• Fixed an issue where Weather effects were played inside caves
• Fixed issues where the Weather effects would appear incorrectly when mounted
• Fixed issues where Ice Borers could not be snapped and deployed
• Fixed issue where minerals and ores would sometimes revert to un-mineable grey chunks
• Frozen ore is now scaled based on ore gained not chance per hit, providing more overall frozen ore
• Fixed issues on Prometheus where Unstuck would place players outside of the map bounds
• Fixed issue where you couldn't upgrade buildings to Scoria Brick

Major Performance Improvements

This weeks update has a number of performance improvements that we’ve been testing out in our experimental build for the past few days. These are focused around reducing stutters and hitches caused by ‘level streaming’ (which is when a map tile is loaded into the world as you move into the area). The key areas include:

Improved load times for a number of in-game objects such as voxels

Changing ‘navmesh’ loading to a background thread

Changing the garbage collector to only run whenever important map tiles are streamed in our out

There’s a couple of core concepts that are tricky to understand, which are important to get the full picture of our improvements this week.

Threading:

Threads are pieces of work that are split up and run on different cores in your CPU at the same time.

Think of of this like queues at a bank. A single threaded game would be like having a single queue for a single bank teller (who represents a CPU core). In this scenario, only one customer can be served at a time (or one piece of work completed in-game on the CPU):

Multi-threading would be having multiple tellers and multiple queues, allowing for you to get through all of the people in the queue faster:

In theory, this means everything would run faster and smoother as the CPU cores could process more work, faster and more efficiently. But in reality, there is a problem on the other side of the bank counter. In order for the teller to process the customers job (the CPU to process the job) they have to go to the banks one and only ledger containing all the customers information (your RAM) in order to read and write the customers data.

In a single teller scenario this is perfectly fine, but once there are multiple tellers trying to do this at the same time, it’s chaos; numbers get overwritten, delays happen, and your bank crashes with weird error messages:

So what’s the solution to this? All modern game engines use a single “main thread” (also known as a “game thread”) to do most of the work. This puts a lot of strain on your CPU as it’s only able to use a single core for this work, resulting in your CPU utilisation showing something like 15% when the game is running poorly. Some work is still done multi-threaded, but it is a case by case basis as not everything is even suitable for it.

To return to the bank analogy, imagine the single queue, single teller setup, but this time we have a special second teller whose job it is to process forms.

1. The customers who need a form processed will join the main queue until they get to main teller.
2. The main teller then sees that the customer is trying to get a form processed. The teller then gets all the information from the ledger that the form needs and attaches it, then sends the customer over to the special teller for that form to get processed.
3. The main teller then continues processing the main queue while the other customer is getting their form processed at the special teller. The special teller doesn’t need to use the ledger because all of the relevant information to get that form processed has been attached, removing the issues from the original multi-threading scenario.
4. Once the form has been processed the customer rejoins the main queue to hand the completed form to the main teller who puts the results into the ledger.

This is an example of using ‘worker threads’ to offload work from the main thread.

The main thread has to prepare the data that the worker thread needs to do its work, and when the worker thread is done ,the main thread has to apply the results back to the game.

This is used in a few places in Icarus, PhysX runs on a thread, as do things like voxel updates when you mine them. The important thing to know here is that threading is a tool can solve some problems but not all.

Time Slicing

Another such tool is “Time Slicing”. Time slicing is used to process a little bit of data each frame up to a certain time limit or number of items.

This smooths out a big spike of work over multiple frames, removing hitches at the cost of a slightly higher frame times while the work is going on.

This method is good for processing things that don’t need the results immediately and aren’t suitable to be threaded because they need access to the game data.

Start up times

When a map tile finishes streaming in, all of the objects on that tile need to start up on the same frame.

Icarus has both a lot of objects on its map tiles, and specifically objects that take a long time to start up, the combination of which led to a significant 'hitch'. While there isn’t a good way to reduce the number of objects on the tiles, a lot of the objects have now had their start up times shortened or spread out over multiple frames.

One major example of this is the voxels. There are roughly 800 to 1000 voxels per map tile, so everything they do during startup is magnified massively.

Voxels now have a lot of their setup done in the assets themselves instead of during their startup scripts.

'Loading their resource type' and 'registering with the shelter subsystem' have both been time sliced over multiple frames. Loading the voxel save data has been given a fast path that can quickly find the matching data without the slow search through all of save data.

In the test scenario these changes reduced the tile start up hitch from about 100ms to 18ms.

Navmesh loading

Navmesh (Navigation Mesh) is used by the AI to know where they can and can’t go.

Icarus being an open world game has a lot of navmesh to cover that surface area, as well as having to update that mesh when buildings are built on top of it. A previous engine update now allows us to use async navmesh loading, which is carried out on another thread. This completely removes navmesh loading hitches.

Garbage Collector

When a map tile streams in or out the engine forces a garbage collection.

This is to ensure that all unused objects get removed from memory and keeps the RAM usage as low as it can be. While this is good for reducing memory usage spikes, it takes a long time to complete a collection.

Garbage collection was happening even for distant tiles changing their LoD’s, resulting in a lot more than expected. We’ve disabled the default garbage collection during level streaming in favour of only doing them when streaming out the main level tiles. This massively reduces the amount of hitching during regular gameplay at a very slight memory usage increase from time to time.

Future work

While these changes have a big impact on the number of streaming hitches there are still some remaining, such as removing objects from a tile when it streams out, and high density foliage loading in (on both CPU and GPU). The remaining hitches are currently being worked on and we will have a patch addressing those in future.

New Content: Wind Power

Wind Power is finally in Icarus.

We’ve added the long-awaited Wind Turbine this week, giving you another way to generate electricity to power your Tier 3 and 4 deployables.

The Turbine requires a fair amount of space around the propellor, so these need to be spread out to be activated. When placing these or hovering over them, you’ll be able to see the range required to be free of any obstructions. The range required around these objects can be seen when highlighting over them (this will not be shown if the ‘Show Item Highlights’ setting is off).

The Wind Turbine is a Tier 4 item, crafted at the new Fabricator. Unlike solar panels, wind turbines will work around the clock, and don’t require cleaning such as water wheels do. This makes them the most efficient and easiest system to deploy and use, which is balanced by their lower power output and cost.

Batteries

With Wind Turbines being our third sustainable power source added to Icarus, providing a storage source for power is something we are actively working towards.

The initial implementation of resource networks doesn’t allow for batteries in the traditional sense. The way we developed these systems, is with machines registering themselves on the network and controlling their own on/off state, rather than it processing within the network itself.

To add batteries, we’ll need to modify this system to allow for them to exist and function as intended. This will allow for proper resource flow control, allowing for brownouts and variable energy flow rates.

Once these changes to the network have been made, the priority will be getting batteries added, so keep an eye out for these in the future.

Coming Soon: Hypatia

Now that New Frontiers has been released, our core development focus is shifting to our next major update, Hypatia.

We have a few more bugs that have been discovered in New Frontiers and other areas of the game that we will be working on, but the majority of the team's effort now moves towards this upcoming milestone.

Because of the massive hit Operations have been in New Frontiers (for those still unclear on what these are, this is the new system that allows for full missions to be triggerable from Open World), we have decided to focus our efforts fully on the Open World Operations in Olympus and Styx, rather than wait for the Biolab feature to be completed, which is still in the early stages of development.

This means that Hypatia with the Operations added to Styx and Olympus can be brought to you sooner, and our timelines will adjust to incorporate the Biolab into a future update.

At the same time, expect our regular, weekly updates with bug fixes, new content and quality of life to continue as they always have.

Next Week: New Mount

Speaking of new content, next week will be bringing some new friends to adventure with. New Mounts are coming to Olympus, Styx, and Prometheus as our free Week 92 update.

Support our Project

If you like what we’re doing with Icarus, and want to support our continued development, consider purchasing one of our DLCs for a few dollars, it would mean a lot to us.

https://store.steampowered.com/app/1648532/Icarus_New_Frontiers/

https://store.steampowered.com/bundle/33813/Icarus_Complete_the_Set/

https://store.steampowered.com/bundle/34141/Icarus_Outposts_Bundle/

Changelog v2.0.1.115492

New Content

Fixed