New daily beta! https://wiki.arcengames.com/index.php?title=AI_War_2:Finalizing_Multiplayer#Beta_3.505_Long-Term-Thread_Efficiency
1. Okay, there has been so much code review today, and for the longest time I didn't really find all that many things of concern. Yes I was able to make some improvements to correctness after pooling in some areas, but it was nothing that would explain the sorts of errors that Zweihand and Setsuna were reporting seeing intermittently. Which was a surprise, because I figured it would be in the external data for factions.
2. After working on verifying that was all okay and fixing up the parts that were not, Badger and I got to talking about the Zenith Trader case in particular. The Zenith Trader uses a data structure for its movement that most of the newer factions didn't use. That data structure was basically a copy of parts the main game data, for purposes of use on the "long term planning" AI/NPC threads. I had mentally flagged that category of data as being potentially problematic when I was making things pooled a bit ago, but a problem with it wasn't obvious.
3. Rather than start testing and debugging that long-term data copy, I decided to strip it entirely out instead. This isn't used by more recent factions for a reason. Essentially, as I've further developed my thinking on ways to speed up multi-threaded simulations (once you take perfect determinism off the table), it was clear that the long-term-data copy was a relic of the old deterministic way of thinking. This had been on my radar for at least a year as something I wished I could kill, but the game was working fine and so there was no sense rocking the boat. Well... now the game was NOT working fine, so it was the perfect time to strip that out.
So what was the result? Well... it's hard to say. Those behaviors that were observed were already really intermittent. This may have been the part of the code that was causing them, or this may have been utterly unrelated to the bug I was ostensibly chasing. But I do know one thing: this data and the way it was generated was one of the biggest slowdowns for the game, on the main sim-coordinating thread. I have wondered for years where the "performance leak as planet counts climb" was, and I think that as I was stripping this stuff out I got my answer. The amount of time spent calculating this stuff was staggering, and also completely unneeded. It's an excellent showcase of why my newer method for multithreading is so superior in the first place.
I'm running on a really beefy CPU these days, so I don't notice a huge difference since I already had full sim speed, but it does seem a little snappier to load the savegames at the very least, or maybe that's my imagination. I'll be curious to hear if there's a noticeable speed boost for anyone playing a more extreme game setup, or with a lower-spec machine.
----
This did, naturally, break all the code mods again. The Arcen-maintained ones and AMU are updated, but I didn't give SirLimbo any warning or any time to update his actual faction mods. Hey, this is a beta after all.
There is some more data along the lines of this long-term data (it's data for short-range planning, in this case), and I may look into stripping that out as well, thus breaking all mods AGAIN.
My final ideas for some of the problems seen in multiplayer are also something I plan on implementing soon, and those are going to require reworking all the notifications and intel tab entries and a lot of unit tooltip addenders, which faction mods also tend to have in abundance. It's a bit of a rough time to be modding via code here!
But the result is going to be a leaner simulation for singleplayer, and a vastly more performant and accurate situation in multiplayer, too. I'm not thrilled with how this is hitting my schedule, but hopefully some of these simplifications also remove the bugs that are lingering at the moment.
More to come soon.
Enjoy!